[Cryptography] Is ASN.1 still the thing?
Ismail Kizir
ikizir at gmail.com
Sat Nov 25 08:46:21 EST 2017
>Ron Garret wrote this message on Mon, Nov 13, 2017 at 18:11 -0800:
> On Nov 13, 2017, at 6:20 AM, David Wong <davidwong.crypto at gmail.com> wrote:
> > If you want something fast (binary), but don't want the awfulness of
> > ASN.1 I believe google's protobuf is the state of the art solution. Or
> > better, you can have a fixed structure (with fixed sized fields) and I
> > believe this is what Wireguard does.
I've been following this discussion for a while.
For years, I've used every solution mentioned here:
I've used JSON, protobuf, msgpack, my own string encodings ...
For a long time, I thought that protobuf was the best solution,
especially, for multi-language projects, like mines, with C backend,
javascript frontend.
But on C side, protobuf is not elegant.
I hate C++.
15 years ago, I was writing all my projects with C++ but for debugging
complexities and readibility, I've migrated to C.
I am writing object oriented C via my own typedef's, old style, very
long, informative names and code is crystal clear for me.
The same thing is valid for protobuf.
I thought it could decrease code readability, but, for my case, it increased!
First, you must use different tools and external libraries. It adds complexity.
For simple structures, it is good.
But for nested, multi level structures it's a pain in the .ss
There is no(wasn't) good protobuf parser for C for nested/iterated structures
Original protobuf library was in C++
After, so many trials, I've decided to develop my simple protocol and
data encoding.
In the heart, there is one function, also used by protobuf:
VarInt and VarUInt encoding.
VarInt encoding is based on VarUInt encoding.
VarUInt encoding is simple:
Every byte is in range 0..127 if highest bit (128) is set, then a new
byte must follow.
The whole javascript code, is very easy to implement. I can even send
you all here:
Here is the whole code including test function, in javascript.
C counterpart is not much different.
It lacks floating point support for the moment. I don't use floating
point in my projects.
It's in public domain if you want to use.
Simple is elegant!
Regards
Ismail Kizir
var TMemBuf = function(InitialSizeOrObj)
{
if (typeof(InitialSizeOrObj) === "number")
{
this.Buf = new Uint8Array(InitialSizeOrObj);
// Our writing position on the buffer
// This is also the actual size of written data to buffer so far
this.WPos = 0;
}
else if (InitialSizeOrObj instanceof Uint8Array)
{
this.Buf = InitialSizeOrObj;
// Our writing position on the buffer
// This is also the actual size of written data to buffer so far
this.WPos = InitialSizeOrObj.length;
}
this.RPos = 0; // Our reading position on the buffer
};
TMemBuf.prototype.SetReadingPos = function(N)
{
this.RPos = N;
};
/**
* Resize resizes the buffer to hold at least EnoughForBytes
* @param EnoughFor There will be least EnoughFor bytes of space on return
* @return 0 on error, non-zero if succesful
*/
TMemBuf.prototype.Resize = function(EnoughFor)
{
var NewSize = this.Buf.length << 1;
if (NewSize < EnoughFor)
NewSize = EnoughFor + 128;
var NewBuf = new Uint8Array(NewSize);
NewBuf.set(this.Buf, 0); // Copy oldbuf to newbuf from pos 0
this.Buf = NewBuf;
};
/**
* MakeRoomFor function ensures that we can write HowMany more bytes
from current buffer position, without resizing it
* @param HowMany : HowMany more bytes from current buffer position,
without resizing it
*/
TMemBuf.prototype.MakeRoomFor = function(HowMany)
{
if (this.WPos + HowMany > this.Buf.length)
this.Resize(this.Buf.length + HowMany);
};
/**
* WriteByte writes a single byte to our buffer from current writing position
* @param B {number [0..255]}
* @returns {undefined}
*/
TMemBuf.prototype.WriteByte = function(B) {
this.MakeRoomFor(1);
this.Buf[this.WPos++] = B;
};
TMemBuf.prototype.ReadByte = function() {
return this.Buf[this.RPos++];
};
/**
* WriteBuf writes contents of Uint8Arr to our buffer from current
writing position
* @param {type} Uint8Arr Array to be written to our buffer from
current position
* @param {len} Optional length to write. If omitted, Uint8Arr.length
will be used
* @returns {undefined}
*/
TMemBuf.prototype.WriteBuf = function(Uint8Arr, Len)
{
if (Len)
{
this.MakeRoomFor(Len);
this.Buf.set(Uint8Arr.slice(0, Len), this.WPos); // Copy oldbuf to
newbuf from current writing pos
this.WPos += Len;
}
else {
this.MakeRoomFor(Uint8Arr.length);
this.Buf.set(Uint8Arr, this.WPos); // Copy oldbuf to newbuf from
current writing pos
this.WPos += Uint8Arr.length;
}
};
/**
* WriteBufAndItsLen writes :
* 1. The length of array as a 32 bit VarInt from current writing position
* 2. Contents of Uint8Arr to our buffer just next to VarInt32 encoded length
* @param {Uint8Arr} Uint8Arr Array to be written to our buffer from
current position
* @param {number} Optional number of bytes to write. If not set, or
set to 0, array length bytes will be copied
* @returns {undefined}
*/
TMemBuf.prototype.WriteBufAndItsLen = function(Uint8Arr, Len)
{
var L = (Len ? Len : Uint8Arr.length);
this.WriteVarUInt32(L);
this.WriteBuf(Uint8Arr, L);
};
TMemBuf.prototype.WriteMemBuf = function(mb)
{
this.WriteBuf(mb.GetBuf(), mb.WPos);
};
TMemBuf.prototype.WriteMemBufAndItsLen = function(mb)
{
this.WriteBufAndItsLen(mb.Buf, mb.WPos);
};
/**
* ReadBufAndItsLen reads a buffer previously written by WriteBufAndItsLen
* It returns a Uint8Array
* @param {type} mb
* @returns A new(sliced from this original array) Uint8Array containing data
*/
TMemBuf.prototype.ReadBuf = function()
{
var L = this.ReadVarUInt32(), Rv=null;
if (L && this.RPos + L <= this.WPos)
{
Rv = this.Buf.slice(this.RPos, this.RPos + L);
this.RPos += L;
}
return Rv;
};
/**
* ReadMemBuf reads a TMemBuf previously written by WriteBufAndItsLen
* @param {type} mb
* @returns A new(sliced from this original array) TMemBuf containing
data or null
*/
TMemBuf.prototype.ReadMemBuf = function()
{
var N = this.ReadBuf();
if (N)
return new TMemBuf(N);
return null;
};
TMemBuf.prototype.Clear = function() { this.WPos = 0; };
/**
* WriteVarUInt32 encodes a 32 bit unsigned integer value in a
portable variable length buffer
* The method of encoding is MOST SIGNIFICANT BIT encoding
* Don't confuse this with most significant byte encoding.
* Simply, we use only 7 bit of every byte.
* We set the most significant bit to indicate there is more data in the buffer
* @param Val Value to be Encoded. Since we use bit shifting, it may
only be a 32 bit unsigned int.
*/
TMemBuf.prototype.WriteVarUInt32 = function(Val)
{
var V = Val;
this.MakeRoomFor(6); // We can write max. 6 chars
while (V > 127)
{
// By OR'in with 128, we indicate there is more bytes to read
this.Buf[this.WPos++] = (V & 127) | 128;
V >>>= 7;
}
this.Buf[this.WPos++] = V;
};
TMemBuf.prototype.CanRead = function(NumBytes)
{
return ((this.RPos + NumBytes) <= this.WPos);
};
TMemBuf.prototype.ReadVarUInt32 = function()
{
var R=0, ShiftCount=0, B;
while (this.RPos < this.WPos)
{
B = this.Buf[this.RPos++];
R |= (B & 127) << ShiftCount;
if ((B & 128) === 0)
break;
ShiftCount += 7;
}
return R;
};
/**
* WriteVarUInt64 encodes a 64 bit unsigned integer value in a
portable variable length buffer
* The method of encoding is MOST SIGNIFICANT BIT encoding
* Don't confuse this with most significant byte encoding.
* Simply, we use only 7 bit of every byte.
* We set the most significant bit to indicate there is more data in the buffer
* @param Val Value to be Encoded.
*/
TMemBuf.prototype.WriteVarUInt64 = function(Val)
{
var V = Val;
this.MakeRoomFor(11); // We can write max. 11 chars
while (V > 127)
{
// By OR'in with 128, we indicate there is more bytes to read
this.Buf[this.WPos++] = 128 + (V % 128);
V = V / 128;
}
this.Buf[this.WPos++] = V;
};
TMemBuf.prototype.ReadVarUInt64 = function()
{
var R=0, Multiplier=1, B;
while (this.RPos < this.WPos)
{
B = this.Buf[this.RPos++];
R += (B & 127) * Multiplier;
if ((B & 128) === 0)
break;
Multiplier *= 128;
}
return R;
};
/**
* WriteVarInt32 encodes a signed integer value of any size in a
portable variable length buffer
* We write the first byte as a sign byte. If it's 1, it means the
following unsigned varint encoded number will be negative. Else
positive
* @param Val Value to be Encoded. Since we use bit shifting, it may
only be a 32 bit unsigned int.
*/
TMemBuf.prototype.WriteVarInt64 = function(Val)
{
if (Val < 0)
{
this.WriteByte(1);
this.WriteVarUInt64(Val * -1);
}
else {
this.WriteByte(0);
this.WriteVarUInt64(Val);
}
};
TMemBuf.prototype.ReadVarInt64 = function()
{
var Sign=this.ReadByte();
var N = this.ReadVarUInt64();
console.log("ReadVarInt64: N="+N);
return (Sign ? -1*N : N);
};
/**
* WriteStr encodes a javascript string as a UTF-8 encoded Uint8Array
and writes this array to our buffer
* @param {type} sDOMStr Any javascript string
* @returns NONE
*/
/* DANGEROUS!!! USE WriteStrAndItsLen INSTEAD OF THIS FUNCTION!!
TMemBuf.prototype.WriteStr = function(sDOMStr)
{
this.WriteBuf(StrToUTF8Arr(sDOMStr));
};
*/
/**
* WriteStr encodes a javascript string as a UTF-8 encoded Uint8Array and :
* 1. UTF8 encoded length of this string as a 32bit VarInt
* 2. utf-8 encoded string as uint8array
* @param {type} sDOMStr Any javascript string
* @returns NONE
*/
TMemBuf.prototype.WriteStr = function(sDOMStr)
{
this.WriteBufAndItsLen(StrToUTF8Arr(sDOMStr));
};
/**
* ReadStr reads a string encoded with WriteStr function
* 1. It first reads VarInt length of encoded binary data
* 2. Then, it converts this utf8 encoded data to a javascript dom string
* @returns DOM string or NULL if data is encoded incorrectly
*/
TMemBuf.prototype.ReadStr = function()
{
var L = this.ReadVarUInt32();
if (L && this.RPos + L <= this.WPos)
{
var Str = UTF8ArrToStrWithStartPosEndPos(this.Buf, this.RPos,
this.RPos + L);
this.RPos += L;
return Str;
}
return "";
};
/**
* GetBuf returns a view of the buffer as uint8array having length
exactly equal to WPos
* Before this operation, this.WPos and Buf.length are usually different
* But this function, returns a buffer containing only the data we've
written to our buffer so far
* @returns {unresolved}
*/
TMemBuf.prototype.GetBuf = function()
{
return this.Buf.subarray(0, this.WPos);
};
function TestMemBuf()
{
var mb = new TMemBuf(8);
var FirstNum = 128000, SecondNum=8989, ThirdNum=200000000991,
S1=-128, S2=-200000000991, S3=98989;
var Str1 = "Selamlar İsmail KİZİR. Nasılsın?";
var N;
mb.WriteVarUInt32(FirstNum);
mb.WriteVarUInt64(SecondNum);
mb.WriteStr(Str1);
mb.WriteVarUInt64(ThirdNum);
mb.WriteVarInt64(S1);
mb.WriteVarInt64(S2);
mb.WriteVarInt64(S3);
N = mb.ReadVarUInt32();
if (N === FirstNum)
console.log(FirstNum+" successfully read from MemBuf!");
else console.log("ERROR: "+FirstNum+" COULD NOT BE READ! We got: "+N);
if (mb.ReadVarUInt64() === SecondNum)
console.log(SecondNum+" successfully read from MemBuf!");
else console.log("ERROR: "+SecondNum+" COULD NOT BE READ!");
var S = mb.ReadStr();
if (S === Str1)
console.log(Str1+" successfully read from MemBuf!");
else console.log("ERROR: "+Str1+" COULD NOT BE READ! We Got: "+S);
if (mb.ReadVarUInt64() === ThirdNum)
console.log(ThirdNum+" successfully read from MemBuf!");
else console.log("ERROR: "+ThirdNum+" COULD NOT BE READ!");
if (mb.ReadVarInt64() === S1)
console.log(S1+" successfully read from MemBuf!");
else console.log("ERROR: "+S1+" COULD NOT BE READ!");
if (mb.ReadVarInt64() === S2)
console.log(S2+" successfully read from MemBuf!");
else console.log("ERROR: "+S2+" COULD NOT BE READ!");
if (mb.ReadVarInt64() === S3)
console.log(S3+" successfully read from MemBuf!");
else console.log("ERROR: "+S3+" COULD NOT BE READ!");
}
TestMemBuf();
On Sat, Nov 25, 2017 at 9:52 AM, John-Mark Gurney <jmg at funkthat.com> wrote:
> Peter Gutmann wrote this message on Fri, Nov 17, 2017 at 10:17 +0000:
>> Santosh Chokhani <santosh.chokhani at gmail.com> writes:
>>
>> >Wow. That is magic. Do not decode a certificate but verify signature and
>> >extract fields.
>>
>> How about "do a string search through the encoded cert data until you find the
>> OID for the public key, then extract the key components from the bytes that
>> follow and use those". That's what you get when the spec mandates the use of
>> a full-blown PKI and the target hardware is a Cortex M0 or an MSP430.
>
> I really hope that's not the case, because if it doesn't fully parse
> all the leading data, you are now open to a packet-in-packet style attack.
>
> --
> John-Mark Gurney Voice: +1 415 225 5579
>
> "All that I will do, has been done, All that I have, has not."
> _______________________________________________
> The cryptography mailing list
> cryptography at metzdowd.com
> http://www.metzdowd.com/mailman/listinfo/cryptography
More information about the cryptography
mailing list