[Cryptography] We could automate the process of detecting, some kinds of insecurities in new ciphers.

[Ray Dillinger]

On 1/22/22 17:21, Grant Schultz wrote:
> Ray Dillinger wrote on 1/22/2022,
>
> >I recently discovered that a candidate cipher had a bit pattern
> >correlation flaw, using a neural network.
>
> What type of neural network was it, and how deep, etc.?  I work with
> NNs and am curious as to the basic architecture/shape you used.

The basic architecture was the first I tried.  Experimentation is likely
to yield better.  It had 128 inputs, 6 internal layers of 256 nodes
each, and 128 outputs.  Each 'case' was a block of ciphertext with 127
inputs at 1 or 0, and 1 input as "unknown" at 0.5.

The "predicted" output was read at the output node corresponding to the
"unknown" input, and feedback (based on the actual value of the
"unknown" bit) was applied at that output alone.

Feedback was based on cube(error) rather than the more usual
abs(sqare(error)). I chose cube(error) for having a smaller derivative
near zero, in the hopes of being more sensitive to infrequent or very
slight biases.

Input was all english-language encrypted ciphertext (the Brown Corpus,
several translation dictionaries, Wikipedia, Gutenberg archive, all my
e-books and saved PDF academic articles, several online stories
archives, whatever I could get my hands on really). I stuck with
English-language text because English has a consistent set of patterns. 
Bit pattern correlation biases are frequently due to cipher operations,
but I supposed it also likely that they could be due to plaintext
patterns, and if the network is to detect correlations due to plaintext
patterns it should be trained on plaintext with a consistent set of
patterns. 

Each ciphertext block was made into a "case" attempting to predict each
bit. I was using batch updates based on average error over 1e6 cases, so
cases made from the same block were independent in the sense that
updates based on having seen a block while attempting to predict bitA
had not been made when the same block was presented in an attempt to
predict bitB.

I did introduce blocks with related plaintext by encrypting each
plaintext at each of the 16 possible byte offsets.  So a cipher block
based on plaintext bytes 0-7, AND a cipher block based on plaintext
bytes 1-8, AND a cipher block based on plaintext bytes 2-9, etc. were
all part of training data. So each plaintext was made into 16 different
ciphertexts and each ciphertext block was made into 128 one-bit cases.

That was an attempt to simply detect nonrandom bits.  That's suitable
for Diehard randomness tests.  It could be extended to work on chunks of
ciphertext longer than a single block.  Additional inputs ?might? make
it easier to detect patterns although I think the only patterns this
would really help with are the class of patterns that are due to
plaintext regularities or stream-cipher state.

In the specific context of detecting cipher weaknesses, inputs in
addition to the ciphertext alone are useful and adding them takes it out
of the context of being a good fit for the framework of Diehard.  For
example you could take plaintext in addition to ciphertext.

            Bear