[Cryptography] "LLMs can hide text in other text of the same length"

[Jerry Leichter]

https://arxiv.org/pdf/2510.20075 describes a novel system that they authors think of as steganography, but could equally well be described as encryption.

Here's my description of the process - they use slightly different language.  A machine has a set T of tokens T1 ... Tn, and a set S of states  S1 ... Sk.  There is a map, Ranking(Si), which produces a permutation of the set of tokens; write R(Si,Tj) be the rank - position - of Tj in Ranking(Si).  Finally, the autoregressive map A: S x T -> S takes a state and a token to a new state.  The authors are interested in the case where T is the set of tokens known to an LLM, the state of the LLM is given by its learned probability weights and some set of previous tokens, A is given by shifting in the next token, and R is given by the probabilities assigned to each possible token in a given state, with some deterministic tie-breaking procedure.  This is a model that corresponds exactly to modern LLM approaches.

A key in the system is a pair (Sin, Sout) of states.  The input is a sequence of tokens Tin[]; the output will be a sequence of tokens Tout[].

Encryption:
    o Initialize two machines, Min to state Sin, Mout to Sout.     
    o Sequentially for each token t in Tin:
      - Determine the rank r of t in the state of Min;
      - Step Min's state by t
      - Let t' be the token with rank r in Mout;
      - Step Mout's state by t'

Decryption is the same thing with the roles reversed.  (There may be some off-by-one errors as I wrote this out.)

If we do this with an LLM trained on the language of the input message, the ranks r will be low (the input tokens will have high probability in Min), so the tokens t' chosen from Mout will also have high probability - so the output will "make sense":  It will "read" like some (perhaps offbeat) bit of LLM-generated text.  That's the sense in which this is steganography - though it's rather different from the traditional view where we perturb a pre-defined cover text.  Here we generate "cover" text.

Sin and Sout are actually specified as token sequences which are fed to the two machines in as prefixes.  Sout can be arbitrary, and will set the general subject and tone of the generated text.  Sin can, in the simplest case, simply be the empty string - though if specified, it should relate to the plaintext being hidden/encrypted so that the token probabilities are high even at the beginning.

As steganography, examples in the paper show that the technique probably works reasonably well.

The paper has a fairly simple-minded security analysis of this approach - though that's not really what the authors are interested in.

                                                        -- Jerry