[Cryptography] Secure erasure in C.

[Ray Dillinger]

On 09/08/2016 10:53 PM, Patrick Pelletier wrote:
> On 9/7/16 1:04 PM, Ray Dillinger wrote:
>> When possible, I do operations that can be done in finite
>> bounded memory using a static volatile buffer to hold
>> sensitive data.  

> If you're writing a program which is single-threaded, and which you know
> is always going to be single-threaded, that works.  But if you're
> writing a library, chances are that someone will want to use it in a
> multithreaded program someday.

I would never recommend multithreaded ANYTHING if
handling sensitive data.  I know you're right; if it's
a library some idiot will want to put it into a
multithreaded program.  But it's hard enough to
control information in a single-threaded program.

Even if you DO use it in a multithreaded program,
you'd put a page barrier around it so that anything
that runs in memory visible to any other thread
doesn't ever contain any sensitive information.

>> Transient copies and virtual-memory caches don't get made
>> because if the compiler can't assume the copies remain in
>> sync to the buffer, then making copies is useless.
> 
> As others have pointed out, virtual memory is completely orthogonal to
> what the compiler is doing, and volatile will in no way prevent
> swapping.  You'd need to lock the pages, using OS-specific calls, to
> avoid swapping.

It doesn't prevent OS swapping unless the OS is aware of
what memory is mapped virtual, which outside of a few
weirdies like Capsicum, they're not.  Even in that case
it's just notifying the OS that caching it is completely
useless and wastes disk space.  To actually forbid caching
to disk, you have to use memlock().

Outside of that it's down to machine configuration; secure
workstations usually have no cache partition at all. If you
run out of memory, "crash is better than cache."

If they do allow cache partitions, they use encrypted cache
partitions. But that's still OS-level; the program usually
has no control over that.

What a volatile buffer does is prevent the compiler from
issuing a stream of instructions that make the variable
name refer to different locations in memory at different
times, or queue up writes to it somewhere before actually
committing them to the buffer.  It forbids mapping it into
cache and then writing the final result back out after a
bunch of operations that don't actually touch the buffer.
And it means that when the program calls for that value
it has to read it from that single memory location, so
there's no optimization goal that can be fulfilled by
making transient copies elsewhere.

This is all well below the level of the OS. This is about
how the stream of instructions from the compiler goes about
carrying out the program - variable relocations, transient
write caching, etc, are normal strategies for current
compilers even in carrying out single-threaded programs.

>> For years I'd been defining 'erase' using
>>
>> //////
>> static void *(*const volatile deleter)(void*,int,size_t)=memset;
>> static void erase(void *buf,size_t len){deleter(buf, 0, len);}
>> //////
> 
> I think the problem is that deleter is static, therefore it only takes
> analysis of a single compilation unit to know that it won't change. If
> you make deleter non-static, then its value could always be changed from
> another compilation unit, and the compiler can't make assumptions about
> its value.  (You might need to remove the "const" as well; I'm not sure.)

Hmmm.  "Static" in this case means internal linkage only;
the function pointer isn't supposed to be made visible by
the linker to anything outside this file, even if they use
the 'extern' keyword to try to get at it. But as a "volatile"
pointer the compiler is strictly forbidden to assume that
its value won't get changed from outside the program.  It
doesn't matter whether other compilation units can see it,
or whether the whole-program optimizations are turned on.

"const" means that the particular compilation unit represented
by the file won't be changing it.  Hmmm.  It should work the
same way if I drop 'const', I guess, because the optimizations
usually allowed by const (partial evaluation, etc) don't apply
to volatile variables anyway.

Maybe an adhoc kluge that drops 'const' and then actually
changes the pointer value occasionally during runtime would
foil this particular optimizer.  And maybe memset_s (the right
answers) will have more widespread support by the time that
kluge stops working.

> Even with whole-program optimization, the optimizer can't know that some
> dynamically loaded library won't change the value of deleter. I suppose
> that if your program is 100% statically linked, and you use a whole
> program optimizer, then maybe it could still outsmart you.  But you've
> still raised the bar substantially over the "static" version.

Yeah.  The thing that surprised me was an 'optimization' based
on speculative caching.  The compiler didn't assume the value
would never change; it just checked whether it had actually
changed during runtime and skipped over the function call if
the pointer still had the same value.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://www.metzdowd.com/pipermail/cryptography/attachments/20160909/01f84a5b/attachment.sig>