[Cryptography] Wrongware: was VW/EPA tests as crypto protocols ?

[Natanael]

Den 25 sep 2015 04:13 skrev "Ray Dillinger" <bear at sonic.net>:
> On 09/24/2015 09:09 AM, Henry Baker wrote:
> > By now, you've all heard of the VW SW that cheats/defeats the EPA
testing protocol.
> >
> > But VW isn't alone, and expect further revelations as the white hats
start investigating these types of misbehavin' SW.
> >
> > So what's a regulator to do?
>
> I'm making up a new term.  The term is wrongware.  Wrongware
> means software that is deliberately wrong, which is provided
> by the exact people from whom a faithful (rather than wrong)
> implementation is expected, specifically to cause misbehavior
> that they find desirable for whatever reason.

Related: http://darkpatterns.org/

That one is about what's essentially scams deliberately encoded in user
interfaces, such as hiding adware under advanced options in software
installers.

Wrongware could be a more general term, encompassing this as well as
software that's not targeting the user, but rather something else - the
intentional error might not affect the user, but another target entirely.
This would be different from bots and other malware - it would not have to
be an active function that target another computer system. Fooling an
auditor would definitely be part of it just like here, but could apply even
in finance and law and not just for machinery.

More generalized, whenever certain behaviors are unwanted for good reasons
by interested parties (regulators, auditors, courts, and civil
organizations) and the software promises either to NOT enable the behavior
or to directly PREVENT it, but yet intentionally doesn't act as promised as
a result of various incentives (profit, career advancement, etc), then that
would classify as wrongware.

> "Wrongware" would encompass the misbehaving VWs that deliberately
> cheat on EPA tests.  It would also describe, eg, deliberate
> backdoors installed by a router manufacturer to enable
> surreptitious access, deliberate vulnerabilities in operating
> systems, USB or disk controllers, deliberately installed
> vulnerabilities in computer BIOSes, "updates" released by a
> software vendor that secretly disable or cripple its own
> product, etc.

FSF's project seems relevant here: https://defectivebydesign.org/

This one is about software that screws over the user.

> As to the case in point:  I think this particular wrongware can
> be reasonably easily defeated.
>
> Develop a self-contained exhaust probe that can live on batteries
> for a few days.  Stick it to the tailpipe using aluminum speed
> tape with tamper-resistant seals, with its tip sticking into the
> tailpipe.  Let the owner drive around normally for a few days,
> then remove the device and download the data.

In short, design for auditability.

Somebody else also mentioned the independent probe idea in the previous
mails, and in the case of cars that is fortunately not very hard to measure
in most cases. In the case of software it is typically incredibly much more
difficult - just defining that you need to measure is hard enough already.
But it has to be done.

Even Zero-knowledge proofs would not necessarily prove what you need to
prove - they can only show a certain computation has happened - they can't
prove you have NOT also done something else. For example you could have a
separate program extract what's supposed to be secrets from a program which
is running in a ZKP generator that promises to protect the secrets. There's
a variety of systems based on trusted CPU designs that promises to achieve
assurances for this, but I'm not sure if any are as good as they promise.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.metzdowd.com/pipermail/cryptography/attachments/20150925/5dfcbd45/attachment.html>