[OTR-users] Fwd: Re: Pretty-please standardize OTR signature storage, per OS.

[subharo at hushmail.com]

----- Forwarded message from "Tamme Schichler" 
<tammeschichler at googlemail.com> -----
(It seems I can't send this message to the list for some reason, so 
I'm
sending it to you directly now. Maybe you can forward it or know 
what
the problem is.)

Hello,

I just subscribed to this list and was planning to just read a bit
longer, but part of this suggestion really bothers me.

Am 07.09.2013 22:51, schrieb subharo at hushmail.com:
> Hello,
> 
> Thanks for the feedback an ideas!  Glad to hear there is some 
> interest.  :D
> 
> On Fri, 06 Sep 2013 14:25:35 -0400 "Daniel Kahn Gillmor" 
> <dkg at fifthhorseman.net> wrote:
> 
>> I'm not convinced that just outlining a location in the 
filesystem 
>> for
>> the data is sufficient, though, because of issues with concurrent
>> updates; e.g. what if Alice's OTR-enabled tool X tries to make a 
>> note of
>> Bob's fingerprint at the same time as OTR-enabled tool Y tries 
to 
>> make a
>> note of Charlie's fingerprint?
> 
> Agreed.
> 
>> use of an agent-driven approach (e.g. a dbus query or talk to a 
>> local
>> daemon bound to the loopback or something) could serialize and
>> centralize access without causing more trouble.  Related to 
>> earlier
>> discussions about looking up long-term OTR identity keys in the 
>> OpenPGP
>> keyserver network, it seems plausible that you could use a 
>> modified
>> monkeysphere validation agent for this purpose; bummed i don't 
>> have time
>> to work on it further at the moment, but if folks want to push 
in 
>> that
>> direction, i'd be happy to kibbitz.
> 
> I was hoping for something simpler and more elegant than this.  
How 
> about simple file locks?  If some flat config file (say, with OTR 
> fingerprints inside, with some nice XML or JSON markups or some 
> such) is in use by some IM client, that IM client "touches" (ie. 
> creates a file of size zero) a lockfile in the config folder 
> (having the same filename, but ending with ".lock").  Then one IM 
> client won't *write* to a config file if it sees that a lockfile 
> already exists (because some other IM client is already locking 
> it).  And it can tell the user to "try again later" if so (after 
> checking itself a few more times, every few seconds).
> 
> This way, more than one IM client can *read* the config files at 
> once, but each IM client can only *write* to the files one at a 
> time.  And a given IM client should *only* create the .lock files 
> for just as long as they're needed to make an update to the file, 
> like say < 1 second (just for *writing*).  OTR fingerprints are 
> probably added by users rather infrequently (no thousands of 
> commits per second here!), so this simple method should suffice 
for 
> about 99.999% of all users, IMHO.
> 
> To make it even more "atomic", the new file is created with a 
> random suffix (a UUID), and then once the lockfile is in place, 
the 
> new file is "moved" overtop the old file, upon committing, then 
the 
> lockfile is removed.  In the unlikely event that an IM client 
> crashes in the middle of that 1 or 2 seconds, then they'll have 
to 
> manually have to delete the lock file and pick which file is the 
> one they want, the new one (with the random suffix) or the 
original 
> one.  And surely one of the two files is in a coherent, ungarbled 
> state, no matter at which instant a crash occurred.  This is not 
an 
> uncommon way of "cheaply" implementing concurrency, as I'm sure 
> we've all had to delete a stale lockfile from time to time.

and precisely that is the problem. As people with technical 
background
deleting a lockfile is trivial for us, but that won't be true for 
most
users, especially if there's a "friendly" error message that doesn't
contain the path of the problematic file.

Speaking from experience, you can't assume that a non-technical end 
user
will reliably find their configuration directory or be able to
understand the problem from a short error message without further
instructions.

A possible fix would be to add a timeout to the specification after
which the lock can be automatically removed, or to require programs 
to
implement an interactive recovery. Both would push implementation 
effort
and the responsibility for a standards-compliant and compatible
implementation to the consumer side though.

> Sure, there are many sophisticated flatfile DB's one could use 
that 
> enforce concurrency (like say SQlite, or Python's Durus, which 
uses 
> nosql), but then you introduce more package dependencies, and you 
> make it less OS-agnostic.  I think going the route of using some 
DB 
> will push a solution about 10 times farther away.
> 
> What I'm advocating here (via simple lockfiles, and almost-atomic 
> commits) is a decent, *OS-agnostic* solution, which is quite 
> elegant, and simple.

I think a loopback daemon that listens for TCP on a specific port 
would
be much simpler. As almost all applications that use OTR would be
networked, a TCP connection would require no additional 
dependencies,
unlike XML or JSON which are not included in every standard library.

It would also be trivial to enforce compatibility: If the daemon 
accepts
only (and validates) e.g. UTF-8 and a specific binary command and 
reply
format, it becomes impossible for a program to work by itself but 
fail
when another one is in use.

The server would only need a few atomic commands, e.g.: Add/Update, 
Mark
as Revoked, Get Info and List Signatures. There would have to be a
reliable specification about how to associate specific accounts 
with the
signatures (and possibly how to attach additional meta-data. I 
think the
client applications should store non-portable data themselves,
separately, though.), but that is the same issue as with files.
All additional debug and management commands (Remove...) and the
underlying representation could be abstracted away from the 
consumer or
optional.

Additionally, it would be trivial to test components by including a 
Test
command that executes the following one without committing changes 
or
maybe even moves the current new connection into a temporary state 
that
is independent of others and discarded when the connection ends.
Obviously it would complicate the first release but would only have 
to
be done once and not in every client.

Finally, having the daemon abstract away everything but a very 
simple
protocol would make it easier to create libraries for different
languages/environments that can't call into C directly or don't work
well with native binaries for some other reason.

The only problem I see with this is that starting the daemon or 
service
wouldn't be platform agnostic, but the configuration files' location
wouldn't be either, so it probably would be a simple branch in 
either
case, in the client.


I know barely any C/C++, so I can't help with the back end, but I'd 
like
to create/help with managed libraries and a test tool to make
interaction with the daemon as simple as possible once the 
specification
is in place.

-Tamme