[OTR-dev] Re: Requirements for libotr4

[Ian Goldberg]

Note: this message is long, and will only be of interest to Uli and
others who care about asynchronous privkey generation.

On Thu, Jun 19, 2008 at 02:37:38PM -0400, Ian Goldberg wrote:
> > 1. Asynchronous key generation. On some systems key generation can
> > take more than one hour. I don't know how other IM-clients do it but I
> > had to create a seperate process (not thread) for key generation which
> > is really not ideal since inter-thread communication is a lot simpler.
> > Also key generation will just finish at some random point in time and
> > update the keys file. What I would like to have is one thread safe
> > function that generates a key (some opaque struct I don't care about)
> > which I can later on(in my main thread) give to libotr for inclusion.
> 
> OK, that's an interesting request.  I think we can do that.  Willy,
> write that down.  ;-)
> 
> I don't want to put anything thready into libotr, but we can have a
> call like:
> 
> gcry_error_t otrl_privkey_generate_start(OtrlUserState us,
>     const char *accountname, const char *protocol,
>     void (*call_when_done)(void *newkey, void *data), void *data);
> 
> which (after some time) will call call_when_done with a pointer to the
> new key and whatever data you passed in.
> 
> Then there can be:
> 
> gcry_error_t otrl_privkey_generate_finish(void *newkey, FILE *fp);
> 
> which writes the newkey into the FILE (and frees newkey).  Your
> call_when_done routine can call that directly if you're single-threaded,
> or signal the main thread that it's ready, or whatever.
> 
> 
> What do you think of that?

I was thinking about it a bit more, and wondered who would be
responsible for checking that we don't have multiple simultaneous key
generations going on for the same account.  If it's the application
(i.e. calling the above otrl_privkey_generate_start with the same (us,
accountname, protocol) triple is just illegal), then the above is fine.

But it would seem to make more sense for libotr to check whether a
generation is in progress.  However, that check involves accessing the
us data structure, so it has to be done on the main thread, not on the
key generation thread.  So the actual API would have to have three
calls, not two:

 /* Call this from the main thread only.  call_when_ready will be called
  * in the main thread. */
 gcry_error_t otrl_privkey_generate_start(OtrlUserState us,
     const char *accountname, const char *protocol,
     void (*call_when_ready)(void *newkey, void *data), void *data);

 /* You can call this from a different thread.  call_when_done will be
  * called in the different thread. */
 gcry_error_t otrl_privkey_generate_calculate(void *newkey,
     void (*call_when_done)(void *newkey, void *data), void *data);

 /* Call this from the main thread only. */
 gcry_error_t otrl_privkey_generate_finish(OtrlUserState us,
     void *newkey, FILE *fp);

The application calls otrl_privkey_generate_start in the main thread,
and passes the callback function call_when_ready.

otrl_privkey_generate_start checks to see if a generation for this
account is already in progress.  If so, it returns a particular error
code, and the application should free any data state it allocated before
calling this routine.

If not, otrl_privkey_generate_start will create an opaque data structure
newkey, and pass it to call_when_ready, along with the data the
application handed it.

call_when_ready (code you write) should spawn a new thread and in it,
call otrl_privkey_generate_calculate with the newkey it was passed, and
a callback call_when_done.  call_when_done will be called in the
subthread.  You can choose to pass the same data you received (which is
what you passed into otrl_privkey_generate_start), or to free that data,
and pass something else into otrl_privkey_generate_calculate; whatever
works for you is fine.  otrl_privkey_generate_calculate guarantees that
will eventually call call_when_done.

call_when_done (also code you write) is called from the subthread.  It
should deallocate any state you allocated as the data parameter, and
signal the main thread that key generation is complete, handing it the
newkey pointer.

Whatever in your main thread receives this signal will call
otrl_privkey_generate_finish to write the privkeys file and update us.


How this signalling is done will depend on your application, of course.

*********************
**NOTE** that the existing routines will continue to exist (though
internally they may just call the above), so if you're not planning to
use threads, don't worry about this at all.
*********************

But I've got a question for you, Uli:  key generation usually happens
when you get an incoming OTR Query or OTR Commit message.  If you spawn
off a thread to generate the key, how will you keep around the state you
need in order to remember to continue the AKE when you're done the key
generation?  In some languages, call-with-current-completion would
probably suffice, but otherwise it seems like a real pain.  In fact, I
think libotr has to know this, since message.c will be the one that has
to deal with this, since it calls ops->create_privkey.  If that routine
spawns a thread to generate the key and return immediately, libotr will
be very confused.


So: even assuming we do the above, your create_privkey callback *must
not* return until the key generation is complete.  You can still perform
the key generation in a subthread, though, and your main thread can
handle UI events to avoid appearing frozen, but it *must not* call
any libotr functions (even in the UI handlers, hmm).


Now what do you think?

   - Ian