[OTR-dev] RSA keys

[Nikita Borisov]

Russell points out that it would be nice to use RSA signature keys 
instead of DSA since RSA is less fragile when poor random number 
generators are used.  Unfortunately, the protocol requires the use of 
DSA in the key exchange message.  Here are my current thoughts on how 
to fix this without being too backwards-incompatible:

Create version 0x0002 of the protocol:
- Modify the layout of the key exchange message to replace the DSA key 
structure with:
   * a key-type tag (0x0001 for DSA, 0x0002 for RSA)
   * followed by either a DSA key (p,q,g,e) or an RSA key (n,e), 
depending on the tag
- Use an RSASIG instead of DSASIG if key-type is RSA

Implement the following logic:
- When initiating a key exchange, use protocol 0x0001 *unless* the user 
has an RSA key, in which case we have to use 0x0002
- When responding to a key exchange, use protocol 0x0001 unless the 
user has an RSA key *or* the incoming key exchange message was protocol 
0x0002
- When receiving a key exchange message, accept either protocol 0x0001 
or 0x0002

This way, out of the three classes of users:
  1. people who use the old plugin version (and have DSA keys),
  2. people who use the new plugin version and have DSA keys, and
  3. people who use the new plugin version and have RSA keys,
only 1 and 3 cannot talk to each other.  Do people think this is 
worthwhile, or should we just go ahead and make an incompatible change 
while the user base is small enough?

- Nikita