[OTR-dev] Verifies that Alice's gy is a legal value...

[Ian Goldberg]

On Sat, May 12, 2007 at 02:49:50AM -0400, Donny Viszneki wrote:
> I've been working on writing my own complete implementation of OTR
> version 2 including implementing all of the functionality it depends
> on from libgcrypt.

Wow; that's cool (if a little dangerous, if your background isn't in
crypto or math).  What language are you writing in?

> I'm not retarded, but my background isn't in cryptography or math. So
> I have a pretty simple question that probably anyone on the list could
> tell me.
> 
> In the OTR protocol version 2 description, a "modulus-2" function is
> referenced twice:
> 
> Verifies that Alice's gy is a legal value (2 <= gy <= modulus-2)
> Verifies that Bob's gx is a legal value (2 <= gx <= modulus-2)
> 
> What is this function? I can't seem to find any information on it.
> Every time I need a break from other OTR-related work, I decided to
> look around for it some more. Now I've finally decided to just ask the
> list. So what's the answer?!

"modulus" is the modulus being used for the Diffie-Hellman calculation
(the 1536-bit value listed under "Encoded Messages" in the spec).
"modulus-2" is the modulus, minus 2.  The reason to check that gx and gy
are in that range is because all of those values have large order
((p-1)/2 or (p-1)).  Values outside that range (like 1 and modulus-1)
can have small order (1 or 2, respectively).  Small order is Bad for
Diffie-Hellman.

Hope that helps,

   - Ian