[OTR-dev] Pidgin + Yahoo not support SMP?

[chris-tuchs at hushmail.com]

On Sun, 09 Aug 2009 15:23:28 -0700 Ian Goldberg <ian at cypherpunks.ca>
>wrote:
>On Wed, Aug 05, 2009 at 09:21:05PM -0700, chris-tuchs at hushmail.com 
>wrote:
>>
>> ?OTR,00002,00004,OCTiVspdIkxpqKlACVXDfCp/lFiI345aBsx60GgPF
>> ?OTR,00001,00004,?OTR:AAIDAQAAAAMAAAAEAAAAwEIH2lnX+NPoHtYY
>> ?OTR,00004,00004,Z+vhQwKssDnBB18z1nFURGZz3k2I2jaOqJek3U8/X
>> ?OTR,00003,00004,84StZgo+6T/vqtG+9OD4x3tHO1DirnzOih3A2Zg/j

That is what I received in the official Yahoo IM client when I
do an SMP dialog from Pidgin.  I can't say if a Yahoo server
reordered the lines, or if the Yahoo client did.  But it seems
much more likely to me that it is a property of the Yahoo
servers than a property of Pidgin, OTR, or the Yahoo client.

I doubt Pidgin or OTR since it is not a 100% repeatable case.
When I was testing about 1 out of 6 fragmented messages had the
fragments reordered.  It was not the case, for example, that the
second message of the SMP protocol always was reordered.

Naturally the problem will be more obvious with smaller MSS and
I was working with MSS of 640 characters in this case.  My
experiments show that I can actually cut and paste 800
characters to the Yahoo client, but only 650 to the Yahoo web
client.  So I conservatively set everybody up to use 640 for
that test.  I have run a couple experiments of SMP from Pidgin
to Pidgin over Yahoo, with no man-in-the-middle.  I was unable
to get SMP to work with MSS set to either 800 or 640 characters.

>That's bizarre.  The OTR protocol specifically assumes that
>messages, although they may be dropped if the network flakes,
>will be delivered in the right order if they're delivered at
>all.  An IM network that doesn't deliver messages in order
>would seem to be pretty poor for conversation, wouldn't it?
>Does GreenLife (or is it Yahoo?) really scramble the order of
>messages you send?

It does *seem* like a safe assumption that IM systems will
deliver messages in the order they are queued.  But my
observations of both Second Life and of Yahoo show that for
whatever reason it is not a valid assumption.

In Second Life under high server load, in a group chat I
sometimes see something like

you said: it's at the usual place
you said: I'll see you at the next meeting

Although I typed the lines in the opposite order.  The time
between my sending "I'll see..." and "it's at..." can vary
depending on the load.  The higher the load the longer I can
wait to send the second message but still have it arrive first.
I haven't checked the Pidgin source closely, but the GreenLife
client adds no delay between injecting the message fragments.

The good news is this is a well understood problem with very
standard solutions.

The cheap quick fix is to make otrl_proto_fragment_accumulate()
tell the other end that the message was messed up, and request a
retransmission.  Currently otrl_proto_fragment_accumulate() just
silently fails.  This simple fix will fail horribly when
messages become large compared to the MSS.

The 'right' fix seems to me to be, as much as I hate the
suggestion, something much fancier.  Something along the lines
of TCP's windowing, ACK, and retransmission algorithm.  Yuck.

(GreenLife is to Second Life as Pidgin is to Yahoo; OTR is to
GreenLife as OTR is to Pidgin.)

>   - Ian

Chris