[OTR-users] libotr4 handshake returns OTRL_MSGEVENT_RCVDMSG_MALFORMED

[Ian Goldberg]

On Wed, Apr 08, 2015 at 06:29:18PM +0300, ketmar wrote:
> Hello.
> 
> i'm migrating from libotr3 to libotr4, and i can't establish handshake.
> i wrote a sample code[1] to test OTR messaging. it emulates message
> sending between two clients (no sockets, only in-memory exchange).
> 
> OTR starts negotiation, but loops with OTRL_MSGEVENT_RCVDMSG_MALFORMED
> error, keep trying and failing.
> 
> you can download sample code and compile it with "compile.sh" script,
> and then just run "./test". it will generate private keys for two
> clients and then will try to initiate OTR session. and then you will
> see what's wrong on stderr (you can redirect stderr to file). note that
> program never finishes, so you have to press ^C to stop it (it SHOULD
> finish, but it can't).
> 
> i've tried this both with libotr 4.1.0 release and with git HEAD.
> libotr is build from scratch. other info:
> 
> GNU/Linux, arch: x86
> gcc 4.9.2
> libgcrypt-1.6.3
> 
> [1] http://ketmar.no-ip.org/test_cl.tgz

The D-H Key messages Alice is creating have a sender instance of 0,
which is not allowed:

$ echo '?OTR:AAMKAAAAALNjdGMAAADAXzazJH0iT8lRcWAVdJH7Az+uaeOIaDfIp2W3le0C024d6T+l3icGBJLYekwjzF4VNPtw3J2m9C7Wz1Yuw1pY9cf8YBsA/KkrpxLVTh4ZR3Y6RZrzIQBVOsCb4W9IdPpQDMA7ltKWAtxbNXNacALMGhzOsaoH3H/SODb5AF+RJMc/wqhlNwWKCNzix/JeHlw3NrHVp4pYgzA1q13nI5ZGWtZDrhZ/6Q/YgvJPo9vw4yec9mflSYICcHErzyMhOSVU.' | otr_parse
D-H Key Message:
	Version: 03
	Sender instance: 0
	Receiver instance: 3009639523
	D-H Key: 5f36b3247d224fc9517160157491fb033fae69e3886837c8a765b795ed02d36e1de93fa5de27060492d87a4c23cc5e1534fb70dc9da6f42ed6cf562ec35a58f5c7fc601b00fca92ba712d54e1e1947763a459af32100553ac09be16f4874fa500cc03b96d29602dc5b35735a7002cc1a1cceb1aa07dc7fd23836f9005f9124c73fc2a86537058a08dce2c7f25e1e5c3736b1d5a78a58833035ab5de72396465ad643ae167fe90fd882f24fa3dbf0e3279cf667e549820270712bcf2321392554


I see you have the create_instag callback set to NULL, so you're not
generating and persisting the instance tags.  That callback should
create the filename of the insttag file, and call

    otrl_instag_generate(userstate, filename, accountname, protocolname);

Can you try that and see if it solves the problem?



That said, an argument could be made for not requiring instags to be
persistent.

message.c:populate_context_instag could:
  - create the instag and link it into the userstate->instag_root linked
    list itself
  - call the create_instag callback (if set) only to write the new
    instag list to the file
  
rather than the current procedure:
  - call the create_instag callback (if set), which should call
    otrl_instag_generate with the right filename to both generate the
    instag (linking it into the usercontext) and write it to the file.

I would be amenable to having a bug filed on bugs.otr.im to allow for
non-persisted instags, containing the above description, particularly if
it also contains a patch.  ;-)


But ketmar, do try implementing the create_instag callback with the
above call to otrl_instag_generate and see if that solves your problem.

Thanks,

   - Ian