XEP-0138: Stream Compression

I am starting to work on this in XIFF.

I was hoping someone to join the effort by adding this feature to Openfire…

Specification: http://xmpp.org/extensions/xep-0138.html

Primarily I will be doing the zlib part.

Anyone interested?

Perhaps this could be then added to Spark as well and wherever applicable.

Hi Juga,

I was hoping someone to join the effort by adding this feature to Openfire…

Specification: http://xmpp.org/extensions/xep-0138.html

Primarily I will be doing the zlib part.

Anyone interested?

The zlib method should already be implemented in Openfire. Please try it and write if you get troubles.

2009.08.02 23:15:22 ConnectionHandler:
java.io.IOException: Unknown error. Error code : 1
at org.apache.mina.filter.support.Zlib.inflate(Zlib.java:136)
at org.apache.mina.filter.CompressionFilter.messageReceived(CompressionFilter.java :159)
at org.apache.mina.common.support.AbstractIoFilterChain.callNextMessageReceived(Ab stractIoFilterChain.java:299)
at org.apache.mina.common.support.AbstractIoFilterChain.access$1100(AbstractIoFilt erChain.java:53)
at org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.messageReceive d(AbstractIoFilterChain.java:648)
at org.apache.mina.filter.executor.ExecutorFilter.processEvent(ExecutorFilter.java :239)
at org.apache.mina.filter.executor.ExecutorFilter$ProcessEventsRunnable.run(Execut orFilter.java:283)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:51)
at java.lang.Thread.run(Unknown Source)

This is Openfire 3.6.4.

Stream compression fails after the client receives the message.

Do you know any working client or libary to reproduce it? Maybe I can start to implement zlib compression for Smack too.

Ups, Smack already provides stream compression and it worked with my Openfire installation:

05:31:09 PM RCV (870314914):

<?xml version='1.0' encoding='UTF-8'?>

<stream:stream xmlns:stream=“http://etherx.jabber.org/streams” xmlns=“jabber:client” from=“cs.uni-potsdam.de” id=“c88d53b7” xml:lang=“en” version=“1.0”>

stream:features

zlib

</stream:features>

05:31:09 PM SENT (870314914):

SmackTest

05:31:09 PM RCV (870314914):

alice@cs.uni-potsdam.de/SmackTest

05:31:09 PM SENT (870314914):

05:31:09 PM RCV (870314914):

05:31:09 PM SENT (870314914):

zlib

05:31:09 PM RCV (870314914):

<compressed xmlns='http://jabber.org/protocol/compress’/>

05:31:09 PM SENT (870314914):

<stream:stream to=“cs.uni-potsdam.de” xmlns=“jabber:client” xmlns:stream=“http://etherx.jabber.org/streams” version=“1.0”>

05:31:09 PM SENT (870314914):

If you will share your sources and help me to get it running, then I could reproduce your problem and will try to help you with the Openfire part.

At least Miranda has an advanced setting for compression.

The error most prorably occured due to wrong format of data I had sent to the server.

Anyhow the server should reply with

<stream:error>
  <undefined-condition xmlns='urn:ietf:params:xml:ns:xmpp-streams'/>
  <failure xmlns='http://jabber.org/protocol/compress'/>
    <processing-failed/>
  </failure>
</stream:error>
</stream:stream>

Where the condition seems to be just this undefined.

Perhaps the new version of rfc 3920, part 5.8, will allow this to be more precise.

Anyway, the compression code will be in XIFF as soon as it works.

Could you record the first message, as is, that is transfered between the client and the server, once the compression has been negotiated?

Paazmaya wrote:

At least Miranda has an advanced setting for compression.

The error most prorably occured due to wrong format of data I had sent to the server.

Anyhow the server should reply with

> <stream:error>
>   <undefined-condition xmlns='urn:ietf:params:xml:ns:xmpp-streams'/>
>   <failure xmlns='http://jabber.org/protocol/compress'/>
>     <processing-failed/>
>   </failure>
> </stream:error>
> </stream:stream>

Where the condition seems to be just this undefined.

Perhaps the new version of rfc 3920, part 5.8, will allow this to be more precise.

I agree with you and will try to find where the Exception is thrown. Therefore I would like to get your code to find and fix the problem but maybe I’ll find it myself.

Could you record the first message, as is, that is transfered between the client and the server, once the compression has been negotiated?

For the stanza

<stream:stream to=“pappanoa.de” xmlns=“jabber:client” xmlns:stream=“http://etherx.jabber.org/streams” version=“1.0”>

the attached screenshot shows the sended raw data (encryption disabled).

Current trunk version of XIFF includes the work done so far related to the topic.

There are some bit level differences between the data Flash players implementation of Zlib sends and what Psi sends.

Whatever the case is, data in correct format or not, Openfire should reply with the described stanza, instead of silently failing.

Would LZW (http://xmpp.org/extensions/xep-0229.html) be easily added?

It seems Mina does not include it at the moment, "An IoFilter which compresses all data using JZlib. Support for the LZW (DLCZ) algorithm is also planned. "

Attached are the stream outputs of the same start stream from:

  • miranda

  • flash player when using big endian

  • flash player when using little endian, which seems to become same as the previous.

Does anyone have any ideas why the Flash player implementation differs with few bytes and thus not work with the other implementations of zlib?
stream-compressions.zip (1016 Bytes)

Hi Juga,

I am running into the same problem when trying to use compression with OpenFire.

Looking at the binary you posted, it seems like the only differences are at byte 3 and at the last 4 bytes. From the ZLib RFC (http://www.ietf.org/rfc/rfc1950.txt), it seems like these are Adler32 checksums, so it would seem like the Flash player method for computing the Adler32 is off?

I’m looking into this now, please let me know if you’ve made progress since your last post!

Cheers,

Karthiki

Hi, great to hear that there is some progress with this.

I was left stuck to that byte difference and never got the time to go further with it.

There is a library written in AS3 which should have Adler32 calculation, but its code is somewhat messy…

I cant even seem to find the library, but came across now this which I haven’t tried:

http://code.google.com/p/calista/

Hi Juga,

I tried using FZLib (http://www.wizhelp.com/fzlib/index.html) which is a port of JZLib in ActionScript after seeing that Smack was using JZLib (see XMPPConnection.java in Smack).

I tried using the same settings for the deflate that Smack is using and then compared the byte output to ByteArray.compress() and indeed there is a few bytes difference. But, when I send the FZLib output byte data to the server, it still doesn’t work! There must be something going on here, will keep looking into it.

Karthik

Getting somewhere…got the initial opening stuff to get a valid response from the server using FZLib…but now it is not getting a response from sending a SASL Anonymous message…

Using current trunk, also seeing similar errors. Using Wireshark here to compare with the successful packets sent by Spark in compressed mode.

Both clients receive:

After this packet is received the Flash client sends: <?xml version="1.0" encoding="UTF-8"?><stream:stream xmlns=“jabber:client” xmlns:stream=“http://etherx.jabber.org/streams” to=“xxxxxx” xml:lang=“en” version=“1.0”>

is it wrong that XIFF resends the opening XML tag while Spark does not?

We’ve taken the opening xml tag off and still no joy sadly.

btw - this post here - http://community.igniterealtime.org/message/207347#207347 details what we’ve looked at so far.

We have this working in XIFF 3.0.0 against an unmodified Openfire 3.6.4, using as3zlib. Please contact me so we can talk about getting this included in trunk!

Mani