We have been running a modified version of openfire and the connection manager for quite some time now, and thought I should start getting some patches back to the community. There was a long-standing bug with connection managers where messages could be received out of order. This first patch attempts to address that.
The reason we are using a CM is so that we could experiment with BOSH without worrying about bringing out main server down (the current BOSH implementation proved to be quite unstable under load)
The patches I am planning to prepare and deliver within the upcoming weeks are:
Fix for out-of-order messages from openfire to the connection manager module
This patch ensures that the same connection is used for each stream ID when sending data to the CM
It also ensures that received messages are processed in order by synchronizing on the client session inside the CM
It fixes an optimization issue with using the lowercase string “utf-8” for encoding instead of “UTF-8” (details)
Allow flash clients to connect to the CM by adding a socket cross-domain policy
Refactoring/redesign of the BOSH module used in the connection manager and openfire
Upgrade to Jetty7, including the new continuation model
Clean up code in general, making it much easier to understand
Fixes stability/OOM issues we were seeing once we got to 1k+ concurrent users
Add ability to tweak stream compression parameters
The current default parameters add about 300K of heap overhead for each connected client (details)
By adding a slightly customized JZLib jar which allows tweaking of window sizes, we were able to reduce this to about 40K, while keeping compression levels very high and limiting the hit on the CPU
Fix some concurrency issues related to DOM4J
We found a lot of lock contention on internal DOM4J objects, which were easily solved with a slight modification to DOM4J and some command line parameters.
The links for the patches for issue #1 are below, please let me know if you’re interested in the others:
Connection manager diff
Openfire diff
Ben
Edit: I should note that these patches were done against stock 3.6.4, but none of the changes are huge, so hopefully integration won’t be a problem.
If you all are comfortable with that, I have no problem doing it. I think it would be easier for everyone involved, as making the patches is actually somewhat painful
I believe we are running one of the largest openfire instances in production (45k concurrent peak, ~30k or so average), so I’m pretty confident in the stability/performance of all the changes I’ve made so far, but would love to get community input as well. We have also made significant changes to the XIFF library (compression/BOSH/stability improvements) that I would like to contribute as well.
Let me know what I need to do to get set up with SVN access and I’d be happy to start committing stuff into a branch so that it could be reviewed and hopefully merged into trunk eventually. I am definitely interested in helping out, this project has given us a lot over the years, and it would be great to contribute.
Hmmm, I can’t seem to invite you the the “Contributors” group. SBS doesn’t appear to recognize your user name in the invite functionality. Could someone else give it a go?
it may make sense to give Ben also Jira rights. Do you plan to review all the changes in detail or should he commit to trunk? Actually there were little commits to trunk in the last two months (except the ones I did).
In Jira we currently don’t have a “Review” or “Test” workflow item, this could be quite useful.
The patches look fine as far as I can tell, anyhow when I review things I don’t make sure that they are 100% error free as every commiter should have the right to introduce new bugs (;