We have a very modest Openfire installation, just 46 users, which ran untended and uninterupted for hundreds of days until November 5th, when we had the first “out of Java memory” crash. We took the enforced downtime as an opportunity to upgrade to 3.6.4.
It too crashed seven days later - out of Java resources…
Today (five days later) Java memory usage was at nearly 90% (of 1GB) so we restarted Openfire to preempt another crash.
Most of our clients are Pidgin (on Windows & Ubuntu), plus a smattering of OS X iChat. A couple of the Ubuntu users switched to Empathy on release of Ubuntu 9.10 (29 October). This does seem to have coincided the onset of our problems.
We have requested Empathy users to switch back to Pidgin and are monitoring closely.
I am running Openfire 3.6.2 on Ubuntu Server 8.10.
Java Version:
1.6.0_0 Sun Microsystems Inc. – OpenJDK Server VM
Appserver:
jetty-6.1.x
I started having this same exact issue after I setup a laptop with Ubuntu Desktop 9.10 with the Empathy IM client. Once I saw this post I turned off the Ubuntu laptop and rebooted the Ubuntu Server. The problem has not come back since.
I have noticed those connections too. Empathy queries possible proxy servers, which is why you see a lot of server-to-server connections being created. Although not very nice, it should be of no concequence. I’m running some quick tests to make sure.
I’ve seen usage of Empathy go up and instead of running out of memory once a month, it’s almost daily now. I can make my heap dumps available to the developers if they are interested in profiling it.
It’ll stop the random servers being called from Empathy, although doesn’t seem related to Guus’s PEP theory.
To enable Proposed updates in Ubuntu, goto “Software Sources” in Administration, goto the updates tab, and selected “Proposed”. Then reload your sources. telepathy-gabble should then appear in update-manager.
Couple Empathy clients connected, no problem so far, the memory use doesn’t show any suddent increase like before. I guess u did it Guus, great job! I will try to add some more Empathy clients to see if it’s stable.
Are there plans for this to be fixed in the next Openfire release (and if so is there a timescale on this)? Or will we have to stick with disabling the PEP?
Sorry that it has taken so long to get back to you.
I’ve been trying to identify the cause of the problem, but so far, have been unable to do so. I’ve asked other developers to have a look too, but none of them have been successful either. Sadly, my attempts are severely hindered by lack of time - I’m doing this in my spare time, which is limited.
I’m not comfortable at all releasing the next version of Openfire with this bug in it, but there’s going to be a point in the near future where I feel we should be pragmatic, and move forward. The release has been postponed for to long now.
In the meantime, the lead developer of the Empathy client has confirmed that updates have been released that should dramatically reduce the impact of the bug. Thanks, Sjoerd! I haven’t tested the new client yet though.
Although I’m having trouble identifying the exact cause, I did manage to make some progress. While investigating, I’ve discovered a number of smaller and bigger issues in the PEP / pubsub routines of Openfire. There’s no obvious direct link between these issues and the memory leak that we’ve been discussing here, but the issues that I’m addressing now are likely candidates, in the sense that these kind of (concurrency-related) bugs are known to cause these kind of problems. I’m currently busy rewriting parts of the PEP routines. I’m hoping that my general improvements make this problem go away (or at least help me to identify the cause). This borders on the edges of educated guessing and wishful thinking, but hey, it’s the holiday season.
Did you get the same error I mentioned in the Jira ticket? If so, you need to convert two of the files from CRLF to unix (dos2unix) and then try to apply the patch
We are having serious memory leak issues that are NOT related to Empathy or pep. I started another thread for that incase anyone else is experiencing these issues: http://www.igniterealtime.org/community/thread/40791