I guess I will add my two cents into the possibility that there is a memory leak somewhere in Openfire 3.10.2. I have two Virtual Private Servers each running Debian 8 x64 with 1Gb of RAM, Openfire 3.10.2, and java version 1.8.0_05. Each server is showing nearly identical performance. I have created the openfire.vmoptions file and increased the memory to Xms128m and Xmx512m which defiantly increased the time it takes to crash significantly. But, there is still an issue. I have also added an snmpd service to each of the servers to monitor the performance.
I started a test with no chat connections last night (as seen with the slow decline of memory), and then started my hardware to communicate through the chat servers at 8AM this morning (as shown by the significant increase in the loss of free memory). Under this test, the chat messages were being passed back and forth at a total IO rate of a constant 120kb/s. Here are some interesting notes from the online logs:
I have also been chasing similar problem for a few days now. We used an old DB (mySQL based) and upgraded to OF 3.10.2 from 3.8.2. We had to use the old DB, because we wanted to retain the chat history, vCards and so on.
However when users logged on to the service and came online, OF started using unusually high amount of network traffic and memory. It saturated our 12MB network pipe, leaving no room for any other kind of traffic on that link. After some time, OF would become unresponsive, clients would randomly disconnect, our only choice was to keep restarting the OF daemon, and clearing/ dropping all network connections on that link.
What we observed was, that the clients were downloading a lot of vcard updates regularly. Usually, we see 5 to 10 updates per client, but after the upgrade 70+ updates were downloaded by all individual clients.
Also using the Smack debugger, we noticed 20 times more IQ packets being received.
The following links also describe similar behavior, but for different versions of openfire. In all of these cases, upgrades never worked for any of these.
This may be a similar problem but they are arrived at in totally different ways. We are using a fresh install (not from an upgrade) and are using the internal database (not an external Apache database). Our test was performed using only two “synthetic” users who pass messages back and forth to each other. Actually, we are using OF to share encrypted sensor data (“synthetic” users) over a geographically wide area. This traffic is generated at a nearly constant rate at about 45kb/s for each node. Our users do not use v-cards, it uses the buddy list to identify when a sensor is on the network and then establishes a one-to-one connection with it.
You indicated a smack debugger. Is this a different debugging than what is in the Admin console?
I am definitely willing to share more information about my issue if I knew what to provide. I am even willing to give you access to one of the VPS’s to allow you to watch the test in process.
I would love to see if you can reproduce this issue on windows or rhel/centos. I’ve noticed an increase in post with memory issues, and alot of them seem to be running Debian/Ubuntu.
Well, frankly I am also not a dev, neither am I too familiar with the XMPP standard…
The Smack debugger I was referring too, could be launched, if you enable “debugging” while starting to login through your Spark client. It could be found on the main Spark window, by clicking on Advanced option.
I installed OF on CentOS 7 x64. I have been running my test for two hours so far and have seen a significant difference in performance. First point, CPU utilization is holding steady at 4% where OF under Debian was in the 8 to 10% range. Also, there has been no decline in available memory. One other point to consider is that I mistakenly grabbed the x32 RPM package of Openfire instead of the x64. I will test this again with the x64 distribution next week.
Just noticed… There does not seem to be a x64 RPM package on the download page. I also see that it installs an older version of Java. That is fine, I’ll consider my issues related to a Debian install. Thanks everyone for all the help.
I’ll be curious to see how things are running after a while. If this proves to work well by changing OS, then may help the devs narrow down the issue.
I just thought about this, but debian uses openjdk right? I wonder if that could be the issue, and what would have happened if you used oracle java instead.
The VPS install of Debian had no JDK. I used Oracle’s jdk-8u60-linux-x64. For an update. Everything is still running along smoothly… except for another problem that I’ll open another thread for.
