Openfire crashing (java.lang.OutOfMemoryError: Java heap space)

Openfire Openfire Support
1

Hi,

I’m using Openfire 3.8.2, Apache 2.2.15 and CentOS 6.

As tried what is proposed in http://community.igniterealtime.org/thread/49545, (AVA_HOME=/opt/openfire/jre) and Openfire has “Search” plugin installed (by Ryan Graham ).

I’m using Asterisk and Openfire in order to distribute extension state. It works fine until Java runs out of memmory. This is what I found in logs when error happened (note that IP 192.168.2.3 is a false one).

2013.10.15 07:59:36 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /192.168.1.26:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

at java.nio.HeapByteBuffer.(Unknown Source)

at java.nio.ByteBuffer.allocate(Unknown Source)

at org.apache.mina.common.SimpleByteBufferAllocator$SimpleByteBuffer.capacity0(Sim pleByteBufferAllocator.java:114)

at org.apache.mina.common.support.BaseByteBuffer.capacity(BaseByteBuffer.java:75)

at org.apache.mina.common.support.BaseByteBuffer.expand(BaseByteBuffer.java:107)

at org.apache.mina.common.ByteBuffer.expand(ByteBuffer.java:329)

at org.apache.mina.common.ByteBuffer.autoExpand(ByteBuffer.java:1801)

at org.apache.mina.common.ByteBuffer.putString(ByteBuffer.java:1166)

at org.jivesoftware.openfire.nio.ByteBufferWriter.write(ByteBufferWriter.java:45)

at java.io.Writer.write(Unknown Source)

at java.io.Writer.write(Unknown Source)

at org.jivesoftware.util.XMLWriter.writeNodeText(XMLWriter.java:1034)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1048)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:900)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:780)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.write(XMLWriter.java:255)

at org.jivesoftware.openfire.nio.NIOConnection.deliver(NIOConnection.java:257)

at org.jivesoftware.openfire.session.LocalClientSession.deliver(LocalClientSession .java:843)

at org.jivesoftware.openfire.session.LocalSession.process(LocalSession.java:281)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToBareJID(RoutingTableImpl. java:503)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableI mpl.java:284)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:234)

at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:109)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:80)

at org.jivesoftware.openfire.pubsub.PubSubModule.sendNotification(PubSubModule.jav a:716)

at org.jivesoftware.openfire.pubsub.Node.sendEventNotification(Node.java:2040)

2013.10.15 08:03:01 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /108.23.125.3:54644, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:02 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

at java.nio.HeapByteBuffer.(Unknown Source)

at java.nio.ByteBuffer.allocate(Unknown Source)

at org.apache.mina.common.SimpleByteBufferAllocator$SimpleByteBuffer.capacity0(Sim pleByteBufferAllocator.java:114)

at org.apache.mina.common.support.BaseByteBuffer.capacity(BaseByteBuffer.java:75)

at org.apache.mina.common.support.BaseByteBuffer.expand(BaseByteBuffer.java:107)

at org.apache.mina.common.ByteBuffer.expand(ByteBuffer.java:329)

at org.apache.mina.common.ByteBuffer.autoExpand(ByteBuffer.java:1801)

at org.apache.mina.common.ByteBuffer.putString(ByteBuffer.java:1166)

at org.jivesoftware.openfire.nio.ByteBufferWriter.write(ByteBufferWriter.java:45)

at java.io.Writer.write(Unknown Source)

at java.io.Writer.write(Unknown Source)

at org.jivesoftware.util.XMLWriter.writeNodeText(XMLWriter.java:1034)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1048)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:900)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:780)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.write(XMLWriter.java:255)

at org.jivesoftware.openfire.nio.NIOConnection.deliver(NIOConnection.java:257)

at org.jivesoftware.openfire.session.LocalClientSession.deliver(LocalClientSession .java:843)

at org.jivesoftware.openfire.session.LocalSession.process(LocalSession.java:281)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToBareJID(RoutingTableImpl. java:503)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableI mpl.java:284)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:234)

at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:109)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:80)

at org.jivesoftware.openfire.pubsub.PubSubModule.sendNotification(PubSubModule.jav a:716)

at org.jivesoftware.openfire.pubsub.Node.sendEventNotification(Node.java:2040)

2013.10.15 08:07:06 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

at java.nio.HeapByteBuffer.(Unknown Source)

at java.nio.ByteBuffer.allocate(Unknown Source)

at org.apache.mina.common.SimpleByteBufferAllocator$SimpleByteBuffer.capacity0(Sim pleByteBufferAllocator.java:114)

at org.apache.mina.common.support.BaseByteBuffer.capacity(BaseByteBuffer.java:75)

at org.apache.mina.common.support.BaseByteBuffer.expand(BaseByteBuffer.java:107)

at org.apache.mina.common.ByteBuffer.expand(ByteBuffer.java:329)

at org.apache.mina.common.ByteBuffer.autoExpand(ByteBuffer.java:1801)

at org.apache.mina.common.ByteBuffer.putString(ByteBuffer.java:1166)

at org.jivesoftware.openfire.nio.ByteBufferWriter.write(ByteBufferWriter.java:45)

at java.io.Writer.write(Unknown Source)

at java.io.Writer.write(Unknown Source)

at org.jivesoftware.util.XMLWriter.writeNodeText(XMLWriter.java:1034)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1048)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:900)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:780)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.write(XMLWriter.java:255)

at org.jivesoftware.openfire.nio.NIOConnection.deliver(NIOConnection.java:257)

at org.jivesoftware.openfire.session.LocalClientSession.deliver(LocalClientSession .java:843)

at org.jivesoftware.openfire.session.LocalSession.process(LocalSession.java:281)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToBareJID(RoutingTableImpl. java:503)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableI mpl.java:284)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:234)

at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:109)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:80)

at org.jivesoftware.openfire.pubsub.PubSubModule.sendNotification(PubSubModule.jav a:716)

at org.jivesoftware.openfire.pubsub.Node.sendEventNotification(Node.java:2040)

2013.10.15 08:07:10 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

at java.nio.HeapByteBuffer.(Unknown Source)

at java.nio.ByteBuffer.allocate(Unknown Source)

at org.apache.mina.common.SimpleByteBufferAllocator$SimpleByteBuffer.capacity0(Sim pleByteBufferAllocator.java:114)

at org.apache.mina.common.support.BaseByteBuffer.capacity(BaseByteBuffer.java:75)

at org.apache.mina.common.support.BaseByteBuffer.expand(BaseByteBuffer.java:107)

at org.apache.mina.common.ByteBuffer.expand(ByteBuffer.java:329)

at org.apache.mina.common.ByteBuffer.autoExpand(ByteBuffer.java:1801)

at org.apache.mina.common.ByteBuffer.putString(ByteBuffer.java:1166)

at org.jivesoftware.openfire.nio.ByteBufferWriter.write(ByteBufferWriter.java:45)

at java.io.Writer.write(Unknown Source)

at java.io.Writer.write(Unknown Source)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:740)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.writeNode(XMLWriter.java:1042)

at org.jivesoftware.util.XMLWriter.writeElementContent(XMLWriter.java:906)

at org.jivesoftware.util.XMLWriter.writeElement(XMLWriter.java:786)

at org.jivesoftware.util.XMLWriter.write(XMLWriter.java:255)

at org.jivesoftware.openfire.nio.NIOConnection.deliver(NIOConnection.java:257)

at org.jivesoftware.openfire.session.LocalClientSession.deliver(LocalClientSession .java:843)

at org.jivesoftware.openfire.session.LocalSession.process(LocalSession.java:281)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToBareJID(RoutingTableImpl. java:503)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routeToLocalDomain(RoutingTableI mpl.java:284)

at org.jivesoftware.openfire.spi.RoutingTableImpl.routePacket(RoutingTableImpl.jav a:234)

at org.jivesoftware.openfire.MessageRouter.route(MessageRouter.java:109)

at org.jivesoftware.openfire.spi.PacketRouterImpl.route(PacketRouterImpl.java:80)

at org.jivesoftware.openfire.pubsub.PubSubModule.sendNotification(PubSubModule.jav a:716)

at org.jivesoftware.openfire.pubsub.Node.sendEventNotification(Node.java:2040)

at org.jivesoftware.openfire.pubsub.NodeAffiliate.sendEventNotification(NodeAffili ate.java:225)

at org.jivesoftware.openfire.pubsub.NodeAffiliate.sendPublishedNotifications(NodeA ffiliate.java:127)

at org.jivesoftware.openfire.pubsub.LeafNode.publishItems(LeafNode.java:270)

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

2013.10.15 08:07:49 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:41635, L: /10.34.216.164:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

Don’t know what is happening. Why is memory running out?

Thank you very much in advance.

B.

2

If you haven’t already, you may want to give this Doc a go: http://community.igniterealtime.org/docs/DOC-1033

specifically the section about Heap Settings.

3

Thank you Jason. I already changed it. I’ll give it a go.

4

No success. Tried what is detailed in http://community.igniterealtime.org/docs/DOC-1033 but it didn’t work. In this configuration details, it says that one must add “-Xms32m -Xmx128m -Xss128k -Xoss128k -XX:ThreadStackSize=128” to INSTALL4J_ADD_VM_PARAMS. As said there, INSTALL4J_ADD_VM_PARAMS should be inserted in /var/lock/subsys/openfire (third line). Ok, that’s what I did BUT:

Looking for this variable inside /var/lock/subsys/openfire, it doesn’t appear anywhere else besides the line i specified. Therefore, it makes no sense :-/ Has this variable to be put somewhere else?

One sign (a guess) regarding changes not taking effect is that the limits in Openfire Web Portal memory bar doesn’t change (see picture: http://picpaste.com/Sin_t__tulo-MLGDPrr3.png).

Looking at the picture, you can see that the upper limit is near 400 MB. Well, when server fails, that limit is reached. Question: Is this normal? Openfire taking that much of space?

The error:

2013.10.17 08:23:59 org.jivesoftware.openfire.nio.ConnectionHandler - ConnectionHandler reports unexpected exception for session: (SOCKET, R: /55.62.35.7:45918, L: /10.73.137.120:5222, S: 0.0.0.0/0.0.0.0:5222)

java.lang.OutOfMemoryError: Java heap space

What do you suggest?

Should I increase java memory more than 400 MB? What is the normal value there?

Where can I specify this?

I really appreciate any help.

Regards,

B.

5

I can’t say what the proper settings should be, as that will vary depending on you usage of the server, but the min and max heap settings of 32 and 128 are very low for a server. I would probably start with 256 and 512, and if you have a server with lots of memory, maybe bump it up to a full gig (1024).