Hello community
Finally I’ve managed to set it up.
The short answer is - UDP.
And the recommendation is look at the coherence docs, not at the openfire. As the openfire clustering feature is completely based on coherence. Find the links in the ende of post.
Actually I wasn’t able to find the information re the infrastructure requirements for clustering (What ports, what protocols, wtf).
Probably I just was searching bad. But this is a point for OF documentation.
Also, I was too dummy to get the answers you gave me there
I’ve just compiled the doc how-to (UNICAST) for dummies like I was, here it is:
Openfirecluster setup
We are going to use openfire unicastmanual configuration as it is faster and safer than multicast.
That approach requires the pre-definingof all servers in the tangosol-coherence-override.xml file. That prevents unauthorized servers from joining cluster. That also reducesthe multicast flood and take load off from the network.
Assuming the oracle java and openfirehas already been installed.
Make sure that 8088 ports on allcluster servers is opened for UDP traffic.
To test the server accessibility oraclecoherence lib has an embedded testing feature.
1 Unpack the coherence distribution in some%COHERENCE_HOME% folder.
unzip coherence.zip
2 Prepare the tangosol-coherence-override.xml
stdout
<severity-levelsystem-property=“tangosol.coherence.log.level”>9
host1
8088
host2
8088
host1
host2
To diagnose problems you can also usethe
stdout
<severity-levelsystem-property=“tangosol.coherence.log.level”>9
inside the elementh.
Where 9 is the max verbose logging.Levels are -1 (no log msgs) to 9 (max level), default is 3.
By default Coherence uses a multicastprotocol to discover other nodes when forming a cluster. If multicastnetworking is undesirable, or unavailable in your environment, theWell Known Addresses feature may be used to eliminate the need formulticast traffic. When in use the cluster is configured with arelatively small list of nodes which are allowed to start thecluster, and which are likely to remain available over the clusterlifetime. There is no requirement for all WKA nodes to besimultaneously active at any point in time. This list is used by allother nodes to find their way into the cluster without the use ofmulticast, thus at least one well known node must be running forother nodes to be able to join.
isrequired elementh, specifies a list of "well known"addresses (WKA) that are used by the cluster discovery protocol inplace of multicast broadcast. If one or more WKA is specified, for amember to join the cluster it will either have to be a WKA or therewill have to be at least one WKA member running. Additionally, allcluster communication will be performed using unicast. If empty orunspecified multicast communications will be used.
If specified, restricts clustermembership to the cluster nodes specified in the collection ofunicast addresses, or address range. The unicast address is theaddress value from the authorized cluster nodes’ unicast-listenerelement. Any number of host-address and host-range elements may bespecified.
Specifies an IP address or hostname. Ifany are specified, only hosts with specified host-addresses or withinthe specified host-ranges will be allowed to join the cluster.
The content override attributes id canbe optionally used to fully or partially override the contents ofthis element with XML document that is external to the base document.
Also you can use host-range instead ofhost-address.
Specifies a range of IP addresses. Ifany are specified, only hosts with specified host-addresses or withinthe specified host-ranges will be allowed to join the cluster.
The content override attributes id canbe optionally used to fully or partially override the contents ofthis element with XML document that is external to the base document.
3 Installation the clustering plugin.
The clustering plugin adds support forrunning multiple redundant Openfire servers together in a cluster. Byrunning Openfire in a cluster, you can distribute the load amongst anumber of servers, as well as having some form of redundency in theevent that one of your servers dies. This plugin requires a validOracle Coherence license.
Follow steps 1 through 4 for addingOracle Coherence libraries to Openfire. Step 5 explains how to addthis plugin to your Openfire setup.
1). Get Oracle Coherence for JavaVersion.
2). Unzip the coherence file andlocate coherence.jar, coherence-work.jar and tangosol.jar in foldercoherence/lib.
3). Copy coherence.jar,coherence-work.jar and tangosol.jar to [openfire_home]/lib.
4). Restart Openfire server.
Navigate to theopenfire_url/available-plugins.jsp
in browser.
In the list of plugins find theclustering plugin and install it.
4 Add additional parameters into the JVM openfireconfiguration on host A.
sudo nano /etc/sysconfig/openfire
-Djava.net.preferIPv4Stack=true
-Dtangosol.coherence.localhost=-Dtangosol.coherence.machineid=10
-Dtangosol.coherence.wka2=-Dtangosol.coherence.wka1=
Substitute the with the hostname. The same filewith substitutions should be presented on the second host (host b).
5 Increasing the socket buffer size.
To help minimization of packet loss, the OS socket buffers need to be large enough to handle the incoming network traffic while your Java application is paused during garbage collection. By default Coherence will attempt to allocate a socket buffer of 2MB. If your OS is not configured to allow for large buffers Coherence will utilize smaller buffers. Most versions of Unix have a very low default buffer limit, which should be increased to at least 2MB.
In openfire_home/logs/nohup.out file you will receive the following warning if the OS failed to allocate the full size buffer.
UnicastUdpSocket failed to set receive buffer size to 1428 packets (2096304 bytes); actual size is 89 packets (131071 bytes). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
Though it is safe to operate with the smaller buffers it isrecommended that you configure your OS to allow for larger buffers.
On Linux execute (as root):
sysctl -w net.core.rmem_max=2096304sysctl -w net.core.wmem_max=2096304
6 Put the wka file into the openfire_home/lib folder.
Copy compiled on the step 2 tangosol-coherence-override.xml
into the openfire/lib folder.
7 Start openfire server.
Start both openfire servers.
sudo /etc/init.d/openfire start
Check the openfire_home/logs/nohup.outfor possible errors.
Navigate to the Server-Clustering menuon the admin panel and check that you see all nodes.
8 Troubleshooting.
1) Navigate tolib folder and runthe accessibility test
java-Dtangosol.coherence.override=tangosol-coherence-override.xml-Dtangosol.coh erence.ttl=255 -jar coherence.jar
Which means run embedded test fromcoherence.jar
using the unicast configuration and tangosol-coherence-override.xml as config file. That test is startingin dev configuration which is supposed to be used by developers whenthey are have couple of cluster nodes on the same machine. So the TTLof frames is set to 0 in this configuration. To ovveride this settingwe use -Dtangosol.coherence.ttl=255 parameter. Actually you can youTTL value just enough to reach from one node to another. In our caseit is 61.
Pay attention onActualMemberSet, it shows the cluster members.
Then run the same command on another node and you should see ActualMemberSet Size =2.
If it is not, then there is problems inserver conversation.
- Coherence datagram test
Included withCoherence is a Datagram Test utility which can be used to test andtune network performance between two or more machines. The Datagramtest operates in one of three modes, either as a packet publisher, apacket listener, or both. When run a publisher will transmit UDPpackets to the listener who will measure the throughput, successrate, and other statistics.
Syntax
The Datagram testsupports a large number of configuration options, though only a feware required for basic operation. To run the Datagram Test utilityuse the following synctax from the command line:
javacom.tangosol.net.DatagramTest <command value …> addr:port...
Run
java-server com.tangosol.net.DatagramTest -local localhost:8088-packetSize 1468
on one of thenodes to listen for UDP pockets on 8088 port
And on anothernode
java -server com.tangosol.net.DatagramTest -local localhost:8088 -packetSize 1468 :8088
to produce the udppockets.
If the everythingis ok, then producer will produce UDP pockets, and show consoleoutput like
oooooooooOoooooooooO
The series of "o"and “O” tick marks appear as data is (O)utput on thenetwork. Each “o” represents 1000 packets, with "O"indicators at every 10,000 packets.
On Node2 youshould see a corresponding set of “i” and "I"tick marks, representing network (I)nput. This indicates that the twotest instances are communicating.
If you see theonly ooooO and no iiiiIii then the servers could not see each other,and you should check networking issues. For the more informationrefer to the
http://coherence.oracle.com/display/COH33UG/Datagram+Test
9) Useful links.
In case of any problems use thecoherence user guide:
http://coherence.oracle.com/display/COH33UG/Coherence+3.3+Home
WKA tuning
http://coherence.oracle.com/display/COH33UG/well-known-addresses
Production check-list
http://coherence.oracle.com/display/COH33UG/Production+Checklist