Introducing Hazelcast ... a new way to cluster Openfire!

Youngho_Kim · April 21, 2013, 12:00pm

Hi @Tom Evans.

Good to see you!

I has composing Openfire HA cluster on Amazon AWS using hazelcast cluster plugin.

Unfortunately! Amazon Service do not support direct connection between their regions.

So, I use the hazelcast cluster plugin on openfire. It seemed to be like which did not have a problem.

But, Increasing of hazelcast cluster node has occured big problem.

First, this service is very unstable between cluster node.

Second, this service has occured latency between clients message about 2~3 second. It tested just two clients.

So, I was find replication property on hazelcast document about “wan-replication”. and I was configured your hazelcast configuration xml file.

But, this do not operate. What is problem on my configuration?

like this:

com.hazelcast.impl.wan.WanNoDelayReplication</replication-imp l>

10.0.0.50:5701

com.hazelcast.impl.wan.WanNoDelayReplication</replication-imp l>

10.1.0.50:5701

kentzen · August 15, 2013, 7:36am

The issuses of after deployed cluster with Hazelcast Clustering Plugin of openfire .

I have three server nodes : A (10.0.1.113),B (10.0.1.176),C (10.0.1.158).

And all of them share a same Mysql database. Initially, I installed Openfire to B & C server.After that I installed Openfire to A. Such an installation made domain of all servers same as domain of A server.All server domains are 10.0.1.113 now. I used Strophe.js accessing A node successfully via BOSH After deployed cluster with Hazelcast Clustering Plugin. But might be owing to domains of B and C node are same as A’s , lead to I can not access B and C via BOSH with Strophe.js but the clients software like Spark can access all of them and works fine. Accessing B and C via BOSH made errors below:

Now my issue is that I need to access the cluster by using BOSH.Is it enough access only A node or I need to be able to access all nodes via BOSH? If I wanna access all nodes via BOSH in the meantime,how should I deploy cluster of openfire.

Here is configuration of Hazelcast

224.2.2.3

54327

10.0.1.113

10.0.1.176

10.0.1.158

10.0.1.113

I appreciate any help ! I hope you could forgive my awful english and understand my issue description.

Dele_Olajide · August 15, 2013, 9:14am

What is your domain name? Have you created a DNS record for your domain name pointing to all three IP addresses?

kentzen · August 15, 2013, 10:07am

No,I have not created.I merely use an IP among these three nodes such as 10.0.1.113.And I test the cluster in Local Area Nework.

Thanks for reply!

Dele_Olajide · August 15, 2013, 2:24pm

I don’t think it will work. The domain name should be a valid DNS name that resolves via the network to the next IP address by cycling through the 3 IP addresses

Alex_Mateescu · August 15, 2013, 4:56pm

I think having A records (IPs) in DNS foe each server is necessary as a fallback mechanism. It’s possible (and recommended) to have SRV records instead.

kentzen, maybe this will help you: http://wiki.xmpp.org/web/SRV_Records

kentzen · August 16, 2013, 1:02am

Yeah.Dele and Alex,Thanks you a lot .It works now.

Here is my steps:

My server is Redhat6 32 bit.

1.Modify all nodes /etc/hosts file .Plus code as following:

127.0.0.1 openfire #Any valid DNS name you want.

2.Change ‘setup’ option to false within ‘/openfire/conf/openfire.xml’,this step means that you could reset configuration for openfire.And type a DNS name you just bound with IP ‘127.0.0.1’ into ‘domain’ option.

3.Check ‘xmpp.domain’ field in ‘ofProperty’ table of openfire database,see if its value was set to ‘admin@openfire’ (or admin@).Change it if it isn’t.

4.Restart openfire.

(note : Do these steps to all nodes,and domain name of each node is the same like ‘openfire’ )

Via steps above .It should work fine! Please contact me if you confuse!

Alex_Mateescu · August 16, 2013, 7:22am

I doubt hardcoding IPs left and right will get you too far, but if it works for now… enjoy.

kentzen · August 16, 2013, 7:34am

Lol .Thanks.

kentzen · August 20, 2013, 3:12am

Which is better , Openfire cluster + nginx or Openfire cluster + Connection Manager?

Udhayakumar · March 25, 2015, 3:13pm

Am able to create openfire cluster with 3 node using hazelcast plugin. I have tried with TCP and Multicast both. But problem is its automatically discovering which i dont want to be part of my cluster. How can we restrict other openfire node from this cluster environment?

Please help. Thanks in advance.

Tom_Evans1 · March 26, 2015, 5:30pm

You can change the multicast group and/or port number in the classes/hazelcast-cache-config.xml file to create individual clusters:

224.2.2.3

54327

Udhayakumar · March 27, 2015, 6:49am

Thanks Tom.

It means, i can have my own multicast configuration. For example,

           <multicast-group>420.3.1.2</multicast-group>*

           <multicast-port>42031</multicast-port>*

*

Is it correct?

and Am deploying my server in EC2 i don’t want to open to many ports, how shall i restrict my servers to look in to only one port, i tried tcp/ip config as well but that doesn’t work. Am trying with private ip.

172.31.34.200:5701*
172.31.34.233:5701*
*

Please help me to configure cluster in secured manner?

Udhayakumar · April 23, 2015, 6:16am

Am using Openfire 3.9.3. Please let me know which hazelcast version will be compatible with this openfire version.

zhangjiyong · April 23, 2015, 7:25am

I use hazelcast1.2.0

get error

com.jivesoftware.util.cache.ClusteredCacheFactory - Failed to execute cluster task within 30 seconds

Udhayakumar · April 23, 2015, 7:48am

Am getting 2 exceptions

Timeoutexception, It is failing to execute cluster task within 30 seconds.
Hazelcastserialiaitonexcepiton, Class not found exception it occurs after 2 hours and 30 minutes and from there it keep on throwing every second.

If someone have fixed this exceptions, please help me to solve this one.

sh.lee · September 1, 2015, 6:27am

"Failed to execute cluster task within 30 seconds " issue help

openfire version 3.10.2
hazelcast plugin version 2.0 (hazelcast 3.4)
two server clustering, and now total session 5k

[error.log]

[debug.log]

sh.lee · September 10, 2015, 4:13am

“Failed to execute cluster task within 30 second” issue.

im guess this, hint from : www.javacreed.com/stopping-the-future-in-time/

edit source : org.jivesoftware.openfire.plugin.util.cache.ClusteredCacheFactory.java

check bold texts . please…

…

public Object doSynchronousClusterTask(ClusterTask task, byte[] nodeID) {

if (cluster == null) { return null; }

Member member = getMember(nodeID);

Object result = null;

// Check that the requested member was found

if (member != null) {

// Asynchronously execute the task on the target member

logger.debug("Executing DistributedTask: " + task.getClass().getName());

Future future = null;

try {

future = hazelcast.getExecutorService(HAZELCAST_EXECUTOR_SERVICE_NAME)

.submitToMember(new CallableTask(task), member);

result = future.get(MAX_CLUSTER_EXECUTION_TIME, TimeUnit.SECONDS);

logger.debug("DistributedTask result: " + (result == null ? “null” : result));

} catch (TimeoutException te) {

** if(future != null) future.cancel(true);**

** logger.error(“future is canceled, cause by TimeoutException”);**

logger.error(“Failed to execute cluster task within " + MAX_CLUSTER_EXECUTION_TIME + " seconds”, te);

} catch (Exception e) {

logger.error(“Failed to execute cluster task”, e);

}

} else {

String msg = MessageFormat.format(“Requested node {0} not found in cluster”, StringUtils.getString(nodeID));

logger.warn(msg);

throw new IllegalArgumentException(msg);

}

return result;

}

…

and see another method and edit : Collection doSynchronousClusterTask(ClusterTask task, boolean includeLocalMember)

Tom_Evans1 · September 16, 2015, 9:19pm

Hi folks … I believe the timeout issue has been identified and will be resolved in the next release of the plugin (2.1.2). Refer to [OF-943] for more info.

sh.lee · September 17, 2015, 6:22am

i try to that source. (written by me - 2015. 9. 10)

but timeout error is not solve …