Realtime Collaboration with Openfire Meetings

One of the reasons I got involved with building real-time web based communication applications with messaging, audio and video audio was because it was a stepping stone on my journey to find the holy grail of cooperative real-time web applications. These are web applications can be used in an N:N model (each user can simultaneously interact with all the other users). These type of applications are closer to real-life and are not restricted by the limitations of technology. As humans, we achieve a lot more when we work in teams and as a community . Now, thanks to the excellent work of the active Jitsi and retired Mozilla Labs teams, I have enabled the Openfire Meetings Collaboration API, a platform to explore real-time collaboration.

Image1.jpgImage3.jpg

Openfire Meetings is an Openfire plugin that combines JItsi Meet, Jitsi Videobridge and TogetherJS to create a web-based platform with unified mutli-user communications (presence, messaging, audio and video) from which real-time collaborative web applications can be developed.

Image4.jpgImage5.jpg

As each meeting participant can see and hear each other and share digital content in real time, we now have a working environment very close to real-life interaction. As a starting point, I have added the following demo/sample applications. The real power is when proper business applications are used, enabling tasks to be completed cooperatively by a team working together in real-time and not in a queue or by tag-teaming as we currently do today with email, social software and most web based business applications.

Co-Browsing

This is the simplest type of application that can be developed. It is a web site that enables joint navigation by all participants in the meeting. It is using the TogetherJS engine to track and show the multiple cursors and administer remote mouse clicks on each page. In the screen-shot above, I am co-browsing the ignite real-time web site which is not completely co-browse friendly as JIve SBS (for security reasons) does not permit it’s pages to be loaded in an iframe. Consequently, the community link click does not work.

Cooperative Editing

This is a modified version of “woot”, a rich-text collaborative editor by kroky. There are tons of stuff on this subject matter. Just look up “google wave”. Etherpad is the most popular and mature implementation so far. I found this presentation very informative especially explaining operational transformation and why woot (without operational transformation) is simpler.

Cooperative Drawing

This is the TogetherJS drawing sample application. Openfire Meetings automatically saves the contents of each application for each meeting on the server as XMPP private data against each user that creates the content. Next time you have the meeting, the drawing or rich text pad opens at where you left it from the last meeting.

Scrum-Board for Post-It Notes

This an adaptation of Ali Asaria’s scrumblr application. It is a very visually appealing application with real-time animation and reminds me of my previous life as a multi-user virtual reality applications developer with VRML.

I have promised to explain in detail how to use Openfire Meetings and over the coming months I plan to post a few more blogs on this and how to use the collaboration api for those who may be interested in taking this journey with me.

4 Likes

very very nice.

Have you checked big blue button project ?

thanks for the kind words. Yes, I know the project very well. We did some work together on the red5phone project

nice platform. but maybe some documentation is necessary if we want involved.

A feature that i need, for example, it’s an integration with outlook, or goggle calendar.

True that Are you volunteering?

i’m system administrator

Hi Dele,

just updated OF to 3.10.2 also in order to use this module but I’m encountering several problems.

  1. The OFMeet works fine with internal user except a shadow user present since the start of the meeting. It’s not clear to me what doeas thi means

  2. Trying to connect external users the a/V stream is not established dumping these errors:

ug 02, 2015 1:29:20 PM net.java.sip.communicator.util.Logger info

INFO: Notify add SSRCtest@conference.xxx.yyy.it/Giorgio SID: 1cn0n0hkormk0

lug 02, 2015 1:29:20 PM org.ice4j.ice.Component addUpdateRemoteCandidates

INFO: Update remote candidate for audio.RTP: 10.34.12.8:36823/udp

lug 02, 2015 1:29:20 PM org.jitsi.util.Logger info

INFO: Starting ICE agent without remote candidates.

lug 02, 2015 1:29:20 PM org.ice4j.ice.Agent startConnectivityEstablishment

INFO: Start ICE connectivity establishment

lug 02, 2015 1:29:20 PM org.ice4j.ice.Agent initCheckLists

INFO: Init checklist for stream data

lug 02, 2015 1:29:20 PM org.ice4j.ice.Component addUpdateRemoteCandidates

INFO: Update remote candidate for audio.RTCP: 10.34.12.8:46561/udp

lug 02, 2015 1:29:20 PM org.ice4j.ice.Agent setState

INFO: ICE state changed from Waiting to Running

lug 02, 2015 1:29:20 PM org.jitsi.util.Logger info

INFO: ICE processing state of IceUdpTransportManager #15958dd (for channels e110cb88e2cde23b) of conference a77ad3f877505b89 changed from Waiting to Running.

lug 02, 2015 1:29:20 PM org.jitsi.util.Logger info

INFO: RECV: 6D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21 :9A:33:79:CB:A9:17:13:90:C6:CE:19< content name=“data”>6D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21 :9A:33:79:CB:A9:17:13:90:C6:CE:19</co ntent>6D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21 :9A:33:79:CB:A9:17:13:90:C6:CE:19< /conference>

lug 02, 2015 1:29:20 PM org.ice4j.ice.Component updateRemoteCandidates

INFO: new Pair added: 195.135.201.241:5000/udp/host -> 10.34.12.8:36823/udp/host (audio.RTP)

lug 02, 2015 1:29:20 PM org.ice4j.stack.Connector run

WARNING: Connector died: 195.135.201.241:4443/tcp -> 192.168.167.51:61999/tcp

java.net.SocketException: Failed to receive data from socket.

at org.ice4j.socket.DelegatingSocket.receiveFromChannel(DelegatingSocket.java:670)

at org.ice4j.socket.DelegatingSocket.receive(DelegatingSocket.java:632)

at org.ice4j.socket.MultiplexingSocket.receive(MultiplexingSocket.java:480)

at org.ice4j.socket.MultiplexingSocket.receive(MultiplexingSocket.java:551)

at org.ice4j.socket.MultiplexedSocket.receive(MultiplexedSocket.java:166)

at org.ice4j.socket.IceTcpSocketWrapper.receive(IceTcpSocketWrapper.java:135)

at org.ice4j.ice.harvest.MultiplexingTcpHostHarvester$PushBackIceSocketWrapper.rec eive(MultiplexingTcpHostHarvester.java:1042)

at org.ice4j.stack.Connector.run(Connector.java:192)

at java.lang.Thread.run(Unknown Source)

lug 02, 2015 1:29:20 PM org.ice4j.ice.Component updateRemoteCandidates

INFO: new Pair added: 195.135.201.241:5001/udp/host -> 10.34.12.8:46561/udp/host (audio.RTCP)

lug 02, 2015 1:29:20 PM org.jitsi.util.Logger info

INFO: Starting ICE agent without remote candidates.

lug 02, 2015 1:29:20 PM org.ice4j.ice.Agent startConnectivityEstablishment

INFO: Start ICE connectivity establishment

lug 02, 2015 1:29:20 PM org.jitsi.util.Logger info

INFO: RECV: 6D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21: 9A:33:79:CB:A9:17:13:90:C6:CE:196D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21: 9A:33:79:CB:A9:17:13:90:C6:CE:196D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21: 9A:33:79:CB:A9:17:13:90:C6:CE:19

lug 02, 2015 1:29:20 PM org.jitsi.util.Logger info

INFO: RECV: 6D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21: 9A:33:79:CB:A9:17:13:90:C6:CE:196D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21: 9A:33:79:CB:A9:17:13:90:C6:CE:196D:02:D4:59:99:76:4A:BC:8A:DC:ED:E1:C3:9D:C3:D7:EA:57:7C:FF:21: 9A:33:79:CB:A9:17:13:90:C6:CE:19

lug 02, 2015 1:29:20 PM org.ice4j.ice.ConnectivityCheckClient startChecks

INFO: Start connectivity checks!

lug 02, 2015 1:29:20 PM org.ice4j.ice.Agent initCheckLists

INFO: Init checklist for stream audio

lug 02, 2015 1:29:20 PM org.ice4j.ice.Component addUpdateRemoteCandidates

INFO: Update remote candidate for audio.RTCP: 151.68.48.158:44785/udp

lug 02, 2015 1:29:20 PM org.ice4j.stack.NetAccessManager handleFatalError

WARNING: Removing connector:ice4j.Connector@195.135.201.241:4443/tcp status: running

java.net.SocketException: Failed to receive data from socket.

at org.ice4j.socket.DelegatingSocket.receiveFromChannel(DelegatingSocket.java:670)

at org.ice4j.socket.DelegatingSocket.receive(DelegatingSocket.java:632)

at org.ice4j.socket.MultiplexingSocket.receive(MultiplexingSocket.java:480)

at org.ice4j.socket.MultiplexingSocket.receive(MultiplexingSocket.java:551)

at org.ice4j.socket.MultiplexedSocket.receive(MultiplexedSocket.java:166)

at org.ice4j.socket.IceTcpSocketWrapper.receive(IceTcpSocketWrapper.java:135)

at org.ice4j.ice.harvest.MultiplexingTcpHostHarvester$PushBackIceSocketWrapper.rec eive(MultiplexingTcpHostHarvester.java:1042)

at org.ice4j.stack.Connector.run(Connector.java:192)

at java.lang.Thread.run(Unknown Source)

the 192.168.167.51 is my internal ip (client)

192.135.201.241 is the public ip of the server (no nat it has a direct public ip).

the ice section of the ofmeet control panel is configured as follows:

{‘iceServers’:[{‘url’:‘stun:stun.l.google.com:19302’}]} and as far as i see the other modules (suck as jinglenodes) says the machine is correctly set up.

Do you have any clue on what can I do to let the system work fine?

Kind regards