For anyone following my projects, you would be aware of me using jVoiceBridge as my audio conferencing and VOIP engine with Openfire. I stopping using Asterisk a while back and all my telephony is done with jVoiceBridge and the SIP plugin for Openfire. The combination of Openfire, jVoiceBridge and WebRTC provides a very compact unfied communications solution.
One of the many cool features of jVoiceBridge I was never able to exploit up until now was the spatial audio feature that creates a 3D audio immersion experience from a stereo mix.
What’s so cool about this?
As a home worker, I have been dreaming of when I would get to hear the separation of conference participant voices across a stereo field. By adding matching visual sitting postions, speech detection and improved voice quality, we might just get audio-conference calls back in fashion.
What makes this now possible is the use of the OPUS codec in WebRTC. However for a while, it looked like it was going to remain a dream unless I could figure out a way to access WebRTC audio streams directly from an Openfire plugin.
Introducing the Relay Plugin for Openfire.
I started out with the source code to the JingleNodes plugin for Openfire and got the WebRTC UDP packets wired into jVoiceBridge conferences. At first, I was limited to 8K mono audio from the PCMU (ulaw) codec, but thanks to the Jitsi project, I was able to add their OPUS decoder/encoder and finally got 48K stereo samples streaming into jVoiceBridge.
How does it work?
I have put together a demo web page of the plugin running on my server. To test, you have to access the same page from at least two PCs in order to test the 3D audio-conferencing effect.** It only works with Google Chrome.**
It will let you do the following things
- Join the demo conference choosing a codec and position on the stereo field (-90 to 90)
- Add a telephone call (SIP address) into the demo conference. OPUS re-sampling does not work with all SIP phones, use PCMU if you get no audio.
- Send DTMF to the telephone call.
- Watch the speech detection events
- Mute your audio from the conference
Where do we go from here?
The plugin will be an official Openfire plugin and the project will be hosted and supported from Igniterealtime. I am planning on exposing all the features of the plugin through the new Rayo XMPP Extension for remote call control. I hope to develop a Strophe plugin for Rayo to enable easy access from XMPP based web applications.
If you are interested in joining up with me to make this all happen sooner than later, send me an email or private message.