Openfire server-side avatar scaling

Thanks for sharing this. I love it when we get back little gems like this. I’d love to incorporate this functionality, although I’d like to apply it somehow to all avatar images - not just LDAP-related ones. From the top of my head, I’m thinking of something like an Openfire plugin that intercepts all vcards, and modifies them. I’ll try to have a crack at it later.

Right - I’ve attached a highly experimental plugin that builds on the stuff that was provided by Mike.

What this plugin attempts to do, is to nest itself between the VCardProvider that’s configured (or the default one, if none is configured). It will simply proxy any and all invocations, but whenever a VCard passes through, it quickly resizes the photo binary (if any is in).

There’s a couple of points:

  • This only works for vCards that are stored on the server (including vcards being stored, and being retrieved). There’s probably a lot of ways around this
  • Pidgin already sends a small image when you update your picture (which makes it a lousy client to test stuff )
  • Resizing always tries to make the image square, but will only kick in of the original image is larger than the intended square (so a smaller rectangle won’t be affected). Is making stuff square a good idea? No clue - that’s what to original code did though.
  • The target height/width (which in a square is the same number) can be configured through the avatar.resize.targetdimension property. It defaults to 96.
  • This works for all providers, not just the LDAP one
  • This was tested about 4 minutes, which is just barely over the ‘it compiles - ship it’ standard.
    I’d love some feedback!
    avatarResizer.jar (7595 Bytes)
1 Like

Oh, and Mike, you should be credited in some way. If you ping back with some details, I’ll get those in the documentation of the plugin.

Sources: Openfire/src/plugins/avatarResizer at avatar-resizer · guusdk/Openfire · GitHub

1 Like

I spun up a test server from that branch of the code. Note that I did not extensively test this, and that I only verified it with LDAP photos, but it did scale the pictures as desired. Additionally, with the scaling, Windows Pidgin clients did not hang nor show CPU/RAM spikes.

1 Like

Guus,

What are your ultimate plans for this code? Are you going to eventually try releasing an official plugin? What do you think the timeline to that is and for what version of Openfire would it be released?

As a rule of thumb, I don’t have to much “ultimate plans” I’m known to have the occasional foggy thought.

We’ll include this in our source tree, which will make it an “official plugin” (although you really shouldn’t read to much into that).

I’ve created an item in our issue tracker here: [OF-1128] Avatar Resizer plugink - IgniteRealtime JIRA

Wrapped up the code in a nice plugin (even found a suitable icon for it). github pull request here: OF-1128: New plugin to resize vCard-based avatars by guusdk · Pull Request #578 · igniterealtime/Openfire · GitHub

Once that gets merged, it’s official! I didn’t bother to search back to what the minimum supported Openfire version is. I expect this to be compatible with pretty much any version (as it’s integration point, the vcard provider, has not changed much in recent years).

1 Like

Well, that was fast. @Daryl Herzmann was faster than his own shadow, and already took care of everything. The new plugin is now available here: Ignite Realtime: Openfire Plugins

2 Likes

Thanks so much for your work on this Guus!

As a follow-up, I deployed this on our production server and while it did help, several users still noted the delay problems. At this point, I am opening a Pidgin ticket to see if I can get any more details/help from them.

It is to expected that the solution only affects some avatars - notably, the ones in your own database/LDAP/AD. It should, however, be ‘as good’ a solution as the original code you provided. If that did not give you issues, I’m somewhat surprised that you now have.

One thing that you might want to check: once a client obtains an avatar, it might cache it for some time. If an avatar was obtained before you installed the plugin, clients might continue to suffer the consequences of having oversized images. Could that explain the problems your users are experiencing?

Yes, I had thought the same thing, but even after closing the client and removing the cached images, at least one test client still showed the problem. Our transition to Openfire 4.0.1 was correlated with a switch to Pidgin 2.10.12 (from 2.10.11) so it’s possible enough of that code changed where the scaling fix no longer does enough; however, since it seems to affect different clients to different extents, this is certainly a Pidgin issue. This plugin works as advertised and does what we want

Nice plugin!

I just looked up my old vCard 2 OFUSER Plugin (public class VCard2OFUSER implements Plugin, VCardListener) and wonder whether one should rewrite it to use delegation like you did. Is there a benefit doing it this way or are there other (dis)advantages?

I implemented a background task iterating over all vcards to get things done in time. Only ‘update’ and ‘create’ events were processed. Calling ‘resize’ for every read-access may cause performance issues.

I did this trick for a proprietary Auth-, User- and GroupProvider, which are very similar to the VCardProvider. I feel that it’s somewhat of a hack, as all of those were not designed to be changeable at runtime in the first place. Instead, they’re supposed to be defined in configuration, before Openfire starts. I’m not sure how things hold up if you unload / reload a plugin like this after the server is in use.

I thought about changing the images, but decided against it. The original use-case was for an LDAP-based provider. It makes sense that those images are re-used by other systems than Openfire. Resizing images in such a shared component might affect other users. On top of that, I’m not even sure if all VCard providers allow changes in the first place. That’s why I went with the read-access approach. A drawback of that, on top of the added resource usage, is that the hash that’s calculated of the image does not correspond with the actual image.

In all, it’s not an ideal solution. I would definitely not want this to be part of the Openfire core. But as a plugin, that will be used only by people that actually need it, I think it’s OK.

If I’m understanding you correctly Guus – the images get resized each time the vcard is requested, as opposed to saving the resized image in the Openfire database? I honestly don’t know, but can you tell from my original snippet if the old one resized the stored images only once? If so, I wonder if this is why the Pidgin clients are still seeing issues – they ultimately get a smaller image which they can handle better, but they still might be requesting at a rate that causes them to block as they wait for the server to respond.

The original code that you provided did the same, Mike: it appears to resize when the vcard is loaded (although the snippet is incomplete, there’s no way to be sure if it was done in other places too).

I do not expect this to take a huge number of resources (unless a large amount of users is doing this at exactly the same time). If it is indeed this plugin being a botleneck, I’d expect you to notice from a sudden increase in CPU and/or memory usage by the Openfire process. You could have a look in your monitoring data.

If you increase your log level, you can more or less determine what time it takes, as the Resizer logs before and after it does the resizing.

That was the only call to it so I believe you are correct in stating that it did it on demand for each read.

I turned on debug logging and found that the resize was taking less than a millisecond, so I doubt that is causing my issues:

2016.04.21 15:32:53 org.igniterealtime.openfire.plugin.avatarresizer.Resizer - Original image size: 17931 bytes.

2016.04.21 15:32:53 org.igniterealtime.openfire.plugin.avatarresizer.Resizer - Original image is 300x300 pixels

2016.04.21 15:32:53 org.igniterealtime.openfire.plugin.avatarresizer.Resizer - Original image is already square (300x300)

2016.04.21 15:32:53 org.igniterealtime.openfire.plugin.avatarresizer.Resizer - Resized image is 96x96.

2016.04.21 15:32:53 org.igniterealtime.openfire.plugin.avatarresizer.Resizer - Resized image size: 2389 bytes.

2016.04.21 15:32:53 org.igniterealtime.openfire.plugin.avatarresizer.Resizer - Replacing original avatar in vcard with a resized variant.

Guus-

Hope you are still around and may be willing to look back into this plugin.

It took a long time for me to find this bug, but it turns out that your avatarResizer plugin replaces the default provider.vcard.className, but that your replacement appears to be missing a few things.

I grabbed this from the log:

2016.09.13 12:51:43 org.jivesoftware.openfire.vcard.VCardManager - Error loading vcard provider: org.igniterealtime.openfire.plugin.avatarresizer.DelegateVCardProvider
java.lang.ClassNotFoundException: org.igniterealtime.openfire.plugin.avatarresizer.DelegateVCardProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.jivesoftware.util.ClassUtils.loadClass(ClassUtils.java:76)
at org.jivesoftware.util.ClassUtils.forName(ClassUtils.java:48)
at org.jivesoftware.openfire.vcard.VCardManager.initialize(VCardManager.java:262)
at org.jivesoftware.openfire.XMPPServer.initModules(XMPPServer.java:566)
at org.jivesoftware.openfire.XMPPServer.start(XMPPServer.java:453)
at org.jivesoftware.openfire.XMPPServer.<init>(XMPPServer.java:169)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at java.lang.Class.newInstance(Class.java:383)
at org.jivesoftware.openfire.starter.ServerStarter.start(ServerStarter.java:105)
at org.jivesoftware.openfire.starter.ServerStarter.main(ServerStarter.java:56) 2016.09.13 13:04:05 org.jivesoftware.openfire.vcard.VCardManager - Error loading vcard provider: org.igniterealtime.openfire.plugin.avatarresizer.DelegateVCardProvider
java.lang.ClassNotFoundException: org.igniterealtime.openfire.plugin.avatarresizer.DelegateVCardProvider
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.jivesoftware.util.ClassUtils.loadClass(ClassUtils.java:76)
at org.jivesoftware.util.ClassUtils.forName(ClassUtils.java:48)
at org.jivesoftware.openfire.vcard.VCardManager.initialize(VCardManager.java:262)
at org.jivesoftware.openfire.vcard.VCardManager$2.propertySet(VCardManager.java:284)
at org.jivesoftware.util.PropertyEventDispatcher.dispatchEvent(PropertyEventDispatcher.java:91)
at org.jivesoftware.util.JiveProperties.put(JiveProperties.java:255)
at org.jivesoftware.util.JiveGlobals.setProperty(JiveGlobals.java:724)
at org.jivesoftware.openfire.admin.server_002dproperties_jsp._jspService(server_002dproperties_jsp.java:156)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
at com.opensymphony.module.sitemesh.filter.PageFilter.parsePage(PageFilter.java:118)
at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:52)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.jivesoftware.util.LocaleFilter.doFilter(LocaleFilter.java:76)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.jivesoftware.util.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:53)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.jivesoftware.admin.PluginFilter.doFilter(PluginFilter.java:80)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.jivesoftware.admin.AuthCheckFilter.doFilter(AuthCheckFilter.java:162)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
at org.eclipse.jetty.server.Server.handle(Server.java:499)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)

The end result of this is that vcards are not returned.

The reason it took me so long to realize this, is that if an openfire installation has a healthy vcard cache built up, that information is returned (and the picture is resized if present and the client doesn’t already have it), thus it looks like everything works. But if the cache is cleared, vcards are not returned.

I have already tried the obvious things like:

  • restarting openfire
  • reloading the plugin
  • verifying the information is still in LDAP

Switching provider.vcard.className back to the default immediately starts to return vcards as expected (just without the picture resizing).

I can recreate this problem on demand, so please let me know if you want help debugging/testing.

Hi Mike,

Thanks for reporting this. I can’t make time available to look at this right away, but I did log it in our issue tracker as: https://issues.igniterealtime.org/browse/OF-1193

maybe this is related? https://issues.igniterealtime.org/browse/OF-1145 I’ll let it to you if it should be linked to the ticket you created.