The context of this discussion is Jive’s decision to end their sponsoring of the hosting for the IgniteRealtime community, as discussed here.
Facts:
Jive has set a migration deadline of June 15, 2016.
Jive is happy to transfer the domain names, whenever the community chooses an agent for ownership, whether an individual or other legal entity (please refer to ‘creating a legal entity to represent the community’ for the discussion on that subject).
The existing hardware is for use of the community, and Jive will have Contegix ship it where-ever.
Jive primarily wants out of the hosting contracts, so, none of this has any bearing on the Jive Cloud forum/community site. Ignite Realtime is free to continue use of the Jive Cloud instance.
This community uses a significant amount of resources: multiple servers, half a terabyte of storage, multiple terabytes of bandwidth per month. Our new home will need to be sizable for it all to fit.
We’re already in possession of open-source software licenses for the tooling that we use (Atlassian, install4j, and on individual basis, Intellij).
We already had various individuals that offered hardware, rack space and even monetary donations. Thanks!
If we can, I feel that we’d best not tie the hosting to any one person, for instance by finding space at someone’s employer. As Daryl put it so elegantly: “I could be fired tomorrow.” Ideally, we find a new home that will survive the termination, departure, death, unwillingness, or other stop to the support of our community of any one individual. (This ties in somewhat to this discussion creating a legal entity to represent the community). For this reason, I’ve been searching for an organization that is willing to sponsor the community as a whole. I think that for the future, that would be a safer option than depending on a specific individual.
I am currently engaged in conversation with the general manager of a hosting company that I did business with in the past. They are open to the kind of sponsoring that we need. I don’t want to jinx things, but I’m optimistic that things will work out with them. At this time, I’m not actively pursuing other venues.
As for the migration itself: I think it is likely that in any new home that we’ll find, there will be less administrative support as what we’ve got from Contegix in the past. There’ll be quite some work involved: setting up the servers, installing software, migrating data. Although a developer, I have some affinity with system administration, and I would be happy to put in the effort. I’m hoping that others will join in as well.
I was thinking about it and I remember UFPR (Universidade Federal do Paraná).
They host Debian, Ubuntu and other distro mirrors. Some years ago I contacted them and they start to host ClamAV mirrors.
I know that it’s a different kind of hosting, but maybe they can have interest in hosting Ignite Realtime, since they use a lot of FLOSS and they are “open source friendly”. Another university with the same profile is USP (Universidade de São Paulo).
If you think that it is interesting, I can talk to them.
We can have this as an option. But i think Guus is trying to get hosting from a company which is specialized in that (hosting servers). So it should be a better option.
As an alternative, we could simply use the public GitHub service to replace the current community infastructure. We are already halfway there.
Migrate issues from Jira to Github and host Ignite Realtime at github.io. I appreciate it will not resolve all the current tooling requirements, but could be the opportunity to simplify the tools we use.
We might want to consider our own dog food. Openfire Social based on WordPress and BuddyPress with a very big open ecosystem of plugins and addins could be a suitable replacement for JIveX
Let’s focus on moving house first. Discussing the replacement of parts or tooling takes away much of the focus we need for other things now. Let’s not slow down that process.
BTW, pretty sure that Jive will keep hosting the community aspects of IgniteRealtime as you guys choose. The main goal is that the IgniteRealtime server(s) hosting the main website, build system, etc to move – the servers are the last ones at a hosting provider and Jive wants to end that contract to simplify data center footprint (as it’s been relayed to me).
Less than 3 months for the migration is not much time. Anyhow I prefer running on virtual machines and getting rid of the hardware which is getting old and will break some day. The IOS builds may be contributed by an individual unless we need nightly builds.
Moving to new vm servers is not as straight forward as shipping hardware, but it’s usually worth the effort.
How many cores and how much RAM do we actually need? 0.5TB storage seems to be quite low.
More is better, obviously. Things are quite sluggish as they are. But for now, I believe (virtualization makes for messy stats though) we’re dividing no more than 4GB of RAM between the two servers that do the bulk of all of our work (databases, atlassian stack, openfire instance, website, misc tooling). It is probably the bandwidth that’s most costly: I think we’re doing a couple of terabytes per month, if I include the downloads.
How about moving the downloads to one of the cloud storage providers? S3 or Google for example? Then perhaps the Atlassian stack to their cloud product? E.g., the more we can get off the servers and into managed cloud environments, the easier?
We’re currently working with @Guus der Kinderen to look at solving the short-term migration challenge. Once the current services are ported we can go on to look at other improvements.
I work for Surevine. Who are Surevine? We are a UK-based SME that delivers collaboration solutions. We specialise in solutions built on platforms such as Jive, though we’re very deliberately not tied to any vendors and often find ourselves working with open source technology. A lot of our focus is on chat, hence our involvement in the XMPP community and Openfire. It’s important that the Ignite Realtime community thrives, hence our offer to get more involved and hopefully address the hosting/community challenge.
We need to look at the options and therefore realistic costs (capital expense as well as time from one or two of our infrastructure team), but we believe we’ll be able to pickup the reigns from Jive by the June deadline. March is a very busy time for us (end of the financial year in the UK) so we’d like to get to April before we start looking at the port in detail. As above, we know the Jive platform well so in time will be able to help with improvements relating to the community. We also use the Altassian stack internally and support collaboration solutions built on this technology for some customers. With our XMPP background, we believe we’re well placed to take this on and it aligns with our open source ethos. As soon as we start on this, we’ll get the options posted to the forum for comment, e.g. running instances on virtual servers (likely Amazon EC2 hosted in the EU), moving more to Github, moving to Atlasssian’s cloud services, etc. If you could give us a couple of weeks, I’ll then put out an update with a plan and also see who in the community may be able to help.
Earlier today, I was contacted by the general manager of the hosting provider I referred to in my initial post to this discussion. He confirmed that they are willing to provide three suitable (virtual) machines, combined with the bandwidth capacity that we’d need to continue our services as is.
Note that this is a distinctly different offer from the one the John just made. We appear to be developing a luxury problem…
Now it’s time for Jive to chime in that they’ve changed their mind As i understand Surevine is more on a cloud solutions front, than just virtual servers. Also they have experience with many tools we use and even can provide a small team to work on migration. The other offer as i understand will provide servers and bandwidth. Only? First one sounds more tempting for a lazy approach, unless we really want to get dirty by migrating/configuring servers and tools ourselves. Of course, even with the first offer some sort of involvement will be still necessary.
In my opinion, we want to avoid maintaining physical hardware. Specifically, the hardware currently being used is quite old. We have updated the Mac mini once to get into 64-bit capable builds, but the HP servers have not seen any updates. Virtual solutions are relatively easy to find for all our current services (based on Linux), which the exception of the Mac OS X build system. That’s really the only hang up I see, but I believe OS X builds could only be done for major releases instead of the more continuous/daily builds which typically happen. This would allow the OS X build system to live remote from the virtualized systems and not incur too heavy a bandwidth burden.
@LG had another interesting idea: ask Contegix if they’d be willing to take us on as a sponsored project rather than a paying customer. They do this for some others. I’ve sent such a request to Contegix, and got a ping back. They’re evaluating the request.
A great benefit of this would be that we perhaps not have to do any work. There is in this the risk of the existing and old hardware breaking down eventually, as @benjamin points out, of course.
I agree on “old hardware breaks”. As for the Mac OS X. It has seen some development lately and having nightly builds for quick testing was handy. I wish there was an easy way to virtualize Mac… Pain in the neck to support it.
I heard back from Contegix. They are offering a very generous discount, but the remainder is still a four figure monthly amount (makes one appreciate Jive footing the bill all that time even more…). I’ve respectfully declined the offer.