On the Release Train at Ignite Realtime
The devs at Ignite Realtime (the open source, community site for Jive Software’s open source Real Time Communications) were brave/kind enough to share a detailed post about their intended changes to their release process.
On the Release Train at Ignite Realtime
I’m impressed they’re willing to share this with us (the global audience). Even more, I’m impressed that they’ve chosen strict, predictable release cycles (see next).
The key points to this model:
- We put out a new release every three weeks (although each release will have gone through a nine week process total).
- Three weeks before each development cycle is reserved for product management; three weeks after each development cycle is reserved for QA. But notice that all three processes are all happening at the same time.
We made a similar choice several years ago. A minor difference is that we chose four week cycles. The key is to chose a cycle that is relatively short (you’ll see it’s likely dramatically short compared to your prior release cycles), but long enough that you’re not burning time money in regression. This latter cost can be mitigated if you have automated regression tests (which we are building).
A major difference we have is overlapping cycles. We also do weekly builds to QA within this four week development cycle. QA tests our code each week (in theory) as the code is delivered. The reason we have overlapping cycles is because we add two weeks regression and a third week of “quiet” before a release. The regression we do is exhaustive and time-consuming. The quiet week is simply to add schedule buffer to that release. A consequence of overlapping cycles is the need for two testing systems.
A few of points of clarification: 1) our release cycle is more complex because of the regression testing window, 2) we do monthly Production deployments, and 3) the development and QA cycles are each only four weeks long. The two weeks of regression do add additional burden to the QA team. This regression “double-duty” on the QA team is non-ideal and we are agressively pursuing automated testing.
In summary, I believe having short, predictable release cycles is a major contributor to the quality code and products we produce. This release cycle won’t work for everyone, but I do believe it should be strongly considered as a viable option when selecting a release management strategy.
Footnote: I’m curious as to why they don’t do their continuous integration off of the trunk. See this comment:
Switch the builds in our continuous integration environment, TeamCity. The new branch becomes the “stable” release of the product and trunk becomes “development”.
To answer your question, we do continuous integration on both trunk and the branch — that’s what “development” and “stable” are. Also, to clarify, we’re not “formerly Jive Software”.
We did move the Openfire and Spark Open Source projects to igniterealtime.org, but they are still very much Jive Software efforts, as is our commercial product Clearspace. Thanks for the blog entry! It was great to read what you guys are doing.
The regularly scheduled release cycle is a key aspect of a disciplined software shop. Personally I place it second only to continuous integration as a practice of critical importance.
A critical aspect of making this a success is scope management. If you don’t have a way to match the scope to your capacity, you can burn-up on a release, and take the next scheduled release with you. Jive mentioned how they manage this. Their release cycle starts with a big meeting to determine which features will go into a release:
“Product management has already done the work of prioritizing the features so we just need to choose exactly what we’re going to work on and come up with the plan about how to get it done”
This sounds simple on the surface - but I think you can only be successful with regular releases if considerable discipline is exercised at this release prioritization phase. In some organizations this is remarkably hard, often just for political and business discipline reasons, although occasionally for product mix/client demand reasons. I’m willing to bet though that its more often political and business discipline that organizations struggle with.