Jenkins User Event CPH

This article was originally posted on my personal blog

Last week, praqma was organizing in Copenhagen a "Jenkins User Event". A User Event, compared to JenkinsConf, is a lighter meeting with (supposed to be) reduced costs organized by volunteers. Praqma got sponsorship from both CloudBees and Programing Research so that they can book a high quality conference room and prepare a nice meeting with all commodities.

That was my first travel to Danemark. I enjoyed the winds-farm on northern sea as well as the duty-free lego shop at airport, but didn’t have much time to discover the city - so will have to come back next year ;)

Thursday was about a Jenkins Code Camp (aka "hackathon"), that I joined late at 2pm due to flight being delayed. 20 geeks were talking about some technical issues, new features, implementation strategies for a large set of topics. I contribute a group to solve an integration issue by creating a new extension point in jenkins-core. Those already confident with jenkins development helped to write this code, some discovered the extension point design as well as way to contribute to jenkins (github pull request, etc), some were looking at jenkins source code for first time so learned a lot.

Day ended with beer then a chinese restaurant (typical Danish food :P), with lots of fun and nice discussions.

Friday was the Jenkins User Event. To reduce costs for such an event, compared to other Jenkins Conferences organized by CloudBees this year, lunch was not provided and conference program was "packed" into afternoon. This let praqma get a "reasonable" cost for this nice event, but still have a high quality conference, with printed programs, goodies, and coffee break. All praqma team was involved to make this event as pleasant as possible for all attendees, thanks a lot to them for contributing !
Conference was sold-out on friday morning, with 80 attendees.
After Lars Kruse welcome speak, and CloudBees to announce partnership with Praqma for DK, the conference started with two options : either an introduction to Jenkins, or an open-space discussion (~barcamp-like) for those that already know it well. I joined a small group first discussing about pre-tested commit, and then we divert speaking about best-practices, job and test performances issues, etc. Was a great exchange with interesting feedback.
The rest of the conference was single-track.

1rst session was about "facilitate strategic reuse of software" using jenkins CI. This session exposed how a industrial company changed it’s internal software development practices and team organization to share components and be more efficient. This for sure introduced some coordinations and integration costs but resulted in a significant productivity improvement. This talk was interesting as it demonstrate that highly industrial companies (here, a low energy consuming water pumps producer) today follow development practice to share component and use continuous integration practice to help. I just wondered speaker said "Clearcase facilitated" sharing components  -I wouldn’t expected those two words in same sentence :P
"Tales from the trenches" was a funny session explaining how Nokia came from stone age (manual integration with code freezes) to modern development practices. After reinventing the wheel with ~15 home made, perl-script based CI tools, they switched to Jenkins and Git as common tooling. Explanation on Git selection, evaluating multiple DVCS popularity, then migration from ClearCase, helping a lot early adopters, and later evaluating benefits (1 day / week / developer) was very interesting. Conclusion was that, "some tools a radically better" and "deep process renewal depends heavily on tools renewal".
"Continuous Code Inspection" talk explained use of industrial C++ coding standard and normative coding convention, with dedicated analysis tools. After explanation on those rules and tooling, a dedicated jenkins plugin was demonstrated. Such jenkins integration makes QA mostly a single checkbox to enable, and provide history graphs, reports, and external tools integration. Introduction was a little slow imho but content was demonstrating the power of jenkins plugin model to adapt software factory to specific industrial needs.

Coffee break with delicious Danish chocolates …​
Sony was presenting its "Huge Jenkins Cluster", with 4 controllers, some of them handling up to 6000 jobs, 300 agents, 7000 builds a day an executing 175,000 tests a day for android platforms. Development teams use a dedicated agent machine with android devices connected through USB.
Such a build farm requires a dedicated support team and monitoring/maintenance tooling. IT only provides the computer and maintain the OS, but all Jenkins stuff is under the hands of a dedicated team. They evaluate plugins and core upgrades, educate teams, and analyse errors.
With 45Gb for a single full android build, they have to monitor available disk space, and developed maintenance scripts to delete old build artifacts and cleanup /tmp. They also use a local git mirror to speed-up cloning, and integrated with CFEngine-managed infrastructure to ensure no update occurs as a agent is running a build. They also significantly optimized build speed by switching from NFS to SAN, and are now evaluating XFS.
Remaining issue is about jenkins build queue (subject discussed on Jenkins Code Camp) because a 9 in the morning, thousand users connect to jenkins controller and the UI widget to expose the queue status hits the queue synchronisation bottleneck.
Next talk was mine, exposing Jenkins Enterprise and demonstrating one ouf our Enterprise plugins. I’m not pleased by my talk, both because my english is crappy (maybe you already noticed?) and also because I was not confident with the standard JE slides. Assuming I had more time to prepare this talk, and as a tribute to this Danish event, I’d have used a bunch of lego bricks pictures to present Cloudbee plugins. So I quickly left the slides to run a demo, setting-up Jenkins Enterprise to run pre-tested commits. Hope you enjoyed the talk.
Last talk was Lars one, exposing praqma "_Corporate approach to opensource". This light, generalist talk was welcome as last one after a heavy-technical afternoon. Lars exposed reason to switch to open-source :

This last point distinguish "innovators" that create new content and contribute to the oss project, and "free riders" that only want to save money and consume other efforts. Lars didn’t went deeper into what "contributing" can be about, but spending some time joining the mailing lists, exposing detailed bug report, or writing blogs or documentation about the issues you encounter is already a huge contribution to opensource. Organizing such a great user event also is ;)
Meeting ended with a "socialize" time, sponsored by Pragmatic Research, with beer and sandwiches. A nice time to discuss with speakers, know a face to match an #irc nickname, discuss about everything geeks like to discuss about, and round off this pleasant day.