The Jenkins Blog

scalability

A sustainable pattern with shared library

Table of Contents Context The Problems The Solution Shared Library Duplication Documentation Scalability Installation Agnostic Feature Toggling This post will describe how I use a shared library in Jenkins. Typically when using multibranch pipeline. If possible (if not forced to) I implement the pipelines without multibranch. I previously wrote about how I do that with my Generic Webhook Trigger Plugin in a previous post. But this will be my second choice,...

Tomas Bjerre October 21, 2020
Generic Webhook Trigger Plugin

Table of Contents The Problem Code Duplication And Security A Branch Is Not A Feature Documentation The Solution Code Duplication And Security A Branch Is Not A Feature Documentation This post will describe some common problems I’ve had with Jenkins and how I solved them by developing Generic Webhook Trigger Plugin. The Problem I was often struggling with the same issues when working with Jenkins: Code duplication and security - Jenkinsfiles in...

Tomas Bjerre December 14, 2019
Scaling Network Connections from the Jenkins Controller

Oleg Nenashev and I will be speaking at DevOps World | Jenkins World in San Francisco this year about Scaling Network Connections from the Jenkins Controller. Over the years there have been many efforts to analyze, optimize, and fortify the “Remoting channel” that allows a controller to orchestrate agent activity and receive build results. Techniques such as tuning the agent launcher can improve service, but...

Jesse Glick September 10, 2018
Project Cheetah - Faster, Leaner Pipeline That Can Keep Up With Demand

Table of Contents Introducing "Project Cheetah" Yes, but what does it DO? How Do I Set Speed/Durability Settings? 1. Globally, you can choose a global default durability setting: 2. Each Pipeline can get a custom Durability Setting: 3. Multibranch Projects can use a new BranchProperty to customize the Durability Setting. Will Performance-Optimized Mode Help Me? Other Goodies How Did You Do It? What Next? Since it launched, Pipeline has had a...

Sam Van Oort February 22, 2018
Best Practices for Scalable Pipeline Code

This is a guest post by Sam Van Oort, Software Engineer at CloudBees and contributor to the Jenkins project. Today I’m going to show you best practices to write scalable and robust Jenkins Pipelines. This is drawn from a combination of work with the internals of Pipeline and observations with large-scale users. Pipeline code works beautifully for its intended role of automating build/test/deploy/administer tasks. As...

Sam Van Oort February 1, 2017
Tuning Jenkins GC For Responsiveness and Stability with Large Instances

This is a cross post by Sam Van Oort, Software Engineer at CloudBees and contributor to the Jenkins project. Today I’m going to show you how easy it is to tune Jenkins Java settings to make your controllers more responsive and stable, especially with large heap sizes. The Magic Settings: Basics: -server -XX:+AlwaysPreTouch GC Logging: -Xloggc:$JENKINS_HOME/gc-%t.log -XX:NumberOfGCLogFiles=5 -XX:+UseGCLogFileRotation -XX:GCLogFileSize=20m -XX:+PrintGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCCause -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC -XX:+PrintAdaptiveSizePolicy G1 GC...

Sam Van Oort November 21, 2016
Jenkins Pipeline Scalability in the Enterprise

This is a guest post by Damien Coraboeuf, Jenkins project contributor and Continuous Delivery consultant. Implementing a CI/CD solution based on Jenkins has become very easy. Dealing with hundreds of jobs? Not so much. Having to scale to thousands of jobs? Now this is a real challenge. This is the story of a journey to get out of the jungle of jobs… Start of the journey At...

Damien Coraboeuf June 15, 2016