Sizing a Jenkins environment depends on a number of factors, which makes it a very inexact science. Achieving an optimal configuration requires some experience and experimentation. It is, however, possible to make a smart approximation to start - especially when designing with best practices in mind.
The following outlines these factors and how you can account for them when sizing your configuration. You are also provided sample configurations and the hardwares behind some of the largest Jenkins installations presented in a Jenkins Scalability Summit.
One of the greatest challenges of properly setting up a Jenkins instance is that there is no one size fits all answer - the exact specifications of the hardware that you will need will depend heavily on your organization’s current and future needs.
Your Jenkins controller will be serving HTTP requests and storing all of the important information for your Jenkins instance in its $JENKINS_HOME folder (configurations, build histories and plugins).
More information on sizing controllers based on organizational needs can be found in the Architecting for Scale section.
The amount of memory Jenkins needs is largely dependent on many factors, which is why the RAM allotted for it can range from 200 MB for a small installation to 70+ GB for a single and massive Jenkins controller. However, you should be able to estimate the RAM required based on your project build needs.
Each build node connection will take 2-3 threads, which equals about 2 MB or more of memory. You will also need to factor in CPU overhead for Jenkins if there are a lot of users who will be accessing the Jenkins user interface.
It is generally a bad practice to allocate executors on a controller, as builds can quickly overload a controller’s CPU/memory/etc and crash the instance, causing unnecessary downtime. Instead, it is advisable to set up agents that the Jenkins controller can delegate jobs to, keeping the bulk of the work off of the controller itself.
The backbone of Jenkins is its ability to orchestrate builds, but installations which do not leverage Jenkins' distributed builds architecture are artificially limiting the number of builds that their controllers can orchestrate. More information on a more distributed architecture can be found in the Architecting for Scale section.
Agents are typically generic x86 machines with enough memory to run specific build types. The agent’s configuration depends on the builds it will be used for and on the tools required by the same builds.
Configuring a machine to act as an agent inside your infrastructure can be tedious and time consuming. This is especially true when the same set-up has to be replicated on a large pool of agents. Because of this, is ideal to have fungible agents, which are agents that are easily replaceable. Agents should be generic for all builds rather customized for a specific job or a set of jobs. The more generic the agents, the more easily they are interchanged, which in turn allows for a better use of resources and a reduced impact on productivity if some agents suffer an outage. Andrew Bayer introduced this concept of "fungibility" as applied to agents during his presentation "Seven Habits of Highly Effective Jenkins Users" at the Jenkins User Conference (Europe, 2014).
The more automated the environment configuration is, the easier it is to replicate a configuration onto a new agent machine. Tools for configuration management or a pre-baked image can be excellent solutions to this end. Containers and virtualization are also popular tools for creating generic agent environments.
More information on estimating the number of executors needed in a given environment can be found in the Architecting for Scale section.
Please submit your feedback about this page through this quick form.
Alternatively, if you don't wish to complete the quick form, you can simply indicate if you found this page helpful?
See existing feedback here.