Jenkins is the way to build industry-leading log management
Building a Flexible and Modern DevOps Platform
Adding more flexibility and power into snapshot and release builds.
Background: Donald Morton was hired by Graylog in early 2020 and was given charge of their Jenkins installation. Graylog is a log management software company that centrally captures, stores, and enables real-time search and log analysis against machine data from any component in IT infrastructure and applications. The infrastructure was well designed, but it also hadn’t been changed in about four years, so it was time for an upgrade.
Goals: More flexibility and power in our snapshot and release builds. Specifically, the goals were:
-
Update Jenkins to the latest version so we can take advantage of the latest features.
-
Use infrastructure-as-code as much as possible. Eliminate manual configuration. We should be able to destroy the VM and recreate it the same as it was before.
-
Rewrite the jobs as declarative pipelines so that we can start to simplify and improve our complex builds.
"Jenkins' flexibility is its real power. When you have complex needs like we do, you need something that can handle anything you can throw at it."
— Donald Morton, Build and Release Manager, Graylog
Solution & Results: We started out writing a Terraform module to spin up infrastructure and an Ansible playbook for installing software on the machines. Jenkins itself runs off of a Docker image. The configuration of Jenkins is handled through the JCasC plugin. **
The freestyle jobs were rewritten in Groovy as declarative pipelines. We set up the Github Branch Source plugin, allowing Jenkins to auto-create jobs. We store the Groovy for the pipelines in git, allowing us to back out changes to jobs if something goes wrong. We did this because Declarative pipelines allow us to split our builds up into multiple stages and we have a shared library so that all our repos could take advantage of the same pipeline logic.**
We just rebuild the Docker image, and it automatically updates Jenkins to the latest LTS and updates all plugins to the latest versions. It only takes a few minutes. The configuration of Jenkins itself is done through a single YAML file.**
Capabilities we used:
-
Declarative Pipelines
-
Jenkins Shared Library
-
Jenkins Docker image
-
Jenkins Configuration-as-Code plugin
-
Github Branch Source plugin
-
Blue Ocean UI
Results so far:
-
Maintainability of job improved through infrastructure-as-code
-
Upgrade time went from never upgraded at all to upgrades within minutes
-
Observability of builds increased through Blue Ocean UI
Want the full story, visit Donald Morton’s Graylog Case Study.