Jenkins is the way to scale your CI/CD pipelines
Shipments At Scale
Building and maintaining dozens of microservices in a continuous deployment fashion with ease.
Background: We needed to have a strategy in place that allowed us to decouple and scale our CI/CD pipelines for the solution we were developing because it consisted of several microservices. This was critical because the pipelines were independent of each other and their external resources. We didn’t want them to be flaky while integrating with our current source code repository. So, we decided to use Jenkins and take advantage of its master/agent architecture in a cluster orchestrated by Kubernetes.
Goals: Managing luxury fashion shipments at scale.
"Using Jenkins, we were able to take our CI/CD pipelines and processes to a whole new level. We’re now building and maintaining dozens of microservices in a continuous deployment fashion with ease."
— Sérgio Martins, Test Automation Engineer
Solution & Results: To accomplish our goals, we used Jenkins integration with Kubernetes, our cluster orchestrator. Taking advantage of its architecture, we could quickly deploy new nodes as pods during runtime to isolate each pipeline, making it focused, fast, and reliable. We also had a set of flaky end-to-end tests in our suite that we were looking to stabilize. After migrating to Jenkins, that improved tremendously, mainly because our testing could now be executed in an isolated pod in a separate Kubernetes namespace, using the required resources dynamically.
The key capabilities we relied on were the Kubernetes and Docker plugins. We also relied heavily on the current portability, documentation, and support we found within the considerable Jenkins community. Cool side note: as it is such a portable solution, even Jenkins runs on the Kubernetes pod. We didn’t even need to purchase new virtual machines in the cloud. That means resource (€€€) efficiency!
Long story short, we managed to accomplish our goals using Jenkins, which was surprisingly a game-changer for us. Because we’re using Jenkins CI/CD infrastructure, everything is infrastructure-as-code. Now, we can share multiple libraries to support our pipelines-as-code across several microservices. Using Jenkins, we could take our CI/CD pipelines and processes to a whole new level. We’re now building and maintaining dozens of microservices in a continuous deployment fashion with ease. And we’re experiencing great results:
-
Deployments are now performed 10x times faster at a larger scale
-
End-to-end tests are currently around 4 to 6x times faster
-
Our CI/CD infrastructure is as IaC (Infrastructure as Code)
-
We’re able to share multiple libraries across several microservices