Jenkins is the way to cast magic of continuous delivery

Improve the Velocity of Software Delivery

Submitted By Jenkins User Xuefeng Shi
When you are one of the leading internet companies in China, it's critical to have a solid DevOps platform to ensure flawless delivery. And that's precisely what these developers built using Jenkins: a high availability CD platform that supports thousands of applications.
Organization: JD.com, China’s largest online retailer, www.jd.com
Team Members: All work for JD.com: Xuefeng Shi, Software Engineer | Zheng Gu, Software Engineer | Chuang Zhang, Software Engineer | Jianye Guo, Software Engineer | Ya Shen, Software Engineer
Industries: Internet | e-commerce
Programming Languages: Java, Python, Vue.js
Version Control Systems: Gitlab
Community Support: Relied on Jenkins.io websites & blogs, networking at Jenkins event, and spoke with colleagues and peers

Improving the velocity of end-to-end software delivery with a high availability CD platform.

Background: JD.com is one of the top 5 internet companies in China. We have more than 400 million users. Every two weeks, we deliver a new version of a mobile app to our end-users. Since it is a very complicated app from both a business and technical perspective, we need to ensure flawless delivery. 

Goals: The goal of our team was to build a continuous delivery pipeline to support different technology and align them cohesively with a solid release plan including iOS, Android, Server, H5, React, etc. The platform needed the flexibility as well as high availability to meet our requirements.

"Jenkins is one of the greatest projects in the last 15 years and keeps evolving with fast cadency. Pipeline as code and DSL give us great inspiration while we design our platform."
image— Xuefeng Shi, Software Engineer, JD.com

Solution & Results: To meet our objectives, Jenkins 2.0 pipeline is the preferred solution but there are challenges. On one hand, not everyone in our company is a Jenkins expert. On the other hand, we have a lot of internal services/platforms which become silos inside their own software delivery process. 

Our solution is Atom as a Service (AAAS). An atom is the minimal reusable ability inside the company. For example, a downloadable code from git is an atom since regardless of what kind of technical project it is, they can use the same ability to grab code from git repository. We designed the shared library for each atom service, and used DSL to describe the interface of each atom. In that way, users do not need to be familiar with the implementation of the atom, they only need to drag and drop the atoms through the visual editor within our platform. The cost is significantly reduced by the editor. 

To achieve more flexibility and fully control the process of the pipeline, we designed BPE(bamboo pipeline engine). We leveraged the following Jenkins capabilities within our platform: Jenkins pipeline, the shared library and groovy. 

Now more than 1000 jobs are sharing one single Jenkinsfile and there are only two lines in it – the shared library information and pipeline entry point. All of the workflow is dynamically generated in the runtime. In the meantime, we no longer need to maintain thousands of jobs for each pipeline on the Jenkins master, because one job is enough since all the configuration including job and pipeline are exactly the same. We’ve experienced great software delivery results including:

  • one platform supports thousands of applications with multiple technical stacks

  • reduced release cycles from one hour to 10 minutes 

  • high availability of continuous delivery platform