Background: Working alongside a team of students from NIT Trichy, I sought to create a CI/CD pipeline for the tuning of hyperparameters of a machine learning model using DevOps tools.
Goals: Optimize the integration of DevOps tools with machine learning for autotuning.
"Jenkins accelerates the phases of testing and helps optimize development and production efficiently."
Solution & Results: When a developer commits a code to create a machine learning model, we deploy webhooks to trigger the Jenkins job as soon as that code is committed on GitHub. In this way, we are able to create a suitable environment for the training of our model. We then can tune hyperparameters to obtain the best possible accuracy for our model.
Using Jenkins, the creation of pipelines is so easy. We also added a code for assembly lines in Groovy to reduce the human intervention to as little as possible. By doing so, we can increase the efficiency and speed up the process of building projects like this.
Jenkins has excellent plugins and a great community that works continuously to make improvements and to make the tool better every day.
As Jenkins is a complete CI/CD pipeline, Jenkins accelerates the phases of testing and helps optimize both development and production efficiency. Topline results included:
Ability to create an environment to complete required tasks
Automation of those tasks using Jenkins jobs
Accuracy of the model obtained by running these jobs
Resource overhead was significantly reduced due to its serverless architecture
Ability to monitor jobs and other things in Jenkins