Before starting anything around, I have to say that I’m not a developer, not saying that I don’t understand the meaning but I am not doing every day the job to produce code.

I’ve been part of product development in my career.

So first let’s start with the definition of each part of a CI/CD pipeline.

What is SCM Source Control Management:

It is an important part of the CI/CD pipeline.

The SCM permit to track and record all code changes by each developer then merge them.

  • All the automation pieces are using SCM:
  • CI get the codes from SM
  • SCM will notify the CI server when code needs to be built.

We will use Git and Github to manage our source code

What is Git and Github:

Git is an Open Source Distributed Version Control System.
  1. Control System: This basically means that Git is a content tracker. So Git can be used to store content — it is mostly used to store code due to the other features it provides.

  2. Version Control System: The code which is stored in Git keeps changing as more code is added. Also, many developers can add code in parallel. So Version Control System helps in handling this by maintaining a history of what changes have happened. Also, Git provides features like branches and merges, which I will be covering later.

  3. Distributed Version Control System: Git has a remote repository that is stored in a server and a local repository that is stored in the computer of each developer. This means that the code is not just stored in a central server, but the full copy of the code is present in all the developers’ computers. Git is a Distributed Version Control System since the code is present in every developer’s computer. I will explain the concept of remote and local repositories later in this article.

Why a Version Control System like Git is needed ?

In the real-life… projects generally have multiple developers working in parallel.

A version control system like Git is needed to ensure there are no code conflicts between the developers.

Additionally, the requirements in such projects change often. A version control system allows developers to revert and go back to an older version of the code.

Finally, sometimes several projects which are being run in parallel involve the same codebase. The concept of branching in Git is very important.

We need to know what are the basics of a Git-based workflow
This workflow is divided by steps that as to be done in an order to make changes to the source code

  • Cloning
  • Adding
  • Committing
  • Pushing
  • Branching
  • Merging
  • Pull Requests

Let’s start by installing GIT
Git repo is available at https://git-scm.com/downloads
Lets install Git in your Linux server:

$sudo apt install git      OR
$yum install git
>then run git config to configure your username email adress globally
$git config --global user.name "your username"
$git config --global user.email [email protected]
      

We need now to configure our private ssh key access

ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /user/.ssh/id_rsa.
Your public key has been saved in /user/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:ssz5u7bGn3VKYbkAs0FBIFFQCKSbox3BR73dQVtHenk
The key's randomart image is:
+---[RSA 4096]----+
| .o.=B+o+o. ..o  |
| o .....  .o o . |
|. o .  o+.... o E|
| o o  . .=.  o . |
|+ .   . S . +    |
|.o . o +   o o   |
|. .   =.    + .  |
|       .+  + o   |
|       o==o .    |
+----[SHA256]-----+
cat /user/.ssh/id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDhsZX0Ru9xmVd77e3UJhZwMQ6mUC+mXe4v0yT+hH2sQSbSbCJPP8kWOHUwOnN4xZwxXikDWQY2Bbj/37egXIK/W4BBU0nd8txBA9Xl58b0FkoC54X2LcXwoM5my6GcxUcvyqeSBAJGIJ51nn9np1Bqtq6PVdg3qW+bWFSmcb5z9vgdgsfdgsdgsdfsdfhsfd7hARCX+d4NFTFCYS8DWeBwHtx02M2WmfOuPMjW381Wqoyu4XwSpadvnfdjinfibnsdibnsdivnfdinsoapeofkTDEVASDFFDGpeofkfpovmoijoimiojompompovmpo/nk6w5kRqo9leEj9mVUkvpeyAdAU2GgSiNwaMTOSW9MJqCjX8hcJw1gfhAi3nC0pmPLrzUGYN7DA8zgSFUHswI9TgM3Q2KJwA8Sk3Er/QdpoHoqSifVFo3Bq/Ypg2m4Olji0qUpqZTAmu7YklDKhJCTtAIiiR7jnTbNLtxJ1hqJAgA2R/23gaDRvDQ79/VSeEBaZvJ0tmszVcad9hfXlWQepcCHM2ud6zTFyfsL7EVfLZgZnq5FKw1KeJNzTg+vmGfqHxGH/CeABZV+yxqm0ONgttBJFCX1BxvIRObD7PCoMYV7ugkdwy1ZbMRxiZUAB6eDAYrpobAPpRytz5FVdCY2C6dSXYmsiC65TohbjveJppK2RTCw1wPvVN7CYS1EBDX07MmgFYuOuNP6ACvGZAIePny21a7a6uS+4QPsL7osvCZuQNaO/N8AwOxAy5VkTAWplS+OBjnTXw==
     

Open your GitHub account and add the public SSH key in the settings:


#git config

Setting your email address for every repository on your computer

How to fork code from Github

Open your GitHub account and browse a repo you would to Fork in your personal system: example here is coming from ReanCloud a development cookbook.
It is a documentation example that permit to easilly understand the Git methodology:

https://github.com/reancloud/cookbook_development#TestDrivenDevelopement

And make a Fork of the repository


The Fork creates a personal copy of the repository for my user account in Github.

How to make changes to the source code using Git.


Using Git clone, status adds, commit and push.

#git clone

  • On your local system, first thing clone the source code coming from https://github.com you obtain a local version of the code.
#git clone
example:#git clone [email protected]ithub.com:/user/EBS_Volume_Backup
example:#git clone https://github.com/FEncrenaz/cicd-pipeline-train-schedule-git
Cloning into 'EBS_Volume_Backup'...
Warning: Permanently added the RSA host key for IP address '140.82.118.3' to the list of known hosts.
Enter passphrase for key '/root/.ssh/id_rsa': 
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 7 (delta 2), reused 6 (delta 1), pack-reused 0
Receiving objects: 100% (7/7), done.
Resolving deltas: 100% (2/2), done.

#ll
drwxr-xr-x  3 user user     4096 janv. 17 10:41 EBS_Volume_Backup/

Another important command to know in Git is #git status
#Git status gives the overall information about the changes in the files of the repository.
Run #git status in the directory where you are working on the repo.

#git add

Stages the changes files for the next commit, this command allows controlling which files are committed and which are not.

#git add
#git add -A

$touch toto.txt
$git add .
$git status
On branch master
Your branch is up to date with 'origin/master'.
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)
	new file:   toto.txt

#git commit

Commit adds your changes to your local repository and not pushing on the remote repository on Github.
Commit will only commit the changes that were stages using #git add

git commit -m "<adding toto txt file>"
"Could open a ext editor to add comment on the modification done"
[master 50f8d72] <adding tot txt file>
 2 files changed, 2 insertions(+), 2 deletions(-)
 create mode 100644 toto.test

#git push

Push the changes done locally to the remote repository like Github.
That you don’t push the modification they are only available in local

$git push
Username for 'https://github.com': user
Password for 'https://[email protected]': 
Counting objects: 5, done.
Delta compression using up to 4 threads.
Compressing objects

Changes are committed in the Github repository

Branches and Tags

#git branches

Branches are used to maintain multiple versions of the code with different changes simultaneously.

Different teams use branches as part of the daily workflow.

Branches can also be used to interact with a CI/CD Pipeline.
Some teams could maintain a “production” branch, and merging changes into this brach initiate automated processes involved with deploying to production.

By default, you start with a master branch.

#git checkout

#git checkout checks out an existing branch.
This means that it puts the contents of the branch into your working tree and your working copy of the source code files
when you commit, whatever branch you have checked out will be the branch that the commit is added.

You are able to create a new branch and check it out immediately with -b flag.

#git tag

Tags are pointers to a particular commit,
It can be used to provide a name that can be used as a reference to committing in the future.

The use case is to tags source code commits with the version of the software you are deploying, eg. v0.0.1 could be tagged as “v.0.0.1” that an indicator.

What are Pull Requests in Github?

As explain teams work using multiple branches to manage all their changes, at some points, the branches need to be merged together (asap).

Merges can be handled locally on the development server, but another technic is to use pull requests.

A pull request is a request made by a developer to merge their changes into another branch and give to other team members a chance to review the changes before running the merge.

In Github in your repository click on BRANCHES then select NEW PULL REQUEST it’s doing a check between your own personal fork your branches and the master, and give the opportunity for the members to review and comment the changes.
Then the merge between the personnel branch and the master.

What is Build Automation

Build Automation covers the automation of tasks needed in order to process and prepare source code for deployment to production.
It is an important component of Continuous Integration.
It covers:

– Compiling.
– Dependency Management.
– Executing Automated Tests.
– Packaging the App for Deployment.

Introducing Gradle

We have now implemented an SCM solution, the next step in building a CI/CD Pipeline. This will introduce another tool Gradle, to implement build automation.

You can use some alternatives like Apache Ant Apache Maven (java).

Installing Gradle

Find installation information here:
https://docs.gradle.org/current/userguide/installation.html

On my Ubuntu server:

$ wget -O ~/gradle-6.1-bin.zip https://services.gradle.org/distributions/gradle-6.1-bin.zip
$ sudo apt install default-jdk
$ mkdir /opt/gradle
$ unzip -d /opt/gradle/ ~/gradle-6.1-bin.zip
$ vim /etc/profile.d/gradle.sh
export PATH=$PATH:/opt/gradle/gradle-6.1/bin
#logout then logon of your session
$ graddle --version
#create a project
$ mkdir /user/myproject
$ gradle wrapper
$ ~/myproject# ./gradlew build

The Gradle Wrapper

The wrapper is a script that invokes a declared version of Gradle, downloading it before and if necessary.
Gradle wrapper is useful:
  • It removes the need to have Gradle installed beforehand in order to run the build.
  • Ensures the project is always built with a specific version of Gradle
  • Lets you build multiple projects with different Gradle versions on one system
  • Anyone or process can run the build quickly and easily, Java is needed

Using Gradle

let's start Gradle build is defined in a groovy script called build.gradle located in the root directory of your project.
Use #gradle init to initialize a new projects.this also sets up the gradle wrapper for you.

Gradle Build

Gradle builds consist of a set of tasks that you can call from the command line


./gradlew Tasks andOtherTasks


This command will run a task named Tasks andOtherTasks.

./gradlew sayHello

Task dependencies

Gradle can depend on one another include the concept of dependencies order and determine which task can run after the other
Task1.dependsOn task2

Gradle Plugins

Plugin is nothing but a set of tasks, almost all useful tasks such as compiling tasks, setting domain objects, setting up source files, etc are handled by plugins. Applying a plugin to a project means that it allows the plugin to extend the project’s capabilities.

~/myproject# vim build.gradle
plugins {
  id 'com.moowork.node' version '1.2.0'
}
task sayHello  {
    println 'Hello, World!'
  }
task anothertasks {
println 'Hello another Task'
}
sayHello.dependsOn anothertasks
:~/myproject# ./gradlew sayhello
~/myproject# ./gradlew sayhello
> Configure project :
Hello, World!
Hello another Task
BUILD SUCCESSFUL in 618ms

Automated Testing

Automated testing is the automated execution of tests that verify the quality and stability of code.
Automated tests are usually coded themselves, so they are code that is written to test other code.
Automated tests are often run as part of the build process and are executed using build tools like Gradle.

What is Continuous Integration (CI)?

Continuous integration is the practice of merging code frequently.

Frequent merges cause always major difficulties:

– What if two the code doesn’t compile after the merge.
– What if one the developers broke something.
– What if the codes change and if the merge combines the code no longer compile.
-Fix every merge can take a long time and create more problems than expected.
-Who going to compile the code to be sure that the code is still working.

Fixing these issue is by introducing A CI server who execute a build of the code and automatically prepares/compiles the code and runs automated tests.
The CI server automatically detects code changes in source control and runs the build whenever there is a change.
When something went wrong the build fails. The development teams get immediate feedback if their change broke something.

Introducing Jenkins Server a as CI solution https://jenkins.io/
Résultat de recherche d'images pour "jenkins""


How to install Jenkins


Documentation can be find @ https://jenkins.io/doc/pipeline/tour/getting-started/
https://jenkins.io/doc/book/installing



What you’ll need

The only things you’ll need are a working instance of Ubuntu 18.04 and a user account with Sudo privileges. With those in hand, let’s make some automation magic.

Installing Java

Jenkins requires Java to function, but Jenkins won’t work with just any release of Java.
Here’s how to install a version of Jenkins that will make the installation and usage of Jenkins possible:

  1. Open a terminal window on your Ubuntu server.
  2. Issue the command sudo apt install default-jdk-headless.
  3. Allow the installation to complete.

To verify Java has been installed, issue the command:

java --version

You should see that Java has, indeed, been installed


Install Jenkins

  • Download and install the necessary GPG key with the command.
$wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add - 
  • Add the necessary repository with the command

$sudo sh -c ‘echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list’

Add the universe repository with the command

$sudo add-apt-repository universe 
  • Update apt with the command
$sudo apt-get update
  • Install Jenkins with the command
$sudo apt-get install jenkins -y
  • Allow the installation to complete

How to access Jenkins

Open a web browser and point it to http://SERVER_IP:8080 (where SERVER_IP is the IP address of the hosting server). You will then be prompted to copy and paste a password that was created during the Jenkins installation. To retrieve that password, go back to the terminal window and issue the command:

cat /var/lib/jenkins/secrets/initialAdminPassword

Paste that password into the required field and click Continue

Once you’ve unlocked Jenkins, you can then finish the installation by either installing the suggested plugins or installing only the plugins you want – Install suggested plugins.

Getting started

Setting up a Jenkins Projects

The configuration which controls what a piece of Jenkins automation does and when it executes is called a project.
A simple way is to start with the freestyle project, we will build a CI build for

https://github.com/vivekjuneja/jenkins-ci-sample-project

I select this project making a local clone and load it, Jenkins, as a freestyle project after several modifications in the pom.xml the web site starts to work installing maven than eclipse, giving me the time to play with all these tools.

Triggering Builds with Git Hooks


One of the most important aspects of a good CI process is quick feedback whenever there is a change.
This means that it is important to execute builds as soon as possible after a code change is pushed to source control.
One of the best ways to do this with GitHub and Jenkins is to use webhooks to have GitHub notify Jenkins when there is a change so that Jenkins can automatically start the build.

Using Webhooks

Webhooks are event notifications made from one application to another over http.

In Jenkins, we can use webhooks to have Github notify Jenkins whenever the code in Github changes Jenkins can respond by automatically running the build to implement any changes.

We can configure Jenkins to automatically create and manage webhooks in Github as necessary.

How to setup Github webhooks in Jenkins.

  • Create an access token in Github that has permission to read and create webhooks.
  • Add a Github server in Jenkins for Github.com.
  • Create a Jenkins credential with the token and configure the Github server configuration to use it.
  • Check “Manage Hooks” for the Github server configuration.
  • In the project configuration, under “Build Triggers”, select “Github hook trigger for GITScm polling”.

Log into your Github account in the right corner select your account and click on Settings.

Select Developer Settings
Create a new

Add the key credential in Jenkins in Configure tab.

Jenkins Pipeline Stages and Steps

The basic structure of a Jenkins pipeline consists of stages and steps.
You need to be familiar with these concepts in order to build your own Jenkins pipelines.
Pipeline Stages are large pieces of the CD process.

For Example:

  • Build the code
  • Test the code
  • Deploy to Staging
  • Deploy to Production
  • Wait for input from a human

Steps are implemented through special declarative keywords in the Jenkins DSL.
Jenkins plugin can add new steps.

When building Jenkins pipelines, you may need to consult the steps reference: https://jenkins.io/doc/pipeline/steps/

To be continued.