Jenkins, From Zero To Hero: Become DevOps Jenkins Master - UDEMY

Freestyle Job:

 

Job Name - my-first-job

 

1.Job Settings

 

2.Configure:

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
					Add Build Step -> Execute Shell -> Command: echo Hello World

 

 

3.Build Now

 

In the lectures, he just explained bash using $1 and $2 which is parameter. It is the normal bash script, it is just to show how it is done in Linux. He kept the script in server which requires parameter.

 

Freestyle Job Parameter:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
	Select the check box "This project is parameterized" -> Add Parameter -> String Parameter Name: FIRST_NAME Default Value: Ricardo Giving another String Parameter Name: SECOND_NAME Default Value: Andre				Add Build Step -> Execute Shell -> Command: echo "Hello $FIRST_NAME, $SECOND_NAME"

Now we don’t get Build button instead we get Build with parameters, by clicking it we will be given with input

 

 

Freestyle Job Choice Parameter:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
	Select the check box "This project is parameterized" -> Add Parameter -> String Parameter Name: FIRST_NAME Default Value: Ricardo Giving another String Parameter Name: SECOND_NAME Default Value: Andre Click "Add Parameter" and in the options choose "Choice" Name: LASTNAME Choices: Smith Gonzalez Doz				Add Build Step -> Execute Shell -> Command: echo "Hello $FIRST_NAME, $SECOND_NAME, $LASTNAME"

 

With choice parameter we get Drop Down of values, in our case Smith, Gonzalez, Doz

 

Freestyle Job Choice Parameter:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
	Select the check box "This project is parameterized" -> Add Parameter -> String Parameter Name: FIRST_NAME Default Value: Ricardo Click "Add Parameter" and in the options choose "Choice" Name: LASTNAME Choices: Smith Gonzalez Doz Add "Boolean Parameter" Name: SHOW Default Value: Check this box				Add Build Step -> Execute Shell -> Command: /tmp/script.sh $FIRST_NAME $LASTNAME $SHOW

 

When Boolean Parameter is checked it will be "true", if not marked it will be "false"

 

In /tmp/script.sh, he had a IF condition checking true:

 

if [ "$SHOW" = "true" ]; then

 

 

Jenkins Security - to disable login:

 

Go to Manage Jenkins -> Configure Global Security -> Uncheck the "Enable Security"

 

Jenkins Security - Sign Up:

 

Configure Global Security	Security Realm	Authorizations
	1.Choose "Jenkins own user database" radio button 2.Choose the check box "Allow users to sign up"

 

Now you will get Create an account

 

Jenkins Security - Install a Plugin - Role Based Authorization Strategy Plugin:

Configure Global Security	Security Realm	Authorizations
		Role Based Strategy

 

After this you will find Manage and Assign Roles under Manage Jenkins

 

User and role:

Manage Jenkins -> User

Manage Jenkins -> Manage and Assign Roles

We can create new role and we can define permission for various access like Overall, credentials, Agent, Job, run, View

 

Role to execute job:

This role will allow to build.

Manage Jenkins -> Manage and assign roles -> Manage Roles

1. Create new role with name execution

Overall	Job
Read	Build and Read

 

Now you will get "Build" or "Build with Parameter" job

 

 

Restrict certain user to certain job:

Giving access based on pattern. For example: Giving access to job starting with word "Ansible"

1. In Manage Roles, Create new role dev

 

In Global Role

Overall	Job
Read

 

In Project Roles

Role to Add -> Ansible

Pattern -> ansible-.*

 

Role	Pattern	Job
Ansible	ansible-.*	Build, Read

 

2. In Assign Roles

 For the user add ITEM ROLES as Ansible

 

 

Jenkins Environment Variable - Creating our Own:

Will be useful when sending email. You can search in google and see in documentation.

 

Manage Jenkins -> Configure System -> Global Properties -> Environment Variables

 

Modify Jenkins URL

Manage Jenkins -> Configure System -> Jenkins Location -> Jenkins URL (It should match DNS)

 

Build Trigger - CRON:

Build Periodically -> Schedule

 

***** -> Means every single minute

 

It will have the message "Started by Timer"

 

Trigger job externally from outside of Jenkins - Bash:

1.Create User Jenkins

2.Add Global Role -> Give Read and Build permission

 

 

To get the URL to trigger from BASH

1.Copy the link for "Build Now"

2.Conifgure Global Security -> CSRF Protection -> Check the CRUMB

 

To get the Token to pass with CURL

crumb=$(curl -u "jenkins:1234" -s 'http://jenkins.local:8080/crumbIssuer/api/xml?xpath=concat(crumbRequestField,":",//crumb)')

 

To Trigger the jenkins:

The URL below is copied directly from Jenkins "Build Now". For "Build With Parameters" follow below example

curl -u "jenkins:1234" -H $crumb -X POST http://jenkins.local.8080/job/backup-db-to-aws/buildWithParameters?DB_HOST=db&DB_NAME=db_name

 

 

Jenkins Maven - FreeStyle Project:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced

 

Building JAR using Maven:

Manage Jenkins -> Global Tool Configuration -> Maven -> Name -> Install from Apache

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package

 

 

Testing using Maven:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test

 

His test were failing, to stop failing, he added an option _JAVA_OPTIONS= -Djdk.net.URLClassPath.disableClassPathURLCheck=true in Environment variable.

 

Deploying the JAR:

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>

 

 

Dashboard - Graph for test result:

The target folder will have the test result in form of XML. Example: target/surefire-reports/*.xml

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0

 

 

Archive the successful artifact:

 

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0 Archive the artifacts Files to archive: target/*.jar

 

The jar will appear in the Jenkins Page itself

 

Email - mailer plugin:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL:GITHUB SAMPLE PROJECT Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0 Archive the artifacts Files to archive: target/*.jar Add Email Notification: Recipients: vk@ok.com Select -> Send e-mail for every unstable build, Send separate e-mails to individuals who broke build

 

 

Jenkins With GIT - Using Authentication:

 

Go to Jenkins -> Credentials

 

Stores Scoped to JENKINS:

Jenkins -> Global Credentials -> Add Credential -> Kind: Username with Password

 

Scope:

Username
Password

ID

Description

 

Go to Maven Job and configure:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL:GITLAB URL Credentials: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0 Archive the artifacts Files to archive: target/*.jar Add Email Notification: Recipients: vk@ok.com Select -> Send e-mail for every unstable build, Send separate e-mails to individuals who broke build

 

 

GIT Hooks

In GITLAB Server, where GIT is running we will have the repository with the directory name same as .git. In the lecture we used maven.git, so go inside maven.git/custom_hooks and write below code. Name the file as post-receive

 

The custom_hooks and post-receive are special directory and file, which denotes hooks

 

#Get branch name from ref head

if ! [ -t 0 ]; then

read -a ref

fi

IFS='/' read -ra REF <<< "${ref[2]}"

branch="${REF[2]}"

 

 

if [ $branch == "master" ]; then

crumb=$(curl -v "jenkins:12345" -s 'https://jenkins:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')

curl -u "jenkins:12345" -H "$crumb" -X POST http://jenkins:8080/job/maven/build?delay=0sec

 

if [ $? -eq 0 ]; then

echo "**** OK"

else

echo "**** ERROR"

fi

fi

 

Jenkins DSL Plugin - SEED Job using "Use the provided DSL Script"

This is to create job using code.

 

Create new freestyle job -> job-dsl

 

Note: the extension is .j2

 

Configuration to create empty job:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
					Add build Step: Process job DSL Choose: Use the provided DSL script   job('job_dsl_created'){}

 

After building this job, it will create a job named "job_dsl_created" with empty configuration.

 

 

DSL to create job with description:

 

Put below DSL in seed job configuration, in Build section.

 

job('job_dsl_example'){

description('This is my awesome job')

}

 

DSL to create Parameters:

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

}

 

DSL to add GIT / SCM

 

 

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

}

 

DSL to add Trigger / CRON

 

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

 

triggers{

cron('H 5 * * 7')

}

}

 

 

DSL to add Shell Script or Steps

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

 

triggers{

cron('H 5 * * 7')

}

 

steps{

shell("echo 'Hello World'")

}

}

 

Shell with multi-line

 

Use the triple quote

shell("""

echo 'Hello World'

echo 'Running Script'

""")

 

DSL to add mailer

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

 

triggers{

cron('H 5 * * 7')

}

 

steps{

shell("echo 'Hello World'")

shell("echo 'Hello World2'")

}

 

publishers{

mailer('me@example.com', true, true)

}

}

 

 

DSL for Ansible:

 

parameters{

choiceParam('AGE', ['21', '22', '23'])

}

 

steps{

wrappers {

colorizeOutput(colorMap = 'xterm')

}

 

ansiblePlaybook('/etc/ansible/plays/some_playbook.ym'){

inventoryPath('/etc/ansible/plays/hosts')

tags('cool')

forks(1)

colorizeOutput(true)

additionalParameters('--vault-password-file $HOME/pass-vault/i2b-cl.txt')

extraVars{

extraVar("PEOPLE_AGE", '${AGE}', false)

}

}

 

}

 

 

DSL for Maven Job:

 

Maven -> Maps to invoke top-level maven target

In SCM -> He has added node -> node, it is to resolve some error.

 

scm{

git('https://github.com/jenkin/maven', 'master', {node -> node / 'extensions' << ''})

}

 

 

steps{

maven{

mavenInstallation('maven-jenkins')

goals('-B -DskipTests clean package')

}

maven{

mavenInstallation('maven-jenkins')

goals('test')

}

shell('''

echo ***********RUNNING THE JAR***************

java -jar /var/jenkins_home/workspace/mvn/target/my-app-1.0-SNAPSHOT.jar

''')        

}

 

publishers {

archiveArtifacts('target/*.jar')

archiveJunit('target/surefire-reports/*.xml')

mailer('vk@ok.com',true, true)

 

}

 

 

Automating DSL Process - Using Hooks:

1. Put the seed job (DSL) in GIT
2. Create directory custom_hooks and the file post-receive. This both are special directory/file for GIT LAB. Inside the post-receive we will have the schell script which invokes Jenkins.
3. Copy the above shell script and just change the curl command to invoke correct job

 

#Get branch name from ref head

if ! [ -t 0 ]; then

read -a ref

fi

IFS='/' read -ra REF <<< "${ref[2]}"

branch="${REF[2]}"

 

 

if [ $branch == "master" ]; then

crumb=$(curl -v "jenkins:12345" -s 'https://jenkins:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')

curl -u "jenkins:12345" -H "$crumb" -X POST http://jenkins:8080/job/maven/build?delay=0sec

 

if [ $? -eq 0 ]; then

echo "**** OK"

else

echo "**** ERROR"

fi

fi

 

4. Now configure the job-dsl (seed Job)

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		GIT: http://gitlab.com/jobs.git Credential -> Choose credential			Choose Look on Filesystem   DSL Scripts: jobs (NOTE: we have defined SCM, so it will have only one file named as jobs)

 

5. Jenkins -> Configure Global Security -> UNCHECK Enable Script Security for JOB DSL Scripts

 

Flow for automated job creation:

1. Create jenkins job-dsl
2. Create GIT repository for storing DSL jobs
3. Create HOOK for the reporsitory and if new DSL is added then trigger the job-DSL which we first created
4. In job-dsl, get the DSL settings from the GIT repository.

 

 

Theory CI/CD

Continuous Integration -> Integration, Functional Testing. Building and Testing your application

Continuous Delivery -> Deploy to Test Env, acceptance Testing. Optional step, deploying to test env

Continuous Deployment -> Deploy/"Promote" to Production

 

 

Pipeline Job - Jenkinsfile - Pipeline Plugin:

Two types - Declarative and Scripted.

 

 

Pipeline Job:

 

General	Build Triggers	Advanced Project Options	Pipeline
			Definition: Paste below Pipeline script

 

Declarative Pipeline

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

echo 'Building ...'

}

}

stage('Test'){

steps{

echo 'Testing ...'

}

}

stage('Deploy'){

steps{

echo 'Deploying'

}

}

}

}

 

Multi-Step Pipeline:

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

sh 'echo "My first pip"'

sh '''

echo "By th way you can do more stuff"

ls -lah

'''

}

}

 

}

 

}

 

 

Retry in Pipeline:

He has purposefully given invalid command. Will retry on failure

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

retry(3){

sh 'I am not going to work :C'

}

}

}

}

}

 

Timeout in pipeline:

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

retry(3){

sh 'echo hello'

}

 

timeout(time:3, unit: 'SECONDS'){

sh 'sleep 5'

}}}}}

 

Environment Variable in Jenkins File:

pipeline{

agent any

 

environment{

NAME='Cristiano'

LASTNAME='Messi'

}

stages{

stage('Build'){

steps{

sh 'echo $NAME $LASTNAME'

}}}}

 

Encrypting password:

 

In below pipeline TEST is the name of credentials defined in Jenkins -> Global credentials -> Kind: Secret Text ->  ID: TEST

pipeline{

agent any

 

environment{

secret = credentials('TEST')

}

stages{

stage('Build'){

steps{

sh 'echo $secret'

}}}}

 

 

Jenkins Post Actions

 

 

pipeline{

agent any

 

stages{

stage('TEST'){

steps{

sh 'echo Fail; exit 1'

}}}

post{

always{

echo 'I will be executed'

}

success{

echo 'I will only executed if this is success'

}

failure{

echo 'I will only get executed if this failes'

}

unstable{

echo 'I will only get executed if this is unstable'

}

}

 

}

 

Docker inside Docker:

It is normal to have docker installation along with Jenkins. Since in this course the Jenkins itself is in Docker, we need docker inside the docker.

 

He uses Dockerfile - It will work only with internet or artifactory: Command commands are

 

RUN

CURL

apt-get

USER

 

Docker Compose:

It is a yml file, build section will have the directory name where we place docker file. We also map docker.cok

 

services:

jenkins:

container_name: jenkins

image: jenkins/docker

build:

context:pipeline

ports:

-"8080:8080"

volumes:

• "$PWD/jenkins_home:/var/jenkins_home"
• /var/run/docker.sock:/var/run/docker.sock

networks:

 -net

 

To build:

docker-compose build

 

To recreate the container

docker-compose up -d

 

Defining steps for pipeline.

Now we have Jenkinsfile and Dockerfile in same directory at same level. Dockerfile has installation steps and Jenkinsfile has Declarative Pipeline script. In same directory he kept "Hellow World" Java-Maven proeject

 

 

Dockerfile

Jenkinsfile

java-app

jenkins/build

 

The objective is to build JAR file in Docker using only the above files. He is going to use Maven alpine image.

 

1. Pull image docker pull maven:3-alpine. Now we are going to build using this image instead of Maven Plugin.
2. Check image

docker images | grep maven

3. Now map the Java volume

docker run -ti -v $PWD/java-app:/app maven:3-alpine sh

 

Now we can see the java-app contents in /app directory inside docker.

4. mvn package
5. Map .m2 directory so that the downloaded files are not lost

docker run --rm -ti -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ maven:3-alpine sh

6. Update the command so that our working directory will be /app where pom.xml is present.

docker run --rm -ti -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven:3-alpine sh

7. Package the code without getting inside the docker image

docker run --rm  -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven:3-alpine mvn -B -DskipTests clean package

8. Now that we know the docker script is working, add the above line in a file mvn.sh in the jenkins/build directory. The $@ will take parameter, so that you can now run any mvn commands.

docker run --rm  -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven:3-alpine "$@"

9. Go to jenkins/build and create Dockerfile-Java

FROM  openjdk:8-jre-alpine

RUN mkdir /app

 

COPY *.jar /app/app.jar

 

CMD java -jar /app/app.jar

 

#Creating Alpine-Java image and copying jar file into it

docker build -f Dockerfile-Java -t test .

 

#Get inside the newly build Java container

docker run --rm -ti test sh

 

#Running docker in detached mode to see the logs

docker run -d test

 

docker logs -f <ID we get from above command>

2. To take to next level we are using docker compose. In the same directory create "docker-compose-build.yml'

 

version: '3'

services:

app:

image: "app:$BUILD_TAG"

build:

context: .

dockerfile: Dockerfile-Java

 

3. Now building using docker compose

docker-compose -f docker-compose-build.yml build

 

cd jenkins/build && docker-compose -f docker-compose-build.yml build --no-cache

 

4. Now putting all things together. Creating build.sh in jenkins/build

 

#Copy the new jar

cp -f java-app/target/*.jar jenkins/build/

 

#Run docker compose

cd jenkins/build && docker-compose -f docker-compose-build.yml build --no-cache

 

5. Adding the script to Jenkinsfile. We just call all the scripts we have written.

pipeline{

agent any

 

stages{

stage('Build'){

 

steps{

sh '''

./jenkins/build/mvn.sh mvn -B -DskipTests clean package

./jenkins/build/buil.sh

'''

}

}

 

}

}