Google Kubernetes

#Instructions from this URL: https://cloud.google.com/sdk/docs/quickstart-debian-ubuntu

# Create environment variable for correct distribution

CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)"

# Add the Cloud SDK distribution URI as a package source

echo "deb http://packages.cloud.google.com/apt $CLOUD_SDK_REPO main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list

# Import the Google Cloud Platform public key

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

# Update the package list and install the Cloud SDK

sudo apt-get update 

sudo apt-get install google-cloud-sdk

#Authenticate our console session with gcloud

gcloud init --console-only

#Create a named gcloud project

gcloud projects create psdemogke --name="Kubernetes-Cloud"

#Set our current project context

gcloud config set project psdemogke

#Enable GKE services in our current project

gcloud services enable container.googleapis.com

#You may have to adjust your resource limits and enabled billing here based on your subscription here.

#Go to https://console.cloud.google.com

#From the Navigation menu on the top left, browse to Compute->Kubernetes Engine.

#Click enable billing. Click Set Account.

#Tell GKE to create a single zone, three node cluster for us. 3 is the default size.

#https://cloud.google.com/compute/quotas#checking_your_quota

gcloud container clusters create cscluster --region us-central1-a

#Get our credentials for kubectl

gcloud container clusters get-credentials cscluster --zone us-central1-a --project psdemogke

#Check out out lists of kubectl contexts

kubectl config get-contexts

#set our current context to the GKE context

kubectl config use-context gke_psdemogke_us-central1-a_cscluster

#run a command to communicate with our cluster.

kubectl get nodes

#Delete our GKE cluster

#gcloud container clusters delete cscluster --zone=us-central1-a 

#Delete our project.

#gcloud projects delete psdemogke

#Get a list of all contexts on this system.

kubectl config get-contexts

#Let's set to the kubectl context back to our local custer

kubectl config use-context kubernetes-admin@kubernetes

#use kubectl get nodes

kubectl get nodes

Azure AKS - Kubernetes

# This demo will be run from c1-master1 since kubectl is already installed there.

# This can be run from any system that has the Azure CLI client installed.

#Ensure Azure CLI command line utilitles are installed

#https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-apt?view=azure-cli-latest

AZ_REPO=$(lsb_release -cs)

echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure-cli.list

sudo apt-key --keyring /etc/apt/trusted.gpg.d/Microsoft.gpg adv --keyserver packages.microsoft.com --recv-keys BC528686B50D79E339D3721CEB3E94ADBE1229CF

sudo apt-get update

sudo apt-get install azure-cli

#Log into our subscription

az login

az account set --subscription "Demonstration Account"

#Create a resource group for the serivces we're going to create

az group create --name "Kubernetes-Cloud" --location centralus

#Let's get a list of the versions available to us, 

az aks get-versions --location centralus -o table

#let's check out some of the options available to us when creating our managed cluster

az aks create -h | more

#Let's create our AKS managed cluster. 

az aks create \

    --resource-group "Kubernetes-Cloud" \

    --generate-ssh-keys \

    --name CSCluster \

    --node-count 3 #default Node count is 3

#If needed, we can download and install kubectl on our local system.

az aks install-cli

#Get our cluster credentials and merge the configuration into our existing config file.

#This will allow us to connect to this system remotely using certificate based user authentication.

az aks get-credentials --resource-group "Kubernetes-Cloud" --name CSCluster

#List our currently available contexts

kubectl config get-contexts

#set our current context to the Azure context

kubectl config use-context CSCluster

#run a command to communicate with our cluster.

kubectl get nodes

#Get a list of running pods, we'll look at the system pods since we don't have anything running.

#Since the API Server is HTTP based...we can operate our cluster over the internet...esentially the same as if it was local using kubectl.

kubectl get pods --all-namespaces

#az aks delete --resource-group "Kubernetes-Cloud" --name CSCluster #--yes --no-wait

Kubernetes Node Setup steps

#For this demo ssh into c1-node1

ssh aen@c1-node1

#Disable swap, swapoff then edit your fstab removing any entry for swap partitions

#You can recover the space with fdisk. You may want to reboot to ensure your config is ok. 

swapoff -a

vi /etc/fstab

#Add the Google's apt repository gpg key

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

#Add the kuberentes apt repository

sudo bash -c 'cat <<EOF >/etc/apt/sources.list.d/kubernetes.list

deb https://apt.kubernetes.io/ kubernetes-xenial main

EOF'

#Update the package list 

sudo apt-get update

apt-cache policy kubelet | head -n 20 

#Install the required packages, if needed we can request a specific version

sudo apt-get install -y docker.io kubelet kubeadm kubectl

sudo apt-mark hold docker.io kubelet kubeadm kubectl

#Check the status of our kubelet and our container runtime, docker.

#The kubelet will enter a crashloop until it's joined

sudo systemctl status kubelet.service 

sudo systemctl status docker.service 

#Ensure both are set to start when the system starts up.

sudo systemctl enable kubelet.service

sudo systemctl enable docker.service

#Setup Docker daemon. This is a change that has been added since the recording of the course.

sudo bash -c 'cat > /etc/docker/daemon.json <<EOF

{

  "exec-opts": ["native.cgroupdriver=systemd"],

  "log-driver": "json-file",

  "log-opts": {

    "max-size": "100m"

  },

  "storage-driver": "overlay2"

}

EOF'

sudo systemctl daemon-reload

sudo systemctl restart docker

#If you didn't keep the output, on the master, you can get the token.

kubeadm token list

#If you need to generate a new token, perhaps the old one timed out/expired.

kubeadm token create

#On the master, you can find the ca cert hash.

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

#Using the master (API Server) IP address or name, the token and the cert has, let's join this Node to our cluster.

sudo kubeadm join 172.16.94.10:6443 \

    --token 9woi9e.gmuuxnbzd8anltdg \

    --discovery-token-ca-cert-hash sha256:f9cb1e56fecaf9989b5e882f54bb4a27d56e1e92ef9d56ef19a6634b507d76a9

#Back on master, this will say NotReady until the networking pod is created on the new node. Has to schedule the pod, then pull the container.

kubectl get nodes 

#On the master, watch for the calico pod and the kube-proxy to change to Running on the newly added nodes.

kubectl get pods --all-namespaces --watch

#Still on the master, look for this added node's status as ready.

kubectl get nodes

#GO BACK TO THE TOP AND DO THE SAME FOR c1-node2.

#Just SSH into c1-node2 and run the commands again.

ssh aen@c1-node2

#You can skip the token re-creation if you have one that's still valid.

Kubernetes Create Master Node Steps

#Only on the master, download the yaml files for the pod network.

#The calico yaml file has changed since the publication of the course and is now avaialble at the URL below.

wget https://docs.projectcalico.org/manifests/calico.yaml

#Look inside calico.yaml and find the network range CALICO_IPV4POOL_CIDR, adjust if needed.

vi calico.yaml

#Create our kubernetes cluster, specifying a pod network range matching that in calico.yaml!

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

#Configure our account on the master to have admin access to the API server from a non-privileged account.

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

#Deploy yaml file for your pod network.

#This line of code has be updated since the publication of the course. 

kubectl apply -f calico.yaml

#Look for the all the system pods and calico pod to change to Running. 

#The DNS pod won't start until the Pod network is deployed and Running.

kubectl get pods --all-namespaces

#Gives you output over time, rather than repainting the screen on each iteration.

kubectl get pods --all-namespaces --watch

#All system pods should be Running

kubectl get pods --all-namespaces

#Get a list of our current nodes, just the master.

kubectl get nodes 

#Check out the systemd unit, and examine 10-kubeadm.conf

#Remeber the kubelet starts static pod manifests, and thus the core cluster pods

sudo systemctl status kubelet.service 

#check out the directory where the kubeconfig files live

ls /etc/kubernetes

#let's check out the manifests on the master

ls /etc/kubernetes/manifests

#And look more closely at API server and etcd's manifest.

sudo more /etc/kubernetes/manifests/etcd.yaml

sudo more /etc/kubernetes/manifests/kube-apiserver.yaml

Kubernetes package install from package manager

#Setup 

#   1. 3 VMs Ubuntu 16.04.5 or 18.04.1.0, 1 master, 2 nodes.

#   2. Static IPs on individual VMs

#   3. /etc/hosts hosts file includes name to IP mappings for VMs

#   4. Swap is disabled

#   5. Take snapshots prior to installations, this way you can install 

#       and revert to snapshot if needed

#Disable swap, swapoff then edit your fstab removing any entry for swap partitions

#You can recover the space with fdisk. You may want to reboot to ensure your config is ok. 

swapoff -a

vi /etc/fstab

#Add Google's apt repository gpg key

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

#Add the Kubernetes apt repository

sudo bash -c 'cat <<EOF >/etc/apt/sources.list.d/kubernetes.list

deb https://apt.kubernetes.io/ kubernetes-xenial main

EOF'

#Update the package list and use apt-cache to inspect versions available in the repository

sudo apt-get update

apt-cache policy kubelet | head -n 20 

apt-cache policy docker.io | head -n 20 

#Install the required packages, if needed we can request a specific version

sudo apt-get install -y docker.io kubelet kubeadm kubectl

sudo apt-mark hold docker.io kubelet kubeadm kubectl

#Check the status of our kubelet and our container runtime, docker.

#The kubelet will enter a crashloop until it's joined. 

sudo systemctl status kubelet.service 

sudo systemctl status docker.service 

#Ensure both are set to start when the system starts up.

sudo systemctl enable kubelet.service

sudo systemctl enable docker.service

# Setup Docker daemon. This is a change that has been added since the recording of the course.

sudo bash -c 'cat > /etc/docker/daemon.json <<EOF

{

  "exec-opts": ["native.cgroupdriver=systemd"],

  "log-driver": "json-file",

  "log-opts": {

    "max-size": "100m"

  },

  "storage-driver": "overlay2"

}

EOF'

#Restart reload the systemd config and docker

sudo systemctl daemon-reload

sudo systemctl restart docker

Jenkins, From Zero To Hero: Become DevOps Jenkins Master - UDEMY

Freestyle Job:

 

Job Name - my-first-job

 

1.Job Settings

 

2.Configure:

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
					Add Build Step -> Execute Shell -> Command: echo Hello World

 

 

3.Build Now

 

In the lectures, he just explained bash using $1 and $2 which is parameter. It is the normal bash script, it is just to show how it is done in Linux. He kept the script in server which requires parameter.

 

Freestyle Job Parameter:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
	Select the check box "This project is parameterized" -> Add Parameter -> String Parameter Name: FIRST_NAME Default Value: Ricardo Giving another String Parameter Name: SECOND_NAME Default Value: Andre				Add Build Step -> Execute Shell -> Command: echo "Hello $FIRST_NAME, $SECOND_NAME"

Now we don’t get Build button instead we get Build with parameters, by clicking it we will be given with input

 

 

Freestyle Job Choice Parameter:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
	Select the check box "This project is parameterized" -> Add Parameter -> String Parameter Name: FIRST_NAME Default Value: Ricardo Giving another String Parameter Name: SECOND_NAME Default Value: Andre Click "Add Parameter" and in the options choose "Choice" Name: LASTNAME Choices: Smith Gonzalez Doz				Add Build Step -> Execute Shell -> Command: echo "Hello $FIRST_NAME, $SECOND_NAME, $LASTNAME"

 

With choice parameter we get Drop Down of values, in our case Smith, Gonzalez, Doz

 

Freestyle Job Choice Parameter:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
	Select the check box "This project is parameterized" -> Add Parameter -> String Parameter Name: FIRST_NAME Default Value: Ricardo Click "Add Parameter" and in the options choose "Choice" Name: LASTNAME Choices: Smith Gonzalez Doz Add "Boolean Parameter" Name: SHOW Default Value: Check this box				Add Build Step -> Execute Shell -> Command: /tmp/script.sh $FIRST_NAME $LASTNAME $SHOW

 

When Boolean Parameter is checked it will be "true", if not marked it will be "false"

 

In /tmp/script.sh, he had a IF condition checking true:

 

if [ "$SHOW" = "true" ]; then

 

 

Jenkins Security - to disable login:

 

Go to Manage Jenkins -> Configure Global Security -> Uncheck the "Enable Security"

 

Jenkins Security - Sign Up:

 

Configure Global Security	Security Realm	Authorizations
	1.Choose "Jenkins own user database" radio button 2.Choose the check box "Allow users to sign up"

 

Now you will get Create an account

 

Jenkins Security - Install a Plugin - Role Based Authorization Strategy Plugin:

Configure Global Security	Security Realm	Authorizations
		Role Based Strategy

 

After this you will find Manage and Assign Roles under Manage Jenkins

 

User and role:

Manage Jenkins -> User

Manage Jenkins -> Manage and Assign Roles

We can create new role and we can define permission for various access like Overall, credentials, Agent, Job, run, View

 

Role to execute job:

This role will allow to build.

Manage Jenkins -> Manage and assign roles -> Manage Roles

1. Create new role with name execution

Overall	Job
Read	Build and Read

 

Now you will get "Build" or "Build with Parameter" job

 

 

Restrict certain user to certain job:

Giving access based on pattern. For example: Giving access to job starting with word "Ansible"

1. In Manage Roles, Create new role dev

 

In Global Role

Overall	Job
Read

 

In Project Roles

Role to Add -> Ansible

Pattern -> ansible-.*

 

Role	Pattern	Job
Ansible	ansible-.*	Build, Read

 

2. In Assign Roles

 For the user add ITEM ROLES as Ansible

 

 

Jenkins Environment Variable - Creating our Own:

Will be useful when sending email. You can search in google and see in documentation.

 

Manage Jenkins -> Configure System -> Global Properties -> Environment Variables

 

Modify Jenkins URL

Manage Jenkins -> Configure System -> Jenkins Location -> Jenkins URL (It should match DNS)

 

Build Trigger - CRON:

Build Periodically -> Schedule

 

***** -> Means every single minute

 

It will have the message "Started by Timer"

 

Trigger job externally from outside of Jenkins - Bash:

1.Create User Jenkins

2.Add Global Role -> Give Read and Build permission

 

 

To get the URL to trigger from BASH

1.Copy the link for "Build Now"

2.Conifgure Global Security -> CSRF Protection -> Check the CRUMB

 

To get the Token to pass with CURL

crumb=$(curl -u "jenkins:1234" -s 'http://jenkins.local:8080/crumbIssuer/api/xml?xpath=concat(crumbRequestField,":",//crumb)')

 

To Trigger the jenkins:

The URL below is copied directly from Jenkins "Build Now". For "Build With Parameters" follow below example

curl -u "jenkins:1234" -H $crumb -X POST http://jenkins.local.8080/job/backup-db-to-aws/buildWithParameters?DB_HOST=db&DB_NAME=db_name

 

 

Jenkins Maven - FreeStyle Project:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced

 

Building JAR using Maven:

Manage Jenkins -> Global Tool Configuration -> Maven -> Name -> Install from Apache

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package

 

 

Testing using Maven:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test

 

His test were failing, to stop failing, he added an option _JAVA_OPTIONS= -Djdk.net.URLClassPath.disableClassPathURLCheck=true in Environment variable.

 

Deploying the JAR:

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>

 

 

Dashboard - Graph for test result:

The target folder will have the test result in form of XML. Example: target/surefire-reports/*.xml

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0

 

 

Archive the successful artifact:

 

 

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0 Archive the artifacts Files to archive: target/*.jar

 

The jar will appear in the Jenkins Page itself

 

Email - mailer plugin:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL:GITHUB SAMPLE PROJECT Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0 Archive the artifacts Files to archive: target/*.jar Add Email Notification: Recipients: vk@ok.com Select -> Send e-mail for every unstable build, Send separate e-mails to individuals who broke build

 

 

Jenkins With GIT - Using Authentication:

 

Go to Jenkins -> Credentials

 

Stores Scoped to JENKINS:

Jenkins -> Global Credentials -> Add Credential -> Kind: Username with Password

 

Scope:

Username
Password

ID

Description

 

Go to Maven Job and configure:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		Git Repository: Repository URL:GITLAB URL Credentials: Add the branch in Advanced			Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals -B -DskipTests clean package   Add Build Step -> Invoke top-level Maven Targets   Maven Version: Goals test   Add Build Step -> Execute Shell java -jar <path_of_Jar>	Publish Junit Test Result report Test Report XML: target/surefire-reports/*.xml Health report amplification factor: 1.0 Archive the artifacts Files to archive: target/*.jar Add Email Notification: Recipients: vk@ok.com Select -> Send e-mail for every unstable build, Send separate e-mails to individuals who broke build

 

 

GIT Hooks

In GITLAB Server, where GIT is running we will have the repository with the directory name same as .git. In the lecture we used maven.git, so go inside maven.git/custom_hooks and write below code. Name the file as post-receive

 

The custom_hooks and post-receive are special directory and file, which denotes hooks

 

#Get branch name from ref head

if ! [ -t 0 ]; then

read -a ref

fi

IFS='/' read -ra REF <<< "${ref[2]}"

branch="${REF[2]}"

 

 

if [ $branch == "master" ]; then

crumb=$(curl -v "jenkins:12345" -s 'https://jenkins:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')

curl -u "jenkins:12345" -H "$crumb" -X POST http://jenkins:8080/job/maven/build?delay=0sec

 

if [ $? -eq 0 ]; then

echo "**** OK"

else

echo "**** ERROR"

fi

fi

 

Jenkins DSL Plugin - SEED Job using "Use the provided DSL Script"

This is to create job using code.

 

Create new freestyle job -> job-dsl

 

Note: the extension is .j2

 

Configuration to create empty job:

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
					Add build Step: Process job DSL Choose: Use the provided DSL script   job('job_dsl_created'){}

 

After building this job, it will create a job named "job_dsl_created" with empty configuration.

 

 

DSL to create job with description:

 

Put below DSL in seed job configuration, in Build section.

 

job('job_dsl_example'){

description('This is my awesome job')

}

 

DSL to create Parameters:

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

}

 

DSL to add GIT / SCM

 

 

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

}

 

DSL to add Trigger / CRON

 

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

 

triggers{

cron('H 5 * * 7')

}

}

 

 

DSL to add Shell Script or Steps

 

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

 

triggers{

cron('H 5 * * 7')

}

 

steps{

shell("echo 'Hello World'")

}

}

 

Shell with multi-line

 

Use the triple quote

shell("""

echo 'Hello World'

echo 'Running Script'

""")

 

DSL to add mailer

job('job_dsl_example'){

description('This is my awesome Job')

 

parameters{

stringParam('Planet', defaultValue = 'world', description = 'This is the world')

booleanParam('Flag', True)

choiceParam('OPTION', ['option 1 (default)', 'option 2', 'option 3'])

}

 

scm{

git('https://github.com/jenkins/simple-maven', 'master')                

}

 

triggers{

cron('H 5 * * 7')

}

 

steps{

shell("echo 'Hello World'")

shell("echo 'Hello World2'")

}

 

publishers{

mailer('me@example.com', true, true)

}

}

 

 

DSL for Ansible:

 

parameters{

choiceParam('AGE', ['21', '22', '23'])

}

 

steps{

wrappers {

colorizeOutput(colorMap = 'xterm')

}

 

ansiblePlaybook('/etc/ansible/plays/some_playbook.ym'){

inventoryPath('/etc/ansible/plays/hosts')

tags('cool')

forks(1)

colorizeOutput(true)

additionalParameters('--vault-password-file $HOME/pass-vault/i2b-cl.txt')

extraVars{

extraVar("PEOPLE_AGE", '${AGE}', false)

}

}

 

}

 

 

DSL for Maven Job:

 

Maven -> Maps to invoke top-level maven target

In SCM -> He has added node -> node, it is to resolve some error.

 

scm{

git('https://github.com/jenkin/maven', 'master', {node -> node / 'extensions' << ''})

}

 

 

steps{

maven{

mavenInstallation('maven-jenkins')

goals('-B -DskipTests clean package')

}

maven{

mavenInstallation('maven-jenkins')

goals('test')

}

shell('''

echo ***********RUNNING THE JAR***************

java -jar /var/jenkins_home/workspace/mvn/target/my-app-1.0-SNAPSHOT.jar

''')        

}

 

publishers {

archiveArtifacts('target/*.jar')

archiveJunit('target/surefire-reports/*.xml')

mailer('vk@ok.com',true, true)

 

}

 

 

Automating DSL Process - Using Hooks:

1. Put the seed job (DSL) in GIT
2. Create directory custom_hooks and the file post-receive. This both are special directory/file for GIT LAB. Inside the post-receive we will have the schell script which invokes Jenkins.
3. Copy the above shell script and just change the curl command to invoke correct job

 

#Get branch name from ref head

if ! [ -t 0 ]; then

read -a ref

fi

IFS='/' read -ra REF <<< "${ref[2]}"

branch="${REF[2]}"

 

 

if [ $branch == "master" ]; then

crumb=$(curl -v "jenkins:12345" -s 'https://jenkins:8080/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')

curl -u "jenkins:12345" -H "$crumb" -X POST http://jenkins:8080/job/maven/build?delay=0sec

 

if [ $? -eq 0 ]; then

echo "**** OK"

else

echo "**** ERROR"

fi

fi

 

4. Now configure the job-dsl (seed Job)

TABS:	General	Source Control Management	Build Triggers	Build Environment	Build	Post Build Actions
		GIT: http://gitlab.com/jobs.git Credential -> Choose credential			Choose Look on Filesystem   DSL Scripts: jobs (NOTE: we have defined SCM, so it will have only one file named as jobs)

 

5. Jenkins -> Configure Global Security -> UNCHECK Enable Script Security for JOB DSL Scripts

 

Flow for automated job creation:

1. Create jenkins job-dsl
2. Create GIT repository for storing DSL jobs
3. Create HOOK for the reporsitory and if new DSL is added then trigger the job-DSL which we first created
4. In job-dsl, get the DSL settings from the GIT repository.

 

 

Theory CI/CD

Continuous Integration -> Integration, Functional Testing. Building and Testing your application

Continuous Delivery -> Deploy to Test Env, acceptance Testing. Optional step, deploying to test env

Continuous Deployment -> Deploy/"Promote" to Production

 

 

Pipeline Job - Jenkinsfile - Pipeline Plugin:

Two types - Declarative and Scripted.

 

 

Pipeline Job:

 

General	Build Triggers	Advanced Project Options	Pipeline
			Definition: Paste below Pipeline script

 

Declarative Pipeline

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

echo 'Building ...'

}

}

stage('Test'){

steps{

echo 'Testing ...'

}

}

stage('Deploy'){

steps{

echo 'Deploying'

}

}

}

}

 

Multi-Step Pipeline:

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

sh 'echo "My first pip"'

sh '''

echo "By th way you can do more stuff"

ls -lah

'''

}

}

 

}

 

}

 

 

Retry in Pipeline:

He has purposefully given invalid command. Will retry on failure

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

retry(3){

sh 'I am not going to work :C'

}

}

}

}

}

 

Timeout in pipeline:

 

pipeline{

agent any

 

stages{

stage('Build'){

steps{

retry(3){

sh 'echo hello'

}

 

timeout(time:3, unit: 'SECONDS'){

sh 'sleep 5'

}}}}}

 

Environment Variable in Jenkins File:

pipeline{

agent any

 

environment{

NAME='Cristiano'

LASTNAME='Messi'

}

stages{

stage('Build'){

steps{

sh 'echo $NAME $LASTNAME'

}}}}

 

Encrypting password:

 

In below pipeline TEST is the name of credentials defined in Jenkins -> Global credentials -> Kind: Secret Text ->  ID: TEST

pipeline{

agent any

 

environment{

secret = credentials('TEST')

}

stages{

stage('Build'){

steps{

sh 'echo $secret'

}}}}

 

 

Jenkins Post Actions

 

 

pipeline{

agent any

 

stages{

stage('TEST'){

steps{

sh 'echo Fail; exit 1'

}}}

post{

always{

echo 'I will be executed'

}

success{

echo 'I will only executed if this is success'

}

failure{

echo 'I will only get executed if this failes'

}

unstable{

echo 'I will only get executed if this is unstable'

}

}

 

}

 

Docker inside Docker:

It is normal to have docker installation along with Jenkins. Since in this course the Jenkins itself is in Docker, we need docker inside the docker.

 

He uses Dockerfile - It will work only with internet or artifactory: Command commands are

 

RUN

CURL

apt-get

USER

 

Docker Compose:

It is a yml file, build section will have the directory name where we place docker file. We also map docker.cok

 

services:

jenkins:

container_name: jenkins

image: jenkins/docker

build:

context:pipeline

ports:

-"8080:8080"

volumes:

• "$PWD/jenkins_home:/var/jenkins_home"
• /var/run/docker.sock:/var/run/docker.sock

networks:

 -net

 

To build:

docker-compose build

 

To recreate the container

docker-compose up -d

 

Defining steps for pipeline.

Now we have Jenkinsfile and Dockerfile in same directory at same level. Dockerfile has installation steps and Jenkinsfile has Declarative Pipeline script. In same directory he kept "Hellow World" Java-Maven proeject

 

 

Dockerfile

Jenkinsfile

java-app

jenkins/build

 

The objective is to build JAR file in Docker using only the above files. He is going to use Maven alpine image.

 

1. Pull image docker pull maven:3-alpine. Now we are going to build using this image instead of Maven Plugin.
2. Check image

docker images | grep maven

3. Now map the Java volume

docker run -ti -v $PWD/java-app:/app maven:3-alpine sh

 

Now we can see the java-app contents in /app directory inside docker.

4. mvn package
5. Map .m2 directory so that the downloaded files are not lost

docker run --rm -ti -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ maven:3-alpine sh

6. Update the command so that our working directory will be /app where pom.xml is present.

docker run --rm -ti -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven:3-alpine sh

7. Package the code without getting inside the docker image

docker run --rm  -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven:3-alpine mvn -B -DskipTests clean package

8. Now that we know the docker script is working, add the above line in a file mvn.sh in the jenkins/build directory. The $@ will take parameter, so that you can now run any mvn commands.

docker run --rm  -v  $PWD/java-app:/app -v /root/.m2/:/root/.m2/ -w /app maven:3-alpine "$@"

9. Go to jenkins/build and create Dockerfile-Java

FROM  openjdk:8-jre-alpine

RUN mkdir /app

 

COPY *.jar /app/app.jar

 

CMD java -jar /app/app.jar

 

#Creating Alpine-Java image and copying jar file into it

docker build -f Dockerfile-Java -t test .

 

#Get inside the newly build Java container

docker run --rm -ti test sh

 

#Running docker in detached mode to see the logs

docker run -d test

 

docker logs -f <ID we get from above command>

2. To take to next level we are using docker compose. In the same directory create "docker-compose-build.yml'

 

version: '3'

services:

app:

image: "app:$BUILD_TAG"

build:

context: .

dockerfile: Dockerfile-Java

 

3. Now building using docker compose

docker-compose -f docker-compose-build.yml build

 

cd jenkins/build && docker-compose -f docker-compose-build.yml build --no-cache

 

4. Now putting all things together. Creating build.sh in jenkins/build

 

#Copy the new jar

cp -f java-app/target/*.jar jenkins/build/

 

#Run docker compose

cd jenkins/build && docker-compose -f docker-compose-build.yml build --no-cache

 

5. Adding the script to Jenkinsfile. We just call all the scripts we have written.

pipeline{

agent any

 

stages{

stage('Build'){

 

steps{

sh '''

./jenkins/build/mvn.sh mvn -B -DskipTests clean package

./jenkins/build/buil.sh

'''

}

}

 

}

}