SysVinit or Upstart - IT PROTV - Percipio

Many system uses older SysVinit.

How are services controlled under Linux?

 It is driven by script files. It is not a binary like systemd. It does different way than systemd. It is since 1970, so still many use it. Amazon Linux is based on CentOS 6 it uses Upstart which is based on Sysvinit. Latest Amazon Linux uses systemd.

How do we know whether we are using SysVinit?

Kernel will fire off initialization system. It should be either sysvinit or systemd. Process ID 1 will be either systemd or sysvinit.

Process number 1 says /sbin/init then it will be sysvinit, sometimes we can't trust. We need to see whether it has symlink to systemd.

Sysvinit starts after the Kernel, what happens next?

BIOS fires up on the system and it does hardware detection, then it reaches out and finds hard drive partition and finds /boot, finds kernel and launches it and Kernel starts /sbin/init. Then it runs the script in serial not in parallel like systemd.

cd /etc/rc.d

rc.sysinit (This script is started first)

rc1.d, rc2.d refers to previous iteration of sysvinit. Now everything is inside rc.d.

Does one script rc.sysinit calls other scripts?

Yes, it call other script depending on the run level. In Linux we have 7 run level, each indicates different status of the machine.

rc0.d -> Means shudown

We have runlevel for shutdown because we might need to runs some script before shutdown like stop all process writing to disk

rc6.d -> Reboot

It is not necessary to have tasks in each runlevel.

1 to 5 runlevels are based on the distro. Check documentation. Lot of the times it will be same but better check. You can also check in initialization table

cat /etc/inittab

 It tells what each run-level does. In Centos 1 is for single user mode, 2 multiuser w/0 NFS, 3 Full multiuser mode, 4 is unused, 5 is X11.

The alternative of runlevel in systemd is target files. By using isolate command.

Based on the run level we choose it will start all the scripts inside the specific run-level directory.

rc.local file is just present for backward compatibility.

Do we have to modify the script for each run level?

SSH is setup to run in runlevel 3 and 5 but they skipped in runlevel 2 because it has no networking. Suppose we have web server and we need it in run level 3 and 5 we can write manual script. But it is handled by using below commands:

sudo yum install httpd

This installation will create file under the run level. Mostly it is symlink.

chkconfig allows to turn on/off the service to run and we  can also specify runlevel

sudo chkconfig httpd on

sudo chkconfig --level 35 httpd off

But in-order to found out which service is in which run level we DO NOT to go and see in the rc directories. Because even after turning off there will be symlink. The better way is to use below command:

chkconfig --list

How does SYSVinit know which run level to boot to?

The run level will be present in /etc/inittab file.

If we want to change the run level temporarily, we can use below command. Put the run level you want after init.

sudo init 3

Scheduling run level?

We can use cron to schedule or we can also use telinit. Using telinit we can delay when the runlevel to change.

sudo telinit 3 -t 30

Be careful, this is old command and it not supported by many distro.

Systemd - ITPRO TV - PERCIPIO

Managing services with systemd:

Powerful technology from 2008. How it works, what it does we can see here.

How are services controlled under Linux?

Man system uses init system. When system boots up it calls Kernel which is wrote by Linus Torvalds. It itself don't do much. After kernel start it starts init. Init will start various other services. It is used in Unix we call sysvinit or System Five.

Drawbacks of SysVinit

It runs in serial order. Like script 1, script 2, and script 3. If script 3 takes long time then it delays starting script 4.

In current situation, we constantly plug-in new devices like thumb drive. In that case we need to re-run the scripts to look for missing hardware. This is problem with sysvinit.

What SystemD does?

It is a system daemon which means any program that runs in the background. It always stays online unlike SysVinit, so that it can detect hardware changes. It can do things in parallel. It is faster and more stable. It does lot of stuff in binary, while SysVinit allows you to change configuration in a text file which many Linux users prefer.

SystemD reaches out to hard drive and find boots files, loads up the Linux kernel. Linux Kernel launches init, in our case it is systemd. SystemD will start everything else we interact with our system

How do we know whether we are using SystemD?

Major distro use SystemD by default. Sometimes people use upstart which is modified version of SysVinit. Distro like Slackware, Devaun (variant of Debian) still uses sysvinit.

How to verify if I have systemd or sysvinit?

Init is our very first process. You can try firing ps aux if you scroll up see the first process it shows what process it started first, it should be systemd. Sometime it says /sbin/init which is SysVinit but you need to dig deeper to check. ls -l /sbin/init this may have symbolic link to systemD. In these cases the ps output will still show init but under the hood it is systemD. /sbin/init should be always present in the Linux system as per the standard. So all of the recent distro would have symlinked to systemD.

SystemD starts right after the Kernel, what happens next?

It uses unit file to start other services. It is stored in /lib/systemd/system. There will be ton of service file with .service and .target extension. .target is for GUI. .socket files are for network which allows two application talk to each other via a network. .mount file contains hard drive partition. We will still have /etc/fstab.

Example exploring sshd.service:

It starts with block [Unit]

After -> will have dependencies. For sshd we need network services. So it will first start network.services

Wants -> Says what else to start along with sshd.service.

[Service] -> This block will specify actual service to be started.

EnvironmentFile -> Where we have environment variables

ExecStart -> Actual command that will start the service

ExecReload -> Usually kill command

KillMode -> Default is process

Restart -> Says when to restart (on-failure) is default

RestartSec -> Wait time after failure

[Install] -> Specifies target

WantedBy=multi-user.target

When command line starts, it starts sshd. Most of the times it is created.

How to modify the Unit files?

It is not modified directly. Don't anything manually inside /lib. Go to /etc/systemd/system.

Sometime the folder won't exists we might need to create it. Again in this folder you will see service, wants and target file. This will override the files in /lib/systemd

SystemD binary is located @ /lib/systemd

Kernel -> Do all hardware interactions

SystemD -> Do all software interactions

Do we have to create new files for Unit services?

Most of the software installation will come with Unit Files. So while installing we get and while uninstalling it get removed.

If we want the service to start when system reboots we can enable using below command:

sudo systemctl enable httpd

This will just create symlink to /etc folder because it takes more precedence than /lib. You can also disable using below command:

sudo systemctl disable httpd

The command only disable starting of service after reboot. You can still start manually.

How systemD keps track of which service to run?

it is done using target files. .target files keep track of what needs to run next. For example: If we need command line then multi-user.target will get triggered. Then it will see all the entries inside the file which has other dependent target files.

If we edit, then it won't recognize unless we reload.

Another example: To Turn Off GUI

sudo systemctl isoloate multi-user.target

 To Turn on GUI

sudo systemctl isoloate graphical.target

Writing Task 2 Ideas - Business and Money

In some countries, a few people *earn extremely high salaries*. Some people think that this is *good for a country*, while others believe that the government should *control salaries* and limit the amount people can earn.

Discuss both views and give your opinion. (Reported 2017, Academic Test)

Type: Discussion/Opinion

BP1: control salaries
wealth is evenly distributed
bring more equality
disheartening for some to receive a low salary while others receive a high salary
capping high salaries will prevent the rich from gaining unfair control
limiting high salaries will mean more funding for public services

BP2: good for country
highly skilled and talented should be rewarded
incentive for others to work harder.
important to entice people to do dangerous or unpleasant work
sign of country's wealth
deter skilled or talented people leaving a country

Percipio - ITPro TV - TCP/IP & Networking

Linux - Network operating system.

How are IP address assigned?

IP's are similar to phone numbers. Computer has IP assigned to them and it has to come from somewhere.

ifconfig

Gives IP address but it is older command.

ip addr

Above command shows IP address. DHCP server will hand out the address. When the computer is started, it asks for IP address and DHCP server responds to it. DHCP do DORA (discover, offer, request, acknowledgement). So when our laptop is started it do discovery and ask could somebody please give me an IP? This message is sent out and DHCP server will hear it and respond back with offer. Our laptop will request to keep the IP that is offered by DHCP server. Finally DHCP server will assign the IP to our server. These are usually automatic and we can even do it manually. IP address come from 32 bit address pool. In our IP /8, /16 or /24 will tell us which class we belong to. We are not following this anymore.

3 types of network

Class A - 255.0.0.0 - /8 - can hold 16 m hosts (255 is network and 0 is host, so if my ip is class A first digit is network and remaining 3 is host, in class b first and second digit is network and 3 and 4 digit is host. It depends on which class I am in)

Class B - 255.255.0.0 -/16 - can hold 65,534 hosts

Class C - 255.255.255.0 - /24 - can hold 254 hosts (common in home network)

In IP address part of it represent network and part of it represent the host. It is like the street address. Lets take an address 71, Ashraya Layout. So there can many house in Ashraya Layout it can be compared with network. However, 71 is specific to our home and it is equivalent to host.

How do we choose which addresses to use?

Private IP can be pick amongst your heart's content. Public IP has to be globally unique. ARIN is the organization register number and they make sure there is no duplicate. On LAN we will have fake ID and NAT Device will translate to public IP. Below ranges are set aside and we can pick any IP

RFC 1918

198.168.x.x/24 - We can use any IP inside it

172.16.x.x -> 172.31.x.x/16

10.x.x.x/8

What is IPV6?

IPV4 -> 32 bit address. 4.3 billion address (earth population was also 4.3 bil and not many had computers)

Now it is around 7billion and many have computers.

IPV6 -> 128 bit address (1 decillion address). With IPV6 we can give 4.3 billion address to each individual

Where does TCP come in?

IP is responsible for addressing, where is your traffic coming from and where is the traffic going to. TCP allows us to establish session.

nslookup towel.blinkenlights.nl (will show the IP)

telnet towel.bilinkenlights.nl

ss -an (we can use to see above session, ss stands for socket statistics, when we open session then it is call socket)

a-> all session

n-> don’t do named lookup

Telnet will have port 23

What port number we should be familiar with?

Some protocol uses both TCP and UDP. FTP uses two port one for commands and another for transferring data. DNS already have re-transmission built-in so we don’t need TCP for that.

20	FTP	TCP
21	FTP	TCP
22	SSH	TCP
25	SMTP	TCP
53	DNS	TCP/UDP
80	HTTP	TCP
110	POP3	TCP
119	NNTP	TCP
123	NTP	UDP
139	NetBios	TCP/UDP
143	IMAP	TCP
161	SNMP	UDP
443	HTTPS	TCP
465	SMTP-SSL	TCP
993	IMAPS	TCP
995	POP3S	TCP

How networking configured in Linux Server: - Video 2

How to view our current network configuration?

15 years ago it is pain, now it is mostly automated.

Different ways are there to see current configuration.

ifconfig  (It is older tools and it wont come by default in newer version)

We see all network adapters and their addresses. If the network adaptor is ON and we have DHCP serer they we see the IP.

route (this command show routint table but it is old)

IP commands are newer and we should start learning this compared to ifconfig and route

ip addr (it shows address information)

ip route (to see routing information)

Is there anything that we should do before we start changing things?

For simple problem like, we set DHCP and not getting DHCP address we just need to restart. DHCP service is not running. Or our dhclient is not running. We can do by running below command.

sudo dhclient

If it is already running we can do retry

sudo dhclient -r

To restart entire network stack

For systemd based system

sudo systemctl restart network

For Sysvinit based system

service network restart

Where are the network interface configurations stored?

In Redhat

Network configuration are stored in /etc/sysconfig/network-scripts We will have network scripts. Each interface have own script. They will have network settings.

Meaning of letter in the name:

en -ethernet

wl -wireless

ww - cellular (wireless wan)

p - pci

o - onboard

s - hot plug support

We can switch back to static IP by using following config

BOOTPROTO=none

IPADDR0=<IP Address you wish to have>

PREFIX0=16 (subnet)

GATEWAY0=10.0.0.1

To take the settings to take effect, restart network stack (network restart) or full OS.

In Ubuntu

/etc/network -> Used to store here

/etc/netplan -> can be here too

Check the interfaces file inside both folders. It should show the interface that we get using "ip addr" command in that file. If it only has 'iface lo inet loopback' then that is not the right one. Under netplan, you will only find yaml file. Look for 99_config.yaml. You should see the interface that we get using "ip addr" here. When using netplan we don't need to restart. It applies the settings directly without restart.

Command to use when netplan config is changed:

sudo netplan apply

What about global settings like DNS servers?

In Centos:

/etc/sysconfig, there is a file called network.

/etc/resolv.conf has DNS. By default it will pull from DHCP server. But we can also change it.

/etc/hosts will override resolv.conf. It has IP address at left and domain name at right.

/etc/hostname is not global settings. It is where our computer name come from. If we edit this file and reboot the settings will be lost. To make permenant changes use hostnamectl command.

sudo hostnamectl set-hostname Centos

It doesn't obey capitilization

Are there any easier ways to configure network settings?

It is easier using GUI when we have desktop version. If we don't have GUI many distro supports network manager. To find out if we have network manager, launch nmcli. It gives command lines interface and takes care everything in backend. It know where the config are present.

nmcli device status

After running above command it should show us some output like DEVICE, TYPE, STATE, CONNECTION with green font. If it doesn’t show any output then we are doing manual configuration. The state will show Connected for which nmcli is managing for others it shows unmanaged.

In Ubuntu

nmcli device status

It shows everything is unmanaged because of netplan migration. First understand whether the GUI is netplan or network manager. Based on that use the same method. If it is netplan use netplan. If it is network manager then use it. You can find whether it is managed using network manager or not using nmcli device status command.

Adaptername - etho

nmcli device show <adaptername>

To change any settings:

nmcli connection edit <adaptername>

The above command will directly take to adapter. You can give below command inside the adapter

set connection.autoconnect yes

set ipv4.method manual

After running above command use save command to save. But it will lost after reboot. If you want even after reboot, use save persistent to save it to hard drive.

Do, sudo nmcli connection reload

The above reload the connection. We notice there are lot of commands but it is best practice as we don't need to chase for files.

How to fail in Microservices? - THAAT Conference - Pluralsight

1. Using Mircoservice when not required for example

If deployment is only once per month

If the team is not ready

If the app just works fine with monolithic

2. Testing

Changes can cause cascading failure

Others depend on our service

Implement red-green deployment - It will only affect small subset of people. Do testing directly in production. Do rollback

3. No Devops

No Logging and Tracing

Distributed tracing -> Used to find which microservice in my code is taking long time to respond. It will help to plot the graph between service a to the end service. And also help us see which service took long time. Kubernetes has side-car. AWS has X-Ray.

If we don't have automated deployments.

If we don't have configuration management. For example: environment specific details Dev, Test, QA, PROD. It should be in a configuration service. Don't check-in hostnames

4. POLYGLOT

Chose appropriate technology. Choosing many random technology makes support difficult

Difficult to do auto-deployment if we have too many different stacks like .net, java, go, python, ruby. use standard tech stack. Better to use single tech stack.

5. Don't expect failures

Lot of failures in microservices than monolithic

Build microservices for resiliency by doing graceful degradation

Do health check before sending request to some other service

Retry again in case of failure. It can be a transient failure. Retry only at command level. Use circuit breakers at service level.

Process asynchronously

Jenkins 2 course pluralsight

Creating NewJob in Jenkins:

https://jenkins.global.com/newJob

Freestyle - Configure Tab

SCM -> Give git link, username/password

Build -> Invoke top-leve maven target -> Goals compile (1st vid)

Build -> Invoke top-leve maven target -> Goals archive (2nd vid)

To archive:

Post build actions -> Archive the artifacts -> <give the path to archive. Note: jar will be in target path>

To Clean and Package:

Invoke top-leve maven target -> Goals clean package (3r vid)

Pipeline Job

Master will have executors. By default it can do 2 tasks and we can allocate more if we need.

Node - main block

parallel chrome:{runTests("Chrome")

}, firefox:{

} -> This is example of parallel. chrome just name. runTests is function. Only parallel is the keyword

Stage name: concurrency:1 -> Shows as stages. For example: We can give our own title for each block of code in pipline like checkout, compile, test and archive. It can also use concurrency option which is used when multiple build is triggered then only latest build will execute.

input - To introduce manual step and it don't require to be inside a node. It uses flyweight executor

Executors -> Flyweight (it don’t allocate node and there is no limit) and Heavyweight (it allocate node). Heavyweight does the actual execution

For e-mail - Try using mailhog docker

Jenkinsfile:

For using jenkins file use option "Pipeline script from SCM". Here we will have option for GIT

When using jenkins file, GIT groovy is not required. Instead give "checkout scm"

Pipeline advantages:

You can break jobs into several stages. This will help us to narrow down where it failed.

Ability to run parallel

If agent goes down pipeline will pause and start whereas freestyle will fail.

Parallel Job:

While running parallel job, if the job runs in same server in workspace it will create folder with name @2, @3 etc based on how many parallel we have

node statement will come inside the def statement.

Parallel code should be written first

Creating Node:

Manage Jenkins - New Node - Give name / check permenant agent checkbox

Give Remote Root Directory -> This should be unique for each agent

Labels -> The name can be used for pooling together agents

Launch method -> Java webstart (JNLP file for Windows and SSH for Linux)

Problems -> Password less SSH connectivity. Paste Private key in Jenkins Credentials Manager, add the Public key to authorized key of the server

Java not in path ->  /usr/local/bin/java (Created symbolic link)

Getting Started with Jenkins 2
By Wes Higbee

Kubernetes Nigel Poulton Get Started Pluralsight

k8s -> Orchestrating containerized apps

K8s cluster -> Master and Node. Similar to Football team. Team are the nodes and Manager is master

K8s Deployment -> We will have code and we will containerize it. This will be defined in an object called Deployment. This is done using YAML file.

Master -> Run in single server on top of Linux. There can be multiple master but complex to configure. It has kube-apiserver which is front-end to control plane, exposes rest-api. Cluster Store is the persistent storage and uses etcd. This is where state and config lives. It is the only stateful part of control plan and rest are stateless. ETCD is open source distributed key store value and kubernetes always look for it. Kube-controller-manager is the controller which controls all controllers. Kube-scheduler assigns work to node.

Nodes -> Node has kubelet (main agent, watches master api-server, reports to master in-case of failure, endpoint port is 10255, enpoints are /spec, /healthz, /pods), container runtime (container management usually docker or core os rocket)and kube-proxy (Use for networking, Makes sure each pods get IP, load balancing  )

Declarative Model and Desired State -> Which means we give desired state similar to ansible. We just give manifest file to Kubernetes saying what our result should look like. Think if we are building kitchen we just say to contractor and he will do the rest. Suppose we ask K8s to run 3 x nginx node but for some reason one goes down, k8s will automatically run one more in existing node.

Pods -> K8s always run container inside pod. Pod will have network stack, volume and container use it. You only need to have two container in pod when the two container needs to share same resource. For example: Web Server and Log Scrapper, because log scrapper need to read the log from web server. Replication Controller sits above pod and Deployment sits above replication controller. Replication controller are used to deploy multiple replica of pod.

Services -> We can't rely on pod IP as the pod gets deleted and next time it spawn with new IP. Service will keep note of new pod's IP. It provides abstraction of multiple pods and also provides load balancing. Every pod will also get labels, it will only loadbalance based on the labels. If service has label PROD, WEB, V.14 and it will loadbalance only those has labels PROD, WEB, V.14. In short service provides stable IP and DNS names


Deployments -> REST objects, we write in YAML file and it is used by apiserver. Rolling updates and rollbacks can be done using deployment. In deployment we say what we want but we don't say what to do. Example: We say we need 4 replicas and it does all the task needed for maintain 4 nodes even if 1 node is down.

Installation:
AWS
we need kubeclt, kops and AWS CLI to install in AWS. Full access to EC2, Route53, S3, IAM and VPC
kops and kubectl were installed from curl. We need on s3 bucket and add the bucket name to the variable KOPS_STATE_STORE=s3://<bucket_name>

Manual Install -> Done using kubeadm
infra1@Hostname[DEV][~] $ sudo yum install docker.io kubeadm kubectl kubelet kubernetes-cni

kubeadm init -> To start cluster, this will give 3 commands, just execute it

Pod:
Contains one or more contains. Pod will get one IP even it has many containers. Every container inside a single pod will share same network namespace. All the pods will communicate using a pod network. Within the pod the containers communicate using localhost interface. We cannot use same port for two containers inside the pod. It is declared in manifest file (Json or YAML) and is sent to API server.

Sample POD YML:
apiVersion: v1
kind: Pod
metadata:
  name: hello-pod
  labels:
    zone: prod
    version: v1
spec:
  containers:
  - name: hello-ctr
    image: nigelpoulton/pluralsight-docker-ci:latest
    ports:
    - containerPort: 8080

We don't work directly with POD. Higher level of pod is replication controller. We always deploy replication controller which in-turn deploys the pod. Example YAML file below:

apiVersion: v1
kind: ReplicationController
metadata:
  name: hello-rc
spec:
  replicas: 5

Replication controller implements desired state and it is high level construct. It will make sure it always have 5 replicas. In YML spec will provide detail of replication and template will contain the pod spec.


Services:
Until now the only way we know to see the service is running is using "kubectl get pods" command. To access our app from outside of the cluster (like web page) or inside the cluster we use services. Services is REST object similar to pod and replication controller we use YML file. Service is our endpoint because pod will always get die and their IP changes. So services provide reliable IP, DNS, and PORT. Service will always make note of new pods that are replaced by dead ones. It uses labels to identify which pod to load balance.

Sample YML:
apiVersion: v1
kind: Service
metadata:
  name: hello-svc
  labels:
    app: hello-world
spec:
  type: NodePort
  ports:
  - port: 8080
    nodePort: 30001
    protocol: TCP
  selector:
    app: hello-world

There are 3 service types we can use in spec: ClusterIP, NodePor, LoadBalancer
nodeport is optional.

Deployment:
Rolling update and rollback. When done using deployment when we make rolling update (Ie v1 to v2) replica set/ replication controller won't get deleted. However there won't be any pod inside the old version.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-deploy
spec:
  replicas: 10
  template:
    metadata:
      labels:
       app: hello-world
     spec:
       containers:
       - name: hello-pod
         image: nigelpoul/pluralsight-docker-ci:latest
         ports:
         - containerPort: 8080

In above YML, if we see apiVersion it has beta1. Just v1 will not work

Updating Deployment
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: hello-deploy
spec:
  replicas: 10
  minReadySeconds: 10
 strategy:
  type: RollingUpdate
  rollingUpdate:
  maxUnavailable: 1
  maxSurge: 1
  template:
    metadata:
      labels:
       app: hello-world
     spec:
       containers:
       - name: hello-pod
         image: nigelpoul/pluralsight-docker-ci:edge
         ports:
         - containerPort: 8080

Green highlighted are added.
minReadySeconds - Says wait for 10 seconds after each new pod comes up before you mark it as ready and move on to the next one.


Commands:
kubectl get nodes
kubectl get pods --all-namespaces -> Will show dns, etcd,kube-proxy details (if DNS is not ready run below)
kubectl apply --filename https://git.io/weave-kube-1.6
kubeadm join --token <token> <IP> -> To join kubernetes node to master
kubectl create -f pod.yml -> (To create a pod, replication controller and service)
kubectl get pods (Will show the list of pods with status)
kubectl get pods/hello-pod -> To get status of specific pod
kubectl describe pods ( will give the pod information, will also give status like Pending, also gives Lablels)
kubectl describe svc hello-svc (will give service information)
kubectl delete pods hello-pod
kubectl apply -f rc.yml (will apply the changes, like updating replica from 10 to 20)
kubectl expose rc hello-rc --name=hello-svc --target-port=8080 --type=NodePort (you will get message service "hello-svc" exposed) (Expose argument will create service object, rc is replication controller, we are creating a service hello-svc )
kubectl describe svc hello-svc (NodePort is the cluster wide port, we will use it in our webpage URL. By default service port will be between 30000 and 32767)
kubectl get svc (To get the services)
kubectl delete svc hello-svc (To delete services)
kubectl get ep (to get endpoint)
kubectl rolling-update -f update-rc.yml (Rolling update the replication controller, but not recommended use deployments)
kubectl create -f deploy.yml (Creating deployment)
kubectl describe deploy hello-deployment (gives description about deployment)
kubectl get rs (to get replica set. It will show how many replica we asked for DESIRED and how many is there CURRENT and READY)
kubectl describe rs (Describing replica set)
kubectl apply -f deploy.yml --record (For applying deployment update)
kubectl rollout status deployment hello-deploy (Run this after deployment to get the status)
kubectl get deploy hello-deploy ( Will show desired, current, up to date and available status)
kubectl rollout history deployment hello-deploy (To see deployment history)
kubectl get rs (To get replica set)
kubectl describe deploy hello-deploy (Run this to see whether the docker image we used is updated in the Image)
kubectl rollout undo deployment hello-deploy --to-revision=1 (IT will roll back to older version)