Thursday, 26 March 2020

Systemd - ITPRO TV - PERCIPIO


Managing services with systemd:
Powerful technology from 2008. How it works, what it does we can see here.

How are services controlled under Linux?
Man system uses init system. When system boots up it calls Kernel which is wrote by Linus Torvalds. It itself don't do much. After kernel start it starts init. Init will start various other services. It is used in Unix we call sysvinit or System Five.

Drawbacks of SysVinit
It runs in serial order. Like script 1, script 2, and script 3. If script 3 takes long time then it delays starting script 4.
In current situation, we constantly plug-in new devices like thumb drive. In that case we need to re-run the scripts to look for missing hardware. This is problem with sysvinit.

What SystemD does?
It is a system daemon which means any program that runs in the background. It always stays online unlike SysVinit, so that it can detect hardware changes. It can do things in parallel. It is faster and more stable. It does lot of stuff in binary, while SysVinit allows you to change configuration in a text file which many Linux users prefer.

SystemD reaches out to hard drive and find boots files, loads up the Linux kernel. Linux Kernel launches init, in our case it is systemd. SystemD will start everything else we interact with our system

How do we know whether we are using SystemD?
Major distro use SystemD by default. Sometimes people use upstart which is modified version of SysVinit. Distro like Slackware, Devaun (variant of Debian) still uses sysvinit.

How to verify if I have systemd or sysvinit?
Init is our very first process. You can try firing ps aux if you scroll up see the first process it shows what process it started first, it should be systemd. Sometime it says /sbin/init which is SysVinit but you need to dig deeper to check. ls -l /sbin/init this may have symbolic link to systemD. In these cases the ps output will still show init but under the hood it is systemD. /sbin/init should be always present in the Linux system as per the standard. So all of the recent distro would have symlinked to systemD.

SystemD starts right after the Kernel, what happens next?
It uses unit file to start other services. It is stored in /lib/systemd/system. There will be ton of service file with .service and .target extension. .target is for GUI. .socket files are for network which allows two application talk to each other via a network. .mount file contains hard drive partition. We will still have /etc/fstab.

Example exploring sshd.service:
It starts with block [Unit]

After -> will have dependencies. For sshd we need network services. So it will first start network.services
Wants -> Says what else to start along with sshd.service.

[Service] -> This block will specify actual service to be started.
EnvironmentFile -> Where we have environment variables
ExecStart -> Actual command that will start the service
ExecReload -> Usually kill command
KillMode -> Default is process
Restart -> Says when to restart (on-failure) is default
RestartSec -> Wait time after failure

[Install] -> Specifies target
WantedBy=multi-user.target

When command line starts, it starts sshd. Most of the times it is created.


How to modify the Unit files?
It is not modified directly. Don't anything manually inside /lib. Go to /etc/systemd/system.
Sometime the folder won't exists we might need to create it. Again in this folder you will see service, wants and target file. This will override the files in /lib/systemd

SystemD binary is located @ /lib/systemd

Kernel -> Do all hardware interactions
SystemD -> Do all software interactions



Do we have to create new files for Unit services?
Most of the software installation will come with Unit Files. So while installing we get and while uninstalling it get removed.

If we want the service to start when system reboots we can enable using below command:
sudo systemctl enable httpd

This will just create symlink to /etc folder because it takes more precedence than /lib. You can also disable using below command:

sudo systemctl disable httpd

The command only disable starting of service after reboot. You can still start manually.

How systemD keps track of which service to run?
it is done using target files. .target files keep track of what needs to run next. For example: If we need command line then multi-user.target will get triggered. Then it will see all the entries inside the file which has other dependent target files.

If we edit, then it won't recognize unless we reload.

Another example: To Turn Off GUI

sudo systemctl isoloate multi-user.target


 To Turn on GUI
sudo systemctl isoloate graphical.target

Sunday, 15 March 2020

Writing Task 2 Ideas - Business and Money

In some countries, a few people *earn extremely high salaries*. Some people think that this is *good for a country*, while others believe that the government should *control salaries* and limit the amount people can earn.

Discuss both views and give your opinion. (Reported 2017, Academic Test)

Type: Discussion/Opinion

BP1: control salaries
wealth is evenly distributed
bring more equality
disheartening for some to receive a low salary while others receive a high salary
capping high salaries will prevent the rich from gaining unfair control
limiting high salaries will mean more funding for public services

BP2: good for country
highly skilled and talented should be rewarded
incentive for others to work harder.
important to entice people to do dangerous or unpleasant work
sign of country's wealth
deter skilled or talented people leaving a country


Tuesday, 10 March 2020

Percipio - ITPro TV - TCP/IP & Networking


Linux - Network operating system.

How are IP address assigned?
IP's are similar to phone numbers. Computer has IP assigned to them and it has to come from somewhere.

ifconfig

Gives IP address but it is older command.

ip addr

Above command shows IP address. DHCP server will hand out the address. When the computer is started, it asks for IP address and DHCP server responds to it. DHCP do DORA (discover, offer, request, acknowledgement). So when our laptop is started it do discovery and ask could somebody please give me an IP? This message is sent out and DHCP server will hear it and respond back with offer. Our laptop will request to keep the IP that is offered by DHCP server. Finally DHCP server will assign the IP to our server. These are usually automatic and we can even do it manually. IP address come from 32 bit address pool. In our IP /8, /16 or /24 will tell us which class we belong to. We are not following this anymore.

3 types of network

Class A - 255.0.0.0 - /8 - can hold 16 m hosts (255 is network and 0 is host, so if my ip is class A first digit is network and remaining 3 is host, in class b first and second digit is network and 3 and 4 digit is host. It depends on which class I am in)
Class B - 255.255.0.0 -/16 - can hold 65,534 hosts
Class C - 255.255.255.0 - /24 - can hold 254 hosts (common in home network)


In IP address part of it represent network and part of it represent the host. It is like the street address. Lets take an address 71, Ashraya Layout. So there can many house in Ashraya Layout it can be compared with network. However, 71 is specific to our home and it is equivalent to host.

How do we choose which addresses to use?
Private IP can be pick amongst your heart's content. Public IP has to be globally unique. ARIN is the organization register number and they make sure there is no duplicate. On LAN we will have fake ID and NAT Device will translate to public IP. Below ranges are set aside and we can pick any IP

RFC 1918
198.168.x.x/24 - We can use any IP inside it
172.16.x.x -> 172.31.x.x/16
10.x.x.x/8

What is IPV6?
IPV4 -> 32 bit address. 4.3 billion address (earth population was also 4.3 bil and not many had computers)
Now it is around 7billion and many have computers.
IPV6 -> 128 bit address (1 decillion address). With IPV6 we can give 4.3 billion address to each individual

Where does TCP come in?
IP is responsible for addressing, where is your traffic coming from and where is the traffic going to. TCP allows us to establish session.

nslookup towel.blinkenlights.nl (will show the IP)

telnet towel.bilinkenlights.nl

ss -an (we can use to see above session, ss stands for socket statistics, when we open session then it is call socket)

a-> all session
n-> don’t do named lookup

Telnet will have port 23

What port number we should be familiar with?

Some protocol uses both TCP and UDP. FTP uses two port one for commands and another for transferring data. DNS already have re-transmission built-in so we don’t need TCP for that.



20
FTP
TCP
21
FTP
TCP
22
SSH
TCP
25
SMTP
TCP
53
DNS
TCP/UDP
80
HTTP
TCP
110
POP3
TCP
119
NNTP
TCP
123
NTP
UDP
139
NetBios
TCP/UDP
143
IMAP
TCP
161
SNMP
UDP
443
HTTPS
TCP
465
SMTP-SSL
TCP
993
IMAPS
TCP
995
POP3S
TCP




How networking configured in Linux Server: - Video 2

How to view our current network configuration?
15 years ago it is pain, now it is mostly automated.

Different ways are there to see current configuration.

ifconfig  (It is older tools and it wont come by default in newer version)
We see all network adapters and their addresses. If the network adaptor is ON and we have DHCP serer they we see the IP.


route (this command show routint table but it is old)

IP commands are newer and we should start learning this compared to ifconfig and route
ip addr (it shows address information)
ip route (to see routing information)





Is there anything that we should do before we start changing things?

For simple problem like, we set DHCP and not getting DHCP address we just need to restart. DHCP service is not running. Or our dhclient is not running. We can do by running below command.

sudo dhclient

If it is already running we can do retry

sudo dhclient -r

To restart entire network stack
For systemd based system
sudo systemctl restart network


For Sysvinit based system
service network restart

Where are the network interface configurations stored?
In Redhat
Network configuration are stored in /etc/sysconfig/network-scripts We will have network scripts. Each interface have own script. They will have network settings.

Meaning of letter in the name:
en -ethernet
wl -wireless
ww - cellular (wireless wan)
p - pci
o - onboard
s - hot plug support

We can switch back to static IP by using following config
BOOTPROTO=none
IPADDR0=<IP Address you wish to have>
PREFIX0=16 (subnet)
GATEWAY0=10.0.0.1

To take the settings to take effect, restart network stack (network restart) or full OS.

In Ubuntu
/etc/network -> Used to store here
/etc/netplan -> can be here too

Check the interfaces file inside both folders. It should show the interface that we get using "ip addr" command in that file. If it only has 'iface lo inet loopback' then that is not the right one. Under netplan, you will only find yaml file. Look for 99_config.yaml. You should see the interface that we get using "ip addr" here. When using netplan we don't need to restart. It applies the settings directly without restart.

Command to use when netplan config is changed:
sudo netplan apply

What about global settings like DNS servers?
In Centos:
/etc/sysconfig, there is a file called network.

/etc/resolv.conf has DNS. By default it will pull from DHCP server. But we can also change it.
/etc/hosts will override resolv.conf. It has IP address at left and domain name at right.
/etc/hostname is not global settings. It is where our computer name come from. If we edit this file and reboot the settings will be lost. To make permenant changes use hostnamectl command.
sudo hostnamectl set-hostname Centos
It doesn't obey capitilization

Are there any easier ways to configure network settings?
It is easier using GUI when we have desktop version. If we don't have GUI many distro supports network manager. To find out if we have network manager, launch nmcli. It gives command lines interface and takes care everything in backend. It know where the config are present.
nmcli device status
After running above command it should show us some output like DEVICE, TYPE, STATE, CONNECTION with green font. If it doesn’t show any output then we are doing manual configuration. The state will show Connected for which nmcli is managing for others it shows unmanaged.

In Ubuntu
nmcli device status


It shows everything is unmanaged because of netplan migration. First understand whether the GUI is netplan or network manager. Based on that use the same method. If it is netplan use netplan. If it is network manager then use it. You can find whether it is managed using network manager or not using nmcli device status command.

Adaptername - etho

nmcli device show <adaptername>

To change any settings:
nmcli connection edit <adaptername>
The above command will directly take to adapter. You can give below command inside the adapter
set connection.autoconnect yes
set ipv4.method manual

After running above command use save command to save. But it will lost after reboot. If you want even after reboot, use save persistent to save it to hard drive.

Do, sudo nmcli connection reload

The above reload the connection. We notice there are lot of commands but it is best practice as we don't need to chase for files.

Thursday, 5 March 2020

How to fail in Microservices? - THAAT Conference - Pluralsight


  1. Using Mircoservice when not required for example
If deployment is only once per month
If the team is not ready
If the app just works fine with monolithic
  1. Testing
Changes can cause cascading failure
Others depend on our service

Implement red-green deployment - It will only affect small subset of people. Do testing directly in production. Do rollback
  1. No Devops
No Logging and Tracing
Distributed tracing -> Used to find which microservice in my code is taking long time to respond. It will help to plot the graph between service a to the end service. And also help us see which service took long time. Kubernetes has side-car. AWS has X-Ray.
If we don't have automated deployments.
If we don't have configuration management. For example: environment specific details Dev, Test, QA, PROD. It should be in a configuration service. Don't check-in hostnames

  1. POLYGLOT
Chose appropriate technology. Choosing many random technology makes support difficult
Difficult to do auto-deployment if we have too many different stacks like .net, java, go, python, ruby. use standard tech stack. Better to use single tech stack.
  1. Don't expect failures
Lot of failures in microservices than monolithic
Build microservices for resiliency by doing graceful degradation
Do health check before sending request to some other service
Retry again in case of failure. It can be a transient failure. Retry only at command level. Use circuit breakers at service level.
Process asynchronously

Golang - Email - Secure code warrior

 package mail import ( "net/smtp" "gobin/config" ) var ( emailConfig config.Email ) type Mail struct { Destinati...