Ansible Fundementals - With Image Partial

Creating a Static Inventory of Managed Hosts

Uses INI style and can also be defined in YAML. Inventory can also be populated dynamically

 

#will supply the path to the Ansible configuration file that's currently in use

ansible --version

 

#We can match the entire CIDR notation of 192.168.4.0/22 using below

192.168.[4:7].[0:255]

 

#To match server01 through server20. Notice we have used 01 instead of 1, so this will match 01, 02, 03 ... instead of 1, 2, 3

server[01:20]

 

#To convert inventory to YAML

ansible-inventory -y --list

 

#To check if the host is defined in inventory:

ansible washington1.example.com --list-hosts

 

#Default ansible config file

/etc/ansible/ansible.cfg

 

#Default inventory file location:

/etc/ansible/hosts

 

#Creating our own inventory file:

[webservers]

web01

web02

 

#Once inventory is added list it using below command

#Since we need output from our own file we use -i and point it to our file "inventory"

#The output will be in json format

ansible-inventory -i inventory --list

 

 

Managing Connection Settings and Privilege Escalation

In this section, we'll explore managing the connection settings and privilege escalation.

 

Agentless architecture.

Linux - SSH and Python

Windows - Windows RM and PowerShell

 

Inventory will describe the list of hosts that we wish for Ansible to manage, we now can go forward and understand how to describe to Ansible the various bits of information.

 

Ansible configuration file:

If you're unsure which configuration file you're using, you can use the ‑‑version flag for the Ansible command.

ansible --version

 

By default, Linux‑based systems will take advantage of the SSH protocol. If you wish to define a separate protocol or a non‑standard port, you'll need to describe that to Ansible as well.

 

The user being used to log into the system can also be described in inventory.

 

Once you've gained access to a system, if you need to escalate privileges to the administrative or root credentials, Ansible will need to understand how that occurs in your environment. By default, it will use the sudo command to do so. Other options exist, and if you do use a different privilege escalation method, such as the su command.

 

Lastly, you can describe to Ansible whether an SSH password should be provided or if key‑based authentication is in place.

 

All of these defaults can be adjusted within the Ansible configuration file or by passing a set of flags on the command line during invocation.

 

 

which Config File Ansible looks for in which order?

It is not uncommon to have several configuration files for different Ansible workloads in your environment.

 

1.Ansible does consult an ANSIBLE_CONFIG environmental variable. If this is set, it will be set to the path of an Ansible configuration file.

2.If environmnet variable is not set, Ansible looks for the configuration file in current working directory.

3.If it doesn't find an ansible.cfg file in the current working directory, it will then look in your home directory for a dot file, or hidden file. .ansible.cfg

4.Lastly, if it hasn't found a configuration file in any of those locations, it will use the default installation file at /etc/ansible/ansible.cfg.

 

Again, just a reminder that the flag ‑‑version for the Ansible command will clearly spell out which configuration file is being consulted. Be sure if you navigate around the file system you can check this because you may have switched directories to one that contains an alternative ansible.cfg file.

 

ansible.cfg:

The ansible.cfg file consists of several sections. Each section contains a heading and has a collection of key value pairs. The section headings, or titles, are enclosed within square brackets, and then the key value pairs are set as key equals value. The basic operations of Ansible executions take advantage of two main sections. One is the default section for Ansible operations, and the second is the privilege_escalation section where Ansible looks to understand how to gain privilege escalation when invoked for your managed hosts. The connection settings we discussed previously will be defined within the default section of the configuration file. This will include three main pieces of information for Ansible to understand.

 

default section: [defaults]

1.Remote_user will explain which user to take advantage of when connecting to managed hosts. If you do not supply a remote user argument, it will use your current username.

2.Remote_port specifies which SSH port you'll use to contact your managed host. By default, this is port 22.

3.The ask_pass argument controls how or whether or not Ansible will prompt you for an SSH password. By default, it does not prompt for a password, as it is most customary and a best practice to use key‑based authentication for SSH connections.

 

privileg escalation section: [privileg escalation]

In the privilege_escalation setting section of the configuration file, several main arguments are used for Ansible to understand how to escalate privileges to a higher‑tiered user, such as the root user.

 

The become key will describe whether or not you will automatically use privilege_escalation. This is a Boolean, and the default is set to no.

The become_user key will define which user to switch to when privilege escalation occurs. By default, this is the root user. The become_method key will determine how Ansible will switch to becoming that escalated user. Sudo is the default implementation; however, there are other options, such as su.

The become_ask_pass key will control whether or not Ansible prompts you for a password when escalating privileges. By default, this is set to no.

 

Example of an ansible.cfg file:

In general, an ansible.cfg file should contain only the keys you're overriding from defaults.

[defaults]

inventory = ./inventory

remote_user = ansible

ask_pass = flase

 

[privilege_escalation]

become = true

become_user = root

become_ask_pass = false

 

In a typical environment, not all hosts are equal, and there could be different properties we wish to set as variables on specific hosts.

 

One of the easiest ways to provide host‑specific variables is to create a host_vars directory. In that directory you'll create a text file that matches the hostname. Within this text file, you can supply a list of key value pairs that are unique to that host. Any variables provided in this fashion will override the one set within the ansible.cfg file. There's also a slight different syntax and naming when it comes to using this method.

 

Host‑based connection an privilege escalation variables:

Ansible_host will specify a different IP or hostname to use when connecting to the host instead of the one that specified an inventory. Think of this as a secondary IP or alternative hostname for that host. Ansible_port will specify the SSH port that you would prefer to use for connecting to that host.

Ansible_user specifies the user for that connection.

Ansible_become will specify whether or not you should use privilege escalation for that host.

ansible_become_user specifies which user to become on that host.

ansible_become_method specifies the methodology on how privilege escalation works, whether this be sudo, su, or something alternative.

 

Example of some host‑based connection variables in a host_vars subdirectory:

Here we have a subdirectory, host_vars, containing the file server1.example.com. The server1.example.com file contains variables specifically used when connecting and manipulating server1.example.com only.

 

File -> project/host_vars/server1.example.com

 

ansible_host: 192.0.2.104

ansible_port: 34102

ansible_user: root

ansible_become: false

 

No other servers will inherit this, but this will override any defaults that are contained in ansible.cfg when you interact with server1.example.com.

 

Creating ansible.cfg:

When creating this file, refer the default file in /etc/ansible/ansible.cfg. They have already listed the defaults, just take them and copy to your file and add setting only if there is change.

 

To filter the inventory:

ansible databases --limit db01 -m pint

 databases -> Heading in inventory

 

 

Running Ad Hoc Commands

Ansible provides a catalogue of modules. These are the underlying code that explains via code how Ansible can provide the automation tasks we'll leverage. Modules exist for a large number of system administrative tasks such as creating and managing users, installing and removing or even updating software, deploying configurations, as well as configuring the network services that run on your systems. Ansible modules are what is known as idempotent. In other words, they'll always check to see if the work being requested is required on the system or if it's already in the desired state. If a system is already in the desired state described by your Ansible work, then it will skip that and report back that no change was necessary. If a change is required, Ansible will then perform that change and report that as well.

 

An ad hoc command runs a single module against a specified host to perform a single task. To run ad hoc commands, we'll use the ansible command.

After the ansible command, you'll need to supply a host‑pattern. This host‑pattern will specify which host this task will run on.

Additionally, you'll need to specify a module using the ‑m flag. Each module takes a unique set of arguments, which you'll provide with the ‑a flag.

Lastly, you'll specify an inventory file with the ‑i flag, where the host can be found for Ansible.

 

One of the simplest ad hoc commands, as well as one of the most common system administrative tasks, is ping. The ping module doesn't actually send an ICMP packet like we're used to as system administrators using the ping command, but it does check to see if Ansible can contact the managed host. Specifically in a Linux‑default implementation, this would be using an SSH interaction.

 

ansible all -m ping

 

When using Ansible ad hoc commands, there're a number of flags available to the user to override default behaviors. The default behaviors are defined in the ansible.cfg configuration file.

We can see that it may be necessary to tell Ansible that we need a prompt for a password for our ad hoc command. You can use the ‑k flag or the ‑‑ask‑pass flag for this behavior.

When you need to specify a specific user for the interaction, the ‑u flag will allow you to do so. This will be overriding the REMOTE_USER setting contained within ansible.cfg.

A ‑b enables privilege escalation akin to the become argument within our configuration file.

The capital ‑K flag denotes that we need to be prompted for a password during privilege escalation, and the flag ‑‑become‑method overrides the default privilege escalation method.

 

With Ansible, the default is sudo. Other valid choices exist such as su and can be seen using the ansible‑doc command. Most Ansible modules take a set of arguments to describe the actions you wish for Ansible to perform.

The ‑a flag in an ad hoc command allows you to supply those arguments. Syntactally, we'll contain those within single quotes and put a space between each key‑value pair. Example,

 

‑m flag to declare the module user

‑a flag to specify those arguments.

 

Also important to consider is the concept of state. Here we've declared a state=present. We'll take a look through this course at state as that is a main approach of how Ansible has you describe the behavior you wish for it to perform.

 

ansible -m user -a 'name=newbie uid=4000 state=present' servera.lab.example.com

 

For example, if we wish to remove this user, we could change the state to absent and rerun this command. That would then remove the user we had just created.

 

Below is the helpful list showing some of the flags you have available to you during ad hoc commands.

 

Configuration Directive	Command-Line Option
inventory	-i
remote_user	-u
become	-b
become-method	--become-method
become_user	--become-user
become_ask_pass	-K

 

To take a look at the list of modules available to us for ad hoc or one‑off commands, we can use the ansible‑doc -l command.

With the ‑l flag, it will list all modules available to us. Use grep to filter

 

ansible-doc ping

 

I'll show a new technique of limiting to a single host. I'll then use an explicit statement for our inventory, the ‑k flag to tell Ansible that I wish for it to prompt me for passwords when authenticating to this host, and then a simple module call to ping. We're prompted for that password, and supplying our user's password for that system, the command continues. Note that instead of all systems responding here, only the limited web01 system responds.

 

ansible all --limit web01 -i inventory -k

 

 

Let's try an ad hoc command to restart sshd on one of our targeted hosts. I'll use an ansible command. This time I'll allow it to rely on the inventory we know it'll be using instead of explicitly stating it to show you how simple ad hoc commands can be. I'll target all hosts. I'll call the module service. And I'll supply the arguments for that module. We'll use the key state setting it to the value restarted. Additionally, we'll name the service we wish to restart.

 

ansible all -m service -a "state=restarted name=sshd"

 

Let's try one more example of ad hoc commands. Let's try to create users on our target machines. First, let's take a look at the Ansible module for user. We can note that the equal sign denotes mandatory fields, and not many of them are mandatory here, but we do have a lot of flexibility with the user module itself. I'll create a simple user by name. Here we can see that name is one of the mandatory fields, so I'll create a simple user using name and set a simple password. Let's use the user module to create a simple user across our web server systems.

 

ansible webservers -m user -a "name=test password=secret state=present"

 

Creating a Simple Playbook

Playbook:

Playbook contains one or more plays, a play is an ordered list of tasks to run against hosts in your inventory. Each task will take advantage of a specific Ansible module to perform some action against your managed hosts. Most of the tasks authored throughout the modules are idempotent and can safely be executed over and over again without issue. The intention of a playbook is to alter lengthy, complex manual system administration into easily repeatable routines. This should provide predictability, as well as reusability for the work you author with Ansible.

 

Create simple playbooks

First thing to know when formatting an Ansible playbook is about YAML. YAML is a simple to author structure with standard file extensions ending in .yml. Two‑space indentation with the space character only is the main concept behind the syntax within YAML files. Note that the spaces cannot be substituted with the tab character. The tab character is not allowed in proper YAML. YAML doesn't place strict requirements on how many spaces are used for the indentation, but the two basic rules come into play

Data elements at the same level in the hierarchy must align with the same indentation.

Items that are children of another item must be indented more than their parents. All children of the same data element, again, must be indented with the same indentation.

Proper playbooks always begin with three dashes to denote the start of a file. These will be all the way left justified.

Failures will result in immediately halting the play execution.

Within our inventories, we may not always find it appropriate to target every node within a group or even the entire inventory. This is where the ‑‑limit flag will allow us to target specific hosts within our inventory. The limit is a host pattern that further limits the hosts for the play. Given our playbook targeting all hosts, we could then supply a ‑‑limit argument and call out a singular host, or even a host pattern for this to execute upon.

ansible-playbook site.yml --limit datacenter2

The Ansible playbook command also provides us a helpful syntax‑check argument. We can call ansible‑playbook, passing in the argument for ‑‑syntax‑check. Then, just simply name your YAML file you wish for the syntax check to be performed upon. If any errors are found, Ansible will do its best to denote where in the file that error exists.

ansible-playbook –syntax-check webserver.yml

It can often be advantageous before performing the actual execution of a playbook to do a test or dry run of its work. The Ansible playbook argument provides the ‑C argument to be able to do just that. You can see an example here of ansible‑playbook using the ‑C argument on the webserver.yml playbook file. The resulting output simulates what would occur if you remove that flag, but does not actually perform that work. Once the work has been validated and you approve for this to carry on, simply remove the flag and run this again.

Gathering Facts is a built‑in feature of Ansible executions where Ansible will profile all the targeted hosts to understand as much as it can about them.

 

Using Variables in Plays

Naming our variables:

There are a few rules. Variable names always start with a letter. Additionally, they can only contain letters, numbers, and underscores. Periods and dashes are not allowed in variable names.

Scope of variable

Once you've begun to create variables, it's important to understand the scope or the available reach for each variable you've created. The concept of global, host, and play‑based scopes exists.

Global variable -> One that is set for every host. An example of this would be extra variables we create within a job template.

Host‑based values -> Are set for a particular host or host group. These would include variables we set in the inventory or in our host_vars directory as explored in a previous module.

Play‑scoped variables -> Are available for all hosts in the context of a currently executing play. These play‑based scoped variables include things included in the vars directive at the top of a play or in the include_vars tasks contained.

 

Variable Precedence:

When variables are defined in multiple places, precedence also has to be considered. If a variable is defined at multiple levels, the level with the highest precedence will take over. A narrow‑scoped variable, in general, will take precedence over a wider‑scoped variable. Considering the types we discussed in our previous slide, this would mean that a play‑scoped variable would override a global‑scoped variable. Variables defined within a playbook are overridden by extra variables defined on the command line during execution. To override in this manner, simply provide the ‑e option and the substituted value for any variables you wish to override when you're calling ansible‑playbook.

Defining Variable:

A common method is to place a vars block at the beginning of the play and then list the variables you wish to define. You can see an example of vars being defined in this way in this block where the user_name and user_state are defined in a vars block at the top of a play.

- hosts: all

  vars:

    user_name: joe

    user_state: present

You could additionally define these variables in an external file. If you do so in this manner, we use the vars_files argument at the top of a play to load variables in from a file located elsewhere. You can see an example here where the vars_files block is created and the relative path to the vars directory and a users.yml file has been provided.

- hosts: all

  vars_files:

    - vars/users.yml   

 

Referencing Variable:

Once defined, variables can then be used within your tasks contained in a playbook. When we're ready to reference a variable within a play execution, we'll substitute its value by using double braces {{ variable_name }}. The double braces will contain the name of the variable we wish to substitute in. You can see an example here where we have a variable defined in the vars block at the top of the play as use_name. The value this is set to is joe. Within our task, we're creating the user Joe by using variable interpolation. You can see the double brace nomenclature utilized to substitute the value joe for the variable user_name. We're doing so in two places, both in the name of the task, as well as in the name provided for the argument for the user module.

- name: Example play

  hosts: all

  vars:

    user_name: joe

 

  tasks:

    - name: Create user {{ user_name }}

      user:

        name:"{{ user_name }}"

        state: present

When referencing one variable as another variable's value, the double brace will start the value. When it does, you may also need to quote around this value. This will prevent Ansible from interpreting the variable reference as starting a YAML dictionary. Ansible provides the helpful hint that the with_items without the quotation marks should be written as with_items including the quotation marks around the double braces -> :"{{ user_name }}".

 

Host‑ based variables and group‑based variables

 As the names denote, host variables apply to a specific host, while group variables apply to all hosts in a host group or group of groups. Host variables will take precedence over any group variables supplied on a host, but variables defined inside a play will then override either of these. You can define both host and group variables in the inventory itself or in subdirectories that contain YAML files that match the names of the host in a host_vars subdirectory or group in a group_vars subdirectory. These YAML files will then contain the list of variables you wish applied with those scopes. Variables defined in the host_vars and group_vars directory have a higher precedence than those defined as inventory variables.

 

To utilize this technique, you'll need to create directories at the same level as your Ansible playbook. Creating the two directories, group_vars and host_vars, will allow you areas to provide YAML files to define variables with this technique. If we had a group defined an inventory named servers, we could then create a subdirectory group_vars that contains the YAML file, servers. Any variables we define in the servers file will then be supplied as variables on all hosts in the servers group. In the example to the right, we see proper YAML syntax for setting variables in this fashion.

ansible_user: devops

newfiles:

• /tmp/a.conf
• /tmp/b.conf

The ansible_user variable is set to the string devops, while the newfiles variable is a list of two different values. If you wish to create variables for a specific host with this technique, create a host_vars directory and contain those variables and a YAML file that matches the host's name.

 

Here's a look at a proper file hierarchy that has examples of this technique.

For example, we could reference the users variable and then the aditya user's username and then their first name by the syntax users, opening the bracket, opening the quotation mark, and naming the username Aditya, and then following that with an open bracket and open quotation mark for the fname.

In a similar fashion, we can get Carlotta's home directory reference with users open bracket, open quotation, Carlotta. Close both of those. And then open bracket, open quotation home.

Register Statement

The register statement will allow us to capture the output of a task and store it in a variable during execution. The output is saved into a temporary variable that could be used throughout the rest of the playbook for either debugging or utilization for another task.

 

This is a common technique that allows us to take advantage of the return values from each module, store them in a variable, and reuse them throughout the rest of our workloads.

These registered variables are only stored in memory and are destroyed once playbook execution completes.

 

From our previous play, let's have a look at what it looks like currently.

We can evolve the value that we're supplying for the username test as a variable at the top of the play.

Alright, up here in the heading keys, we can add a new key for vars. Since this is a child, we'll need to two‑space indent beneath it, but we'll simply supply the vars. Let's create a variable named username. We'll set this user name to test. We'll then utilize it down below in the play by doing variable substitution. Since we're substituting a variable, we'll needs a first enclose quotation marks, the double braces, and include the variable name in the middle here, username. 

ansible-playbook example.yml

 

Let's supply a command line variable substitution.

ansible-playbook -e “username=student”

We'll say ansible‑playbook and, providing extra variables, we can supply username and set it equal to a second username. It will add both test user and student user.

Oftentimes, we may want the variable to contain a list of values. For example, we may have additional information we want to supply. We'll transform the username variable into more of a dictionary of values. Here, I'll remove test for now and then begin building child values underneath username. Each of these child keys will be indented two spaces further. And as we provide more values for the test user, we can then further indent two additional spaces. (Working) Now that we've evolved our variable, we'll need to update the interpolation below. The notation we'll use here will involve brackets and single quotation marks to iterate through the fields. Let's also take advantage of that new value that we've supplied. The user module also provides a key comment to allow for additional commentary within the etc/passwd file.

We've previously taken a look at the host_vars concept, but let's also evolve our playbook to take advantage of that technique. I'm going to go ahead and create the group_vars alongside the host_vars directory. This is what we currently have.

To clean up our work for further exercises, I'm going to go ahead and remove the db01 host variables we previously set by simply deleting that file. After that clean up, we have the current structure in place.

Since we've been performing the work on the webservers group, we can create a file in the group_vars directory for the webservers group. We can migrate the variables we just created into that file and take advantage from the playbook in the exact same fashion.

Let's create the file group_vars/webservers. In this file, we'll simply paste our variable information from playbook.

 It can be helpful to provide a comment at the beginning of each file, so we understand what the file's intention is. In example.yml remove those values. Since we have no additional variables currently, we can leave the key and have it blank. But since we don't provide variables, I'll go ahead and remove it as well. It's best to keep your playbooks as clean as possible.

Ansible-playbook example.yml

 

 

Ansible Vault:

 

Create a new encrypted file:

ansible-vault create filename

 

View an encrypted file

ansible-vault view filename

 

Edit an encrypted file

ansible-vault edit filename

Encrypt existing file

ansible-vault encrypt filename

 

Save to new file

--output=new_filename

 

Decrypt file

ansible-vault decrypt filename

 

Now that we have encrypted information, we'll want to use that within our playbooks. We can provide the vault password that we set when encrypting the file with the ‑‑vault ‑id option. You can see an example command

ansible‑playbook ‑‑vault ‑id @prompt filename

 

The @prompt option ensures that Ansible understands it needs to receive user input for the password. If you do not provide that password, Ansible will return an error.

You may have different passwords for various files that are encrypted using Ansible Vault. When you need to supply multiple passwords, we'll have to understand the technique that allows us to do that. Using the ‑‑vault ‑id option, we can set labels on the encrypted file. We can then use this as many times as necessary to label the various files we have encrypted and ask Ansible to prompt us for the different passwords when we need to supply them. Have a look at this last example.

ansible‑playbook ‑‑vault ‑id vars@prompt –vault-id playbook@prompt site.yml

Ansible‑playbook calls the vault‑id and supplies a vars@prompt argument. It calls it again, providing a playbook@prompt argument before then calling the playbook site.yml. Ansible will then prompt you with this execution for two different passwords, one for vars and one for playbook. Given that you provide the two appropriate passwords, the files will be decrypted when utilized by the playbook, and execution will proceed; else Ansible will provide an error.

 

Once we've created a password, we may need to change that on an encrypted file.

Ansible-vault rekey filename

To change the password of an encrypted file, we'll use the subcommand rekey for the Ansible Vault command. You can use this subcommand on multiple data files at once, providing a helpful way to rekey a bunch of files to the same password. The rekey subcommand will prompt for the current password and then the new password you wish to set for these encrypted files.

While we're discussing sensitive information, sometimes Ansible output can include sensitive values. When this is the case, you may want to suppress the output from a given task that could do that. When we want to suppress that output, we can use the key no_log. By using this value, Ansible will suppress the output of the task so that sensitive information is not displayed. Have a look at these two examples.

 

PKI Certificate

PKI Uses

Understood Nothing

You've probably heard the term PKI being used in a work environment. And if you're into IT security, you'll be well familiar with the term. PKI stands for Public Key Infrastructure. Essentially, what it is, if we can water it down to a single sentence, is it's a hierarchy of digital security certificates, where unique public and private key pairs are issued for each certificate. So certificate authorities, otherwise called CAs, are authorities that issue certificates. They can renew certificates because certificates have an expiration date, and they could be renewed before they expire. They also do have an expiration date where they will simply expire by themselves, where they can no longer be used.

And also certificate authorities are responsible for revoking certificates, maybe due to compromise of a station or because a user is leaving the organization. We also have the option of working with our own private internal certificate authority. You might install a product like Microsoft Active Directory Certificate Services or maybe you would use OpenSSL. But either way you can have a private CA where you sign your own certificates. The only issue with that is that devices out there on the network don't trust your private, self-created certificate authority.

So you would have to install the root certificate for your CA on those devices. Of course, we've got public CAs where we could have a certificate signing request that we've generated issued up to the public CA. And after they've done their due diligence, their CA can digitally sign and return our certificate. We might do that, for example, if we need a website certificate that's trusted globally. So what is PKI used for? Well, it uses a public key. And the public key can be shared publicly with any user or device, that's why it's called a public key.

So the recipient public key then would be used when we encrypt a message that we're sending to that recipient. We need their public key to encrypt it. Public keys are also used to verify digital signatures. Now the private key must be available only to the key owner, whether that's a device or whether it's a user. So it's not like a public key. Private keys are going to be kept private only to the owner, and they can also be embedded in things like cards, like smartcards. So encrypted messages then are decrypted using this key.

Remember up above you need a public key to encrypt the message to somebody. You need their public key. Well, they need their mathematically related private key from the key pair to decrypt that message. The private key is also used to create a digital signature. So if we have a need, let's say, of e-mail confidentiality, how does PKI serve this need? Well, we can encrypt the message with the recipient's public key. The second item here is what if we need e-mail authenticity and integrity? Well, a solution is to generate a message hash and encrypt that hash with the sender's private key.

And often, that's done seamlessly and transparently in modern e-mail programs. What if we have a need to secure network communication to a web server? Well, we would use a PKI certificate issued to that web server and make sure that TLS version 1.1 or higher are used. And then we would enable HTTPS on the web server. Now TLS version 1.1 is a secure way to secure network communications using a PKI certificate.

Using the older SSL security protocol should never be done due to its large number of vulnerabilities. And there are even vulnerabilities associated with TLS v1.0. And that's why we're saying here you should start at TLS version 1.1. What if we have a need to use Multi-Factor Authentication to a VPN? Well, PKI could solve that because we might embed a private key on a smartcard, which would be used. The smartcard would have to be available and presented when we need to authenticate the VPN.

Finally, what if we have a need to have a single card, a physical card that people carry with them, that would allow things like facility access, as well as computer access? Well, that's called a Common Access Card or a CAC. Essentially, it's a smartcard that has multiple uses that contains an embedded private key.

 

PKI Hierarchy

Understood Nothing

The PKI hierarchy is a hierarchy of digital security certificates, where certificates are issued and also managed after they're issued by a certificate authority. We could have a private certificate authority or a private CA that we set up within our own organization and that private CA issues certificates and manages them. The only thing we would have to consider is on all devices that need to trust certificates issued by that private CA, we would have to install that private CA's root certificate.

Public CAs are available for a fee as well. Where we could send a certificate signing request up to the public CA, either through a web form or through an e-mail file attachment. And after the public certificate authority does their due diligence to ensure that the information that we have provided in the request is valid, they can use their CA, private key to digitally sign our certificate.

And if it's a public CA, it will be trusted globally. So we might do this for example, if we have a public website that we want to secure using HTTPS.

The PKI hierarchy starts at the very top with the root certificate authority or root CA. Now from there, the root CA could issue certificates directly, but in a larger enterprise, you would have subordinate CAs.

Maybe for different parts of the world, different countries, different provinces, different regions. Or even within a large corporation, we might have different CAs for different departments, or different projects. And each of those subordinate CAs, in turn, can issue its own certificates.

A diagram explaining the PKI hierarchy is displayed. The Root CA is at the top. The Root CA is connected to Subordinate CA1 and Subordinate CA2. Subordinate CA1 and Subordinate CA2 are connected to Certificates. 

Now if that's the case, then the root CA doesn't need to be there, doesn't need to be online and available all the time, unless you're going to be doing something like creating a new subordinate CA. So the issuing certificate authority digitally signs issued certificates. And as we were saying, the root CA really, from a security perspective, should be brought offline when it's not being used.

The reason is because if the root CA is always left online and by chance, it gets compromised. Now it has a higher degree of being compromised if it's always online. If that root CA at the top of the hierarchy is compromised, all subordinate certificates are compromised, including subordinate certificate authorities. So you can see why this is a big deal.

Now, if we have a compromised subordinate CA, then certificates issued by that CA are compromised, nothing above it in the hierarchy. So it's always important then that we consider the PKI hierarchy when we're planning our public key infrastructure, or if we're making changes to an existing configuration and trying to harden it.

 

Create a PKI Certificate Authority Using Linux

In this demonstration, I'm going to use Linux to create a PKI Certificate Authority. The first thing you're going to need to do is to make sure you have the appropriate module or tools installed to work with PKI in Linux. So for example, I could use the apt-get install command if I wanted to install the openssl package. Here I can see that openssl is already installed with the newest version.

He opens the root@kali: / window on the Linux operating system. The root@kali: /# prompt is displayed. Then he executes the following command: apt-get install openssl. The output displays the status of the latest version of the openssl installed. 

So now what I'm going to do is make a directory here on the root of the file system, and I'm going to call it pki. And I'm going to change directory into the pki directory. Because any files that are generated, I want organized in this location.

So what I want to do then is use openssl, this is the command that's built-in once you've installed this. And I got to tell it I want to generate some RSA keys, rsa, I want to use aes256 bit keys. So AES is the algorithm. And I have to give it the name for an output file. So I'm going to call this let's say ca_priv for private.key. That's the private key file that will be generated. And then we're going to tell it that at the RSA level, we want to make sure that we use 2048 bits. And we going to go ahead and press Enter.

 

Generating CA Private Key:

He executes the following command: openssl genrsa -aes256 -out ca_priv.key 2048. The output displays a message: Generating RSA private key, 2048 bit long modulus, and asks to enter a pass phrase for ca_priv.key.

Okay, so we now have a private key that's 2048 bits, that's good. So the next thing we're going to do is enter a pass phrase for the private key. I'm going to put in a password, and then I will verify that same password.

Now the next thing that we want to do is actually generate a CA or certificate authority certificate. So to do that I'm going to run openssl req for request -new and then -x509. x509 is the standard that's used for PKI certificates. I'm going to specify my private key with -key. So that was called ca_priv.key. Then I'm going to specify the signature algorithm, because remember a certificate authority digitally signs certificates.

Here it's going to use sha256. I'm going to specify in days, the lifetime of this certificate, so it's going to be let's say, only 365 days, that's not very long for a CA. Commonly it might be five years, maybe even ten years, but here I'm just going to make it one year. And I'm going to specify the name of the output file for this CA certificate. So -out and I'm going to call it ca, let's say .pem, standard file format.

He executes the following command: openssl req -new -x509 -key ca_priv.key -sha256 -days 365 -out ca.pem. The output prompts to Enter the pass phrase for ca_priv.key. 

Now because we referenced our private key it wants the pass phrase for it, that's good. You always want a pass phrase on a private key file. So I'll go ahead and enter that, and then I have to fill in some details because we're establishing a certificate authority, so it has questions. So the country name, two letter code, in this case, for Canada, I'll put in CA.

He enters the pass phrase and presses the Enter key. The output includes a message that reads, You are about to be asked to enter information that will be incorporated into your certificate request. Country Name (2 letter code) [AU]:. 

 

 State or province, I'll fill in the appropriate value, same with the locality, that's city name. Locality is filled in, the organization name, I'll just put in here as FakeCo and organizational unit, maybe Hq, headquarters. And a common name, so this would be for my certificate authority, I'm going to call it MyCA. And then an email address for a user. So that would be an administrative account, for example, so I'll fill that in.

He enters the email address as user@fakeco.com. He again presses the Enter key and the prompt remains the same. 

ca_priv.key -> Private Key

ca.pem -> Certificate Authority

 

Web of Trust

When you acquire a passport or a driver's license, that is trusted by other government departments, and agencies, and institutions, because they trust the issuer of the passport or the driver's license. And with Public Key Infrastructure or PKI, the same type of web of trust exists.

So in the PKI hierarchy, the issuing certificate authority or CA digitally signs issued certificates using its private key. Now, only the CA owner has access to that private key and the signatures created from it. Now, the signature is then used to establish trust, because if we trust at the certificate authority then by extension, we also trust any certificates that have that certificate authority digital signature. Pictured on the screen, we've got a screenshot on the right of the trusted root's certification authority's store on a Windows machine.

The screenshot of Trusted Root Certification Authorities displays a table which includes Issued To and Issued By column headers. 

 

 In other words, different types of operating systems, in this case, Windows, have a list of public certification authorities who they trust. And if they trust that public certification authority, by extension, they trust any certificates issued by that authority. So devices then, like your laptop, your desktop, your smartphone, needs to trust certificate signers. Now, we're looking at the Windows trusted certificate store, that's what we were talking about. But the same type of system exists on a Linux host using the Linux CA certs key store.

So certificate trust stores then have a list of trusted root and subordinate certificate authorities. And, as we know, if you trust the certificate authority, then you also trust all the certificates issued by that certificate authority. That's what the web of trust is about. Now, with private certificate authorities that you might create within your own organization, as opposed to public certificate authorities. Certificate trust stores on devices will not automatically trust the CA. They won't contain the certificate, the root certificate, and so they will not trust any issued certificates.

So the trusted root certificate then you would simply have to import on devices in your organization. Now, the private CA will issue certificates and remember that they will then be trusted on devices. The trust is established based on the CA signature contained within issued certificates. So again, the overall theme is very simple. If you trust the certificate authority, then you trust all of these certificates that are issued by that authority. Just as in the same case, if you trust an issuer of passports within a country, then you will trust all of the passports issued by that country.

 

PKI Certificates

With Public Key Infrastructure, PKI certificates are issued by a certificate authority, by a CA. And these certificates could be issued to a user. Or may have a certificate issued to a device like a smartphone that needs to authenticate to a VPN. Or we can even have certificates issued to be used by software components or services. So these certificates are issued by the CA for security purposes such as to encrypt files or encrypt network communications.

To verify file integrity or message integrity over a network to ensure it's not been changed. And also for authentication such as over the network with messages so we can trust that the message came from the sender it says it came from. So we can store PKI certificates in many different places. It might be stored in a file, in the file system on a device. A PKI certificate might be stored on a smartcard. So there are many different ways that we can store these PKI certificates.

As a matter of fact, if you're using a business computer like a laptop, you might even have the trusted platform module TPM firmware chip which can also incidentally store PKI certificate information. So what's in a PKI certificate anyway? How does this work to allow security in an environment? Well, we know the PKI certificate is issued by a CA and that it can be stored in a variety of different ways. What's in it?

Well, the version of the X.509 standard that's used to generate that PKI certificate. The digital signature of the certificate authority, which is important to establish trust. The signature algorithm that was used by the certificate authority to create the signature. For example, maybe it was SHA256. The certificate serial number is certainly stored within the certificate.

The date the certificate was issued, when it expires, how the certificate and the keys can be used whether only for encryption, or maybe only for authentication, or both. The subject name which is important. The subject name could be the URL of a website, which has to match what people are typing in for the certificate to be trusted. Or maybe the subject name is a user e-mail address, if we're using user certificates to encrypt and digitally sign e-mail communications. So the certificate can also contain public and, in some cases, private keys.

Now the purpose here would be for encryption and decryption. For example, we would encrypt with the recipient's public key if we sent something to them. And they decrypt with the related private key. Digital signatures are created with the private key and verified with the public. In some cases, the private key will not be stored within a certificate. So it really depends on the environment and how the certificates are being used.

PKI certificates can also be issued from templates depending on the solution that you're using. And as you would guess, the template is just a blueprint of all of the details that will be placed within the PKI certificate. The items that we just talked about. Certificates and public or private keys can also be exported.

Now if we think about this, we might have a certificate that contains a private key. And we might want to backup the private key in a safe place. So we can actually export it as long as when that certificate was issued, exportation of the private key was marked as being allowed. Now public keys can be shared with anybody. And so it's okay to provide a public key to anyone over the internet, whether you trust them already or not, however, not the private key. And as we said, you might export the private key from a certificate for backup purposes. Private keys should only be available to the entity to which it was issued, be that a piece of software, or user, or a device.

Now in the PKI certificate, we mentioned the subject name field. And we said that this must match the entity such as the URL of the website or the e-mail address of the user. So if that's not the case, then we won't have trust established. So in the case of a secured website, if the URL that people are typing in doesn't match the URL in the subject name field in the certificate, then depending on the web browser people are using will determine exactly what message pops up. But in essence, that site will not be trusted. And that is not a good thing.

 

Issue a PKI Certificate Using Linux

In this demonstration, I will use OpenSSL in Linux to issue a PKI certificate to be used by a Linux web server.

The first time here, we're going to use it to generate a private key file that will be used by our web server certificate. And to do that, I'll run openssl genrsa, I want to generate an RSA type of key, that's the algorithm that will be used. And the output file will be, let's say, the file name can be anything, but I'm going to say www.site.com.priv.key. So I know it's the private key and I want it to be a 2048 bit RSA key.

He executes the following command: openssl genrsa -out www.site.com.priv.key 2048. The output displays a message: Generating RSA private key, 2048 bit long modulus. 

 

Now, I have to generate a certificate signing request. What I'm doing is preparing a file here that needs to be submitted to the certificate authority whether it's on this host or whether we send it through e-mail or paste it in to a web form field. One way or another, the certificate signing request has to be sent to the CA because we need the CA to digitally sign it and send it back to us. So let's generate a certificate signing request, openssl req -new.

And I have to tell it to the key -key, the private key that is, that I've generated previously for this web server certificate. And then I have to specify an output file for this certificate signing request. So how about we call it www.site.com.csr, for certificate signing request.

He executes the following command: openssl req -new -key www.site.com.priv.key -out www.site.com.csr. The output includes a message that reads, You are about to be asked to enter information that will be incorporated into your certificate request and the field: Country Name (2 letter code) [AU]. 

It's going to ask me for a bunch of stuff like the Country name and so on. I'm going to specify, let's say, CA two character country code, state or a province of some kind, so I'll spell that out. A city name, organization name, and I'll just fill in some values here. And the common name is important because if people will be entering, let's say, www.site.com in a web browser to access my secured web server, then I'm going to have to make sure that that is in the certificate. So I'm going to put that in as the Common Name, for the Email Address, maybe admin@fakeco.com.

He types CA for the Country Name (2 letter code) [AU] and presses the Enter key. The output prompts: State or Province Name (full name) [Some-State] and he types Nova Scotia. He presses the Enter key, the output prompts: Locality Name (eg, city) [], and he types Halifax. He presses the Enter key, the output prompts: Organization Name (eg, company) [Internet Widgits Pty Ltd], and he types FakeCo. He presses the Enter key, the output prompts: Organizational Unit Name (eg, section) [], and he types Hq. He presses the Enter key, the output prompts: Common Name (e.g. server FQDN or YOUR name) [], and he types www.site.com. 

Now, when you do this with public certificate authorities out on the internet, you don't just make this stuff up. It has to be filled in correctly. So do I want a challenge password to be used with this signing request? No, so I'll just press Enter, optional company name, Enter.

And if I clear the screen and type ls, we now have a certificate signing request file, csr.

So as you might have guessed, the next order of business is we need to ask the certificate authority, which is on the same host, it doesn't have to be. But we've got the private key for the certificate authority and the certificate file for a certificate authority I've previously created already available here.

He highlights the private key: ca_priv.key and certificate file: ca.pem in the output. 

 

So that in conjunction with the certificate signing request will allow us to actually generate a usable PKI certificate for the web server.

In the real world, when you request a certificate from a public authority, they'll do some due diligence to make sure that the information you provided is accurate and correct. So trust is very important with this. Now notice here, I've got a file I've already prepared called otherinfo.ext. It's extension, or additional information, you can think of it as a template that will be used to issue certificates. And if I used the cat command to display the contents of that file, you're going to see what I've typed in here.

So I've typed in a bunch of stuff, notably, keyUsage, how can this be used, for digitalSignature, for keyEncipherment. In other words, for encryption of keys and also for the encryption of data, dataEncipherment. But what's really important here is the subjectAltName, san, S-A-N which is referring to a section further in the file called @alt_names. And here it is down here. So the first DNS name, DNS.1 is going to be the URL people will enter to connect to my site, www.site.com. So I actually want that injected into the certificate that we're about to issue. Okay, so let's just clear the screen, let's just do an ls so we can see our file names again here.

So I've created the command here. It's openssl x509, that's the type of certificate we're creating, a PKI certificate. We are looking at a request for an actual certificate, and we're getting that from our certificate signing request input file, so -in. And I'm just referring to the name of the CSR that we generated. Then I'm going to specify the CA certificate file which happens beyond this machine that's called ca.pem, then the CA private key file, same thing. It's on this machine.

I'll specify that and then the output file I want to create here which is my actual web server certificate file, the number of days that the web server certificate will be valid. So here it's only one year, 365 days. The signature algorithm that will be used by the CA to digitally sign the certificate, in this case, sha256 is the algorithm that will be used, telling it to create a serial number. Because PKI certificates are uniquely tracked by a serial number and then I'm specifying to use that otherinfo.ext file with the extensions file parameter -ext file. And remember, that maps our subject name to our alternative name, so the URL of the host. And I'm going to go ahead and press Enter.

He executes the following command: openssl x509 -req -in www.site.com.csr -CA ca.pem -CAkey ca_priv.key -out www.site.com.crt -days 365 -sha256 -CAcreateserial -extfile otherinfo.ext. The output prompts: Enter pass phrase for ca_priv.key. 

So of course, it wants the pass phrase for the CA private key, I'm going to go ahead and specify that value, and then I'll press Enter. And that's it, it's done. If I clear the screen with the clear command and type ls, we've now got an actual PKI certificate file that can be used, in this case, for a web server, such as with the Apache web server.

So then when we configure our web server engine whether it's nginx or Apache or whatever it is to use that as the PKI file to secure the connection normally over port 443.