Ubuntu

We have initialized a new repository for our project. Now we will learn how to store file modifications using git add and git commit.

Getting ready

Make sure you have initialized a new git repository and created sample files under your project directory. Follow the previous recipes to get more details.

How to do it…

Now that we have a new repo initialized for our project, let's go ahead and check in our files.

Before we add any files, simply check the current status of the repo with the git status command. This should list all the files under the Untracked files list, as follows:

$ git status

As shown by git status, none of our files are being tracked by Git. We need to add those files before Git tracks any changes to them.

Let's add all the files to the tracking list with git add:

$ git add .

This command does not create any output, but stages all untracked files to be added to the repo. The symbol (.) specifies the current directory and processes all files under the current directory. You can also specify file name(s) to add specific files.

Now check the git status again. This time, it will show newly added files marked by green text and a message saying Changes to be committed:

Next, commit the current state of the files with the git commit command. Commit means asking Git to save the current state of staged files:

$ git commit -m "First commit"

The git commit command will display details of updates to the repository, along with the commit ID (4459fcc). In this case, we have added three new files without any new insertion or deletion of contents.

Now if you check the git status again, it should show the nothing to commit message:

$ git status

On branch master

nothing to commit, working directory clean

Next, make some changes in any file and check the repo status again. This time, it should show the modified files as follows:

You can check the exact differences to the previous version and current modifications with the git diff command. Use git diff without any file name to get all modifications in all files, or use it with a file name to check specific files:

$ git diff

Now you can repeat the add and commit process to store these changes. We have modified an existing file without creating new files. We can use the -a flag with git commit to stage changes and commit them in a single command, as follows:

$ git commit -a -m "index.html updated"

The -a flag will stage all modified files and commit will proceed with newly staged contents. Note that this only works with modified files. If you have created any new file, you need to use git add to stage them.

How it works…

This recipe uses two primary commands: git add and git commit. The first one stages the content for the next commit, and the second actually stores the current state of the content. The git add command is used to add new files, stage updates to existing files, and remove any entries of deleted files. All these modifications to the current working tree are staged for the next commit. The command can be used multiple times to stage multiple modifications. Additionally, you can stage all files under the current directory at once by adding a single file, naming it explicitly, or even choosing a single line from a bunch of updates in the single file.

Once the modifications are staged, you can use git commit to store the updates. When the changes are committed, Git stores the updates in the revision history and changes Git Head to point to the latest revision. All updated files are stored in the form of a binary large object (blob) as a new snapshot. The commit process also triggers some hooks or events that can be used to execute external scripts to carry out some additional functions. Later in this article, we will discuss Git hooks in more detail.

Other than git add and git commit, we have used git status and git diff commands. As the name suggests, git status shows the current status of the repository in question. It lists all files that have been modified after the last commit, newly created or deleted files, and any updates that have already been staged. The git diff command can be used to list all modifications to a given file. It compares the current state of a file against its last committed or indexed state. Note that you can use git diff before indexing any file with git add.

There's more…

Another useful command is git checkout. It can be used to discard any modifications and restore a file to its previous state, or restore the deleted file to its known revision.

DHCP is a service used to automatically assign network configuration to client systems. DHCP can be used as a handy tool when you have a large pool of systems that needs to be configured for network settings. Plus, when you need to change the network configuration, say to update a DNS server, all you need to do is update the DHCP server and all the connected hosts will be reconfigured with new settings. Also, you get reliable IP address configuration that minimizes configuration errors and address conflicts. You can easily add a new host to the network without spending time on network planning.

DHCP is most commonly used to provide IP configuration settings, such as IP address, net mask, default gateway, and DNS servers. However, it can also be set to configure the time server and hostname on the client.

DHCP can be configured to use the following configuration methods:

Manual allocation: Here, the configuration settings are tied with the MAC address of the client's network card. The same settings are supplied each time the client makes a request with the same network card.

Dynamic allocation: This method specifies a range of IP addresses to be assigned to the clients. The server can dynamically assign IP configuration to the client on first come, first served basis. These settings are allocated for a specified time period called lease; after this period, the client needs to renegotiate with the server to keep using the same address. If the client leaves the network for a specified time, the configuration gets expired and returns to pool where it can be assigned to other clients. Lease time is a configurable option and it can be set to infinite.

Ubuntu comes pre-installed with the DHCP client, dhclient. The DHCP dhcpd server daemon can be installed while setting up an Ubuntu server or separately with the apt-get command.

Getting ready

Make sure that your DHCP host is configured with static IP address.

You will need an access to the root account or an account with sudo privileges.

How to do it…

Follow these steps to install a DHCP server:

Install a DHCP server:

$ sudo apt-get install isc-dhcp-server

Open the DHCP configuration file:

$ sudo nano -w /etc/dhcp/dhcpd.conf

Change the default and max lease time if necessary:

default-lease-time 600;

max-lease-time 7200;

Add the following lines at the end of the file (replace the IP address to match your network):

subnet 192.168.1.0 netmask 255.255.255.0 {

range 192.168.1.150 192.168.1.200;

option routers 192.168.1.1;

option domain-name-servers 192.168.1.2, 192.168.1.3;

option domain-name "example.com";

}

Save the configuration file and exit with Ctrl + O and Ctrl + X.

After changing the configuration file, restart dhcpd:

$ sudo service isc-dhcp-server restart

How it works…

Here, we have installed the DHCP server with the isc-dhcp-server package. It is open source software that implements the DHCP protocol. ISC-DHCP supports both IPv4 and IPv6.

After the installation, we need to set the basic configuration to match our network settings. All dhcpd settings are listed in the /etc/dhcp/dhcpd.conf configuration file. In the sample settings listed earlier, we have configured a new network, 192.168.1.0. This will result in IP addresses ranging from 192.168.1.150 to 192.168.1.200 to be assigned to clients. The default lease time is set to 600 seconds with maximum bound of 7200 seconds. A client can ask for a specific time to a maximum lease period of 7200 seconds. Additionally, the DHCP server will provide a default gateway (routers) as well as default DNS servers.

If you have multiple network interfaces, you may need to change the interface that dhcpd should listen to. These settings are listed in /etc/default/isc-dhcp-server. You can set multiple interfaces to listen to; just specify the interface names, separated by a space, for example, INTERFACES="wlan0 eth0".

There's more…

You can reserve an IP address to be assigned to a specific device on network. Reservation ensures that a specified device is always assigned to the same IP address. To create a reservation, add the following lines to dhcpd.conf. It will assign IP 192.168.1.201 to the client with the 08:D2:1F:50:F0:6F MAC ID:

host Server1 {

hardware ethernet 08:D2:1F:50:F0:6F;

fixed-address 192.168.1.201;

}

This recipe covers the installation of the Ampache server. It is a simple PHP-based web application. Once installed and set up, you can use a web interface to play your audio/video files or use any of the various popular streaming clients to stream content over the intranet or even the Internet.

Getting ready

We will be using Ubuntu Server 16.04, but you can choose to have any version of Ubuntu.

Additionally, we will need the Samba server. It will be used as shared network storage.

As always, access to a root account or an account with sudo privileges will be required.

How to do it…

Ampache is a web application developed in PHP. We will start the installation with the LAMP stack. This recipe covers installation with the Apache web server, but you can choose any other web server:

Install the LAMP stack if it's not already installed:

$ sudo apt-get update

$ sudo apt-get install apache2 mysql-server-5.5 php7 \

php7-mysql php7-curl libapache2-mod-php7

Next, download the latest Ampache server source code. Ampache is a PHP application:

$ wget https://github.com/ampache/ampache/archive/3.8.0.tar.gz

Extract achieve contents under a web root directory

$ tar -xf 3.8.0.tar.gz -C /var/www

$ mv /var/www/ampache-3.8.0 /var/www/ampache

We also need to create some configuration files. You can use the default configuration that ships with the Ampache setup and rename the existing files:

$ cd /var/www/ampache

$ mv rest/.htaccess.dist rest/.htaccess

$ mv play/.htaccess.dist play/.htaccess

$ mv channel/.htaccess.dist channel/.htaccess

The Ampache web setup will save the configuration under the config directory. It will need write access to that directory:

$ chmod 777 -R config

Next, we need to configure the Apache web server, enable mod_rewrite, and set a virtual host pointing to the Ampache directory.

Enable mod_rewrite with the following command:

$ sudo a2enmod rewrite

Create a new virtual host configuration:

$ cd /etc/apache2/sites-available/

$ sudo vi ampache.conf

Add the following lines to ampache.conf:

DocumentRoot /var/www/ampache

DirectoryIndex index.php

AllowOverride All

Order allow,deny

Allow from all

ErrorLog ${APACHE_LOG_DIR}/error.log

LogLevel warn

CustomLog ${APACHE_LOG_DIR}/access.log combined

Now, disable any default configuration that is using port 80, or alternatively you can use a port other than 80 for Ampache installation.

Reload the Apache server for the changes to take effect:

$ sudo service apache2 reload

Here, we have installed and configured the base setup. Now, we can move on to configuration through a web-based installer. You can access the web installer at the domain name or IP address of your server. The installer should greet you with a big Ampache logo and a language selection box; something similar to the following:

Select the language of your choice and click the Start configuration button.

On the next page, Ampache will check all the requirements and show you a list of settings that need to be fixed. These are mostly the configuration changes and file permissions.

Most of these requirements should already be marked with a green OK button. You need to fix things that are marked in red. The requirements screen will look as follows:

Click the Continue button when you are done reviewing all the requirements.

On the next page, you need to configure the MySQL settings. Fill in the necessary details and select Create Database User to create a new Ampache user under the MySQL server:

Click Insert Database to configure database settings.

The next screen will confirm the database settings and write the configuration changes to a file under the config directory. You can choose to change the installation type and enable transcoding configuration from this screen. Once done, click the Continue button to write the configuration file. If you see any errors, scroll to the bottom of the page and click the write button to write config changes.

Finally, the web setup will ask for admin account credentials for the Ampache server. The Create Admin Account form will be shown with Username and Password fields, as follows. Set the admin account username and password and click the Create Account button:

Once the account is created, the Ampache installation script will redirect you to the web player screen. If it shows a login screen, use the admin account credentials created in the last step to log in. The landing page of the web player will be rendered as follows:

You have completed the Ampache setup process. Now you need to upload content and enjoy your own streaming server. We will learn to create a catalog and upload content in the next recipe.

How it works…

Ampache is a web application written in PHP. We have downloaded the latest Ampache code and set it to work with our web server using Virtual Host configuration. Ampache provides sample htaccess files that set required redirection rules. We have enabled respective rules by renaming the sample files. If you are using a web server other than Apache, make sure you check the Ampache documentation for your web server. It supports Nginx and lighttpd as web servers.

Ampache has made it easy to cross-check all requirements and configure your database connection using the web installer. The installer checks for the required PHP settings and extensions and returns a simple page with things that need to fixed. Next, we can configure database settings and push schema directly from the web installer.

Once everything is configured, the web installer returns the login page, from where you can access the Ampache server.

There's more…

The Ampache community have created a Docker image for the Ampache server. If you have a Docker system set up, you can quickly get started with Ampache with its Docker image.

You can get the Dockerfile at https://github.com/ampache/ampache-docker .

Ampache is also available in the Ubuntu package repository and can be installed with the following single command:

$ sudo apt-get install ampache mysql-server-5.5

The currently available version of Ampache is 3.6. If you don't care about the latest and greatest updates, you can use the Ubuntu repository for quick and easy installation.

See also

Ampache installation guide: https://github.com/ampache/ampache/wiki/Installation

Sometimes, we have a large number of servers to install. In this case, the manual installation will take a lot of time to perform a repetitive task. To solve this problem, there is the automation installation, or what we call the network boot.

For this, we need a machine equipped with a DHCP server and a TFTP server that

will provide us the services and configuration files that we need for the system

to be installed.

The PXE process

The client computer (our future server) will boot its network interface in the PXE (Preboot Execution Environment) mode. Then, the DHCP present on the network will send it the pxelinux.0file; this will be explained later. Thus, the client computer accesses the pxelinux.cfg configuration file via TFTP, which contains the necessary information required to launch the installation process.

The PXE installation procedure

Let's start with the server installation:

1. First of all, install the DHCP server by using the sudo apt-get install isc-dhcp-server -y command, and then configure it by using the /etc/default/isc-dhcp-server file to use the network that you want for listening (such as eth0).

In the /etc/dhcp/dhcpd.conffile, you should configure some parameters

such as the subnet and the address range. Then, restart it by using the following command:

sudo service isc-dhcp-server restart

2. After this, install the following packages that are necessary if you wish to set up the PXE environment:

sudo apt-get install apache2 tftpd-hpa inetutils-inetd

Now, it is time to configure the TFTP service. To do this, add the following two lines to the /etc/default/tftpd-hpa file:

RUN_DAEMON="yes"

OPTIONS="-l -s /var/lib/tftpboot"

Also, add the following line at the end of the /etc/inetd.conf file:

tftp dgram udp wait root /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /var/lib/tftpboot

Then, reboot the service using the sudo /etc/init.d/tftpd-hpa restart command.

3. Now, we need to copy the installation files to the PXE server. In our example, I used the ISO image that I have in my home directory. First of all, mount it by using the following command:

sudo mount loop /home/abdelmonam/ubuntu-15.04-server-amd64.iso /mnt

Then, copy the required files to the server by using the following commands:

cd /mnt sudo cp -fr install/netboot/* /var/lib/tftpboot/ sudo mkdir /var/www/Ubuntu sudo cp -fr /mnt/* /var/www/ubuntu/

After doing this, modify the /var/lib/tftpboot/pxelinux.cfg/default PXE config file by adding the following lines at the end:

label linux kernel ubuntu-installer/amd64/linux append ks=http://192.168.1.1/ks.cfg vga=normal

initrd=ubuntu-installer/amd64/initrd.gz ramdisk_size=16432 root=/dev/rd/0 rw -

Be careful when adding the IP address.

4. The last step required to set up the PXE server is to add the following lines at

the end of the /etc/dhcp/dhcpd.conf file:

allow booting; allow bootp; option option-128 code 128 = string; option option-129 code 129 = text; next-server 192.168.1.1; filename "pxelinux.0";

Then, reboot the DHCP server by using the following command:

sudo service isc-dhcp-server restart

Let's move on to the client configuration. In our case, I used a virtualboxinstance to test this kind of installation:

1. Create the virtual machine with the characteristics that you want via the virtualbox manager.
2. Then, go to the Settings of the machine and select the System tab. In the Boot Order part, deselect all options and select Network, as shown in the following screenshot:
3. Select the Network tab and configure the network adaptor to act as a bridge.
4. Finally, start your VM. You will see the following interface:

Enjoy watching the server installation if you were doing it locally from a CD.

The PXE installation can be used to install a lot of machines in parallel as

well as to install Ubuntu Server on machines without a CD-ROM driver. The installation process will be entirely automated if you combine the PXE method with a kickstartand/or preseedfile. A good starting point for working with kickstartis https://help.ubuntu.com/ community/KickstartCompatibility .

Additional resources

Since this book consists of the essentials for the Ubuntu Server, we can't cover topics in depth. Therefore, here are some useful links that will help you go as far as you want in this subject:

• For details regarding the installation of the Ubuntu desktop from a PXE server, visit https://help.ubuntu.com/community/Installation/ LocalNet . You can use the same procedure to install Ubuntu Server.
• To learn how to prepare a PXE server that allows you to boot multiple distributions (Ubuntu, Debian, Fedora, CentOS, and openSUSE), visit https://help.ubuntu.com/community/PXEInstallMultiDistro.
• For those of you searching for how to set up a MAAS server that will deploy a Ubuntu system through netboot, a very good guide is available at http://maas.ubuntu.com/docs/install.html .

Summary

In this article, we had a look at how to install Ubuntu Server in different modes—manually and automated—with the help of a simple or an advanced installation.

Now, we can start managing our server, which is the subject that we will cover in the next article.

In this article, we will cover the following recipes:

• Installing LXD, the Linux container daemon
• Deploying your first container with LXD
• Managing LXD containers
• Managing LXD containers – advanced options
• Setting resource limits on LXD containers
• Networking with LXD
• Installing Docker
• Starting and managing Docker containers
• Creating images with a Dockerfile
• Understanding Docker volumes
• Deploying WordPress using a Docker network
• Monitoring Docker containers
• Securing Docker containers

In this recipe, we will learn how to install Samba as our network storage server. Samba is a collection of open source applications that implement Server Message Block (SMB) and Common Internet File System (CIFS) protocols on Unix systems. This allows Samba to be accessible across different types of network system. Samba provides various other functionalities, such as a domain controller for the networks of Windows systems. In this recipe, we will focus on using Samba as a storage server.

Getting ready

You will need access to a root account or an account with sudo privileges

If your server is using any firewall system, make sure to open the necessary network ports. Samba runs on TCP 139 and 445 and UDP ports 137 and 138. Check article 2, Networking, for more details on firewall configuration.

How to do it…

Follow these steps to install the Samba server:

Install the Samba server with the following command:

$ sudo apt-get update

$ sudo apt-get install samba -y

After installation is complete, you can check the Samba version with the following command:

$ smbd --version

Next, we need to configure Samba to enable sharing on the network. First, create a backup of the original configuration file:

$ sudo cp /etc/samba/smb.conf /etc/samba/smb.conf.orignl

Next, open smb.conf and replace its contents with the following:

[global]

workgroup = WORKGROUP

server string = Samba Server

netbios name = ubuntu

security = user

map to guest = bad user

dns proxy = no

[Public]

path = /var/samba/shares/public

browsable =yes

writable = yes

guest ok = yes

read only = no

create mask = 644

Next, we need to create a shared directory:

$ sudo mkdir -p /var/samba/shares/public

Change the directory permissions to make it world writable:

$ sudo chmod 777 /var/samba/shares/public

Restart the Samba service for the changes to take effect:

$ sudo service smbd restart

Now you can access this Samba share on the Windows client. Open Windows Explorer and in the address bar, type in \\ubuntu or \\your-server-ip. You should see the shared directory, Public, as follows:

How it works…

Samba is quite an old technology, especially in the age of Cloud storage such as Dropbox and Amazon S3. However, when it comes to private networking, Samba offers a hassle-free setup and is always available for free. All you need is a small server with some free storage space. The release of Samba 4 has added Active Directory (AD) support. Now it's possible to set up Windows AD on Linux servers. Support for AD comes with a wide range of other features, including DNS for name resolution, centralized storage, and authentication with LDAP and Kerberos.

As you can see in the preceding example, setting up Samba is quick and easy, and you can easily get started with network storage within minutes. We can install the Samba server with a single command, as Samba packages are available in the Ubuntu default package repository. After installation, we have created a new quick and dirty configuration file which defines a few parameters, such as the server name (netbios name) and a share definition. We have created a publicly-shared directory where everyone can read and write the contents.

Once you are done with installation and initial testing, make sure that you remove public sharing and enable authenticated access to your Samba shares. You don't want the server to fill up with data from unknown people. In the next recipes, we will take a closer look at user management and access control for Samba shares.

There's more…

To secure your Samba installation and limit access to your local network or subnet, you can use the following configuration parameters:

[globals]

hosts deny = ALL

hosts allow = xxx.xxx.xxx.xxx/yy 127.

interfaces = eth0 lo

bind interfaces only = Yes

This configuration limits Samba to listen only on listed interfaces. In this case, its eth0, the Ethernet network, and lo, localhost. Connection requests from all other hosts are denied.

Tools for personal file sharing

If you need a simple file sharing tool for your personal use and do not want to set up and configure Samba, then you can try using a tool named OwnCloud. It is very similar to Dropbox and is open source. It gives you web access to all your files and documents. Plus, you get desktop and mobile client apps to sync all files to a remote server.

Another good tool is BitTorrent Sync. Again, this is a file synchronization tool, but this time it is peer-to-peer file synchronization. If you really care about the privacy and security of data, then this tool is made for you. All files are synchronized between two or more systems (say, your desktop and laptop) without the use of any centralized server.

See also

Ubuntu server guide for Samba at https://help.ubuntu.com/lts/serverguide/samba-fileserver.html

Modern CPUs generally do not become bottlenecks for performance. The processing power is still far ahead of the data transfer speeds of I/O devices and networks. Generally, the CPU spends a big part of processing time waiting for synchronous IO to fetch data from the disk or from a network device. Tracking exact CPU usage is quite a confusing task. Most of the time, you will find higher CPU use, but in reality, the CPU is waiting for data to become available.

In this recipe, we will focus on tracking CPU performance. We will look at some common tools used to get CPU usage details.

Getting ready

You may need sudo privileges to execute some commands.

How to do it…

Let's start with the most commonly used monitoring command that is top command. The top command shows a summarized view of various resource utilization metrics. This includes CPU usage, memory and swap utilization, running processes, and their respective resource consumption, and so on. All metrics are updated at a predefined interval of three seconds.

Follow these steps to monitor the CPU:

To start top, simply type in top in your command prompt and press Enter:

$ top

As you can see in the preceding screenshot, a single Python process is using 80% of CPU time. The CPU is still underutilized, with 58% time in idle processes:

Optionally, you can use the htop command. This is the same process monitor as top, but a little easier to use, and it provides text graphs for CPU and memory utilization. You will need to install htop separately:

$ sudo apt-get install htop # one time command

$ htop

While top is used to get an overview of all running processes, the command pidstat can be used to monitor CPU utilization by an individual process or program. Use the following command to monitor CPU consumed by MySQL (or any other task name):

$ pidstat -C mysql

With pidstat, you can also query statistics for a specific process by its process ID or PID, as follows:

$ pidstat -p 1134

The other useful command is vmstat. This is primarily used to get details on virtual memory usages but also includes some CPU metrics similar to the top command:

Another command for getting processor statistics is mpstat. This returns the same statistics as top or vmstat but is limited to CPU statistics. Mpstat is not a part of the default Ubuntu installation; you need to install the sysstat package to use the mpstat command:

$ sudo apt-get install sysstat -y

By default, mpstat returns combined averaged stats for all CPUs. Flag -P can be used to get details of specific CPUs. The following command will display statistics for processor one (0) and processor two (1), and update at an interval of 3 seconds:

$ mpstat -P 0,1 3

One more command, sar (System Activity Reporter), gives details of system performance.

The following command will extract the CPU metrics recorded by sar. Flag -u will limit details to CPU only and -P will display data for all available CPUs separately. By default, the sar command will limit the output to CPU details only:

$ sar -u -p ALL

To get current CPU utilization using sar, specify the interval, and optionally, counter values. The following command will output 5 records at an interval of 2 seconds:

$ sar -u 2 5

All this data can be stored in a file specified by the (-o) flag. The following command will create a file named sarReport in your current directory, with details of CPU utilization:

$ sar -u -o sarReport 3 5

Other options include flag –u, to limit the counter to CPU, and flag A, to get system-wide counters that include network, disk, interrupts, and many more. Check sar manual (man sar) to get specific flags for your desired counters.

How it works…

This recipe covers some well known CPU monitoring tools, starting with the very commonly used command, top, to the background metric logging tool SAR.

In the preceding example, we used top to get a quick summarized view of the current state of the system. By default, top shows the average CPU usage. It is listed in the third row of top output. If you have more than one CPU, their usage is combined and displayed in one single column. You can press 1 when top is running to get details of all available CPUs. This should expand the CPU row to list all CPUs. The following screenshot shows two CPUs available on my virtual machine:

The CPU row shows various different categories of CPU utilization, and the following is a list of their brief descriptions:

us: Time spent in running user space processes. This reflects the CPU consumption by your application.

sy: Time taken by system processes. A higher number here can indicate too many processes, and the CPU is spending more time process scheduling.

ni: Time spent with user space processes that are assigned with execution priority (nice value).

id: Indicates the time spent in idle mode, where the CPU is doing nothing.

wa: Waiting for IO. A higher value here means your CPU is spending too much time handling IO operations. Try improving IO performance or reducing IO at application level.

hi/si: Time spent in hardware interrupts or software interrupts.

st: Stolen CPU cycles. The hypervisor assigned these CPU cycles to another virtual machine. If you see a higher number in this field, try reducing the number of virtual machines from the host. If you are using a cloud service, try to get a new server, or change your service provider.

The second metric shown is the process level CPU utilization. This is listed in a tabular format under the column head, %CPU. This is the percentage of CPU utilization by each process. By default, the top output is automatically sorted in descending order of CPU utilization. Processes that are using higher CPU get listed at top. Another column, named TIME+, displays total CPU time used by each process. Check the processes section on the screen, which should be similar to the following screenshot:

If you have noticed the processes listed by top you should see that top itself is listed in the process list. Top is considered as a separate running process and also consumes CPU cycles.

With top, you can get a list of processes or tasks that are consuming most of the CPU time. To get more details of these tasks, you can use the command, pidstat. By default, pidstat shows CPU statistics. It can be used with a process name or process ID (pid). With pidstat, you can also query memory usages, IO statistics, child processes, and various other process related details. Check the manual page for pidstat using the command man pidstat.

Both commands, top as well as pidstat, give a summarized view of CPU utilization. Top output is refreshed at a specific interval and you cannot extract utilization details over a specific time period. Here comes the other handy command that is vmstat. When run without any parameters, vmstat outputs a single line with memory and CPU utilization, but you can ask vmstat to run infinitely and update the latest metrics at specific intervals using the delay parameter. All the output lines are preserved and can be used to compare the system stats for a given period. The following command will render updated metrics every 5 seconds:

$ vmstat 5

Optionally, specify the count after delay parameter to close vmstat after specific repetitions. The following command will update the stats 5 times at 1 second intervals and then exit:

$ vmstat 1 5

The details provided by vmstat are quite useful for real-time monitoring. The tool sar helps you to store all this data in log files and then extract specific details whenever needed. Sar collects data from various internal counters maintained by the Linux kernel. It collects data over a period of time which can be extracted when required. Using sar without any parameters will show you the data extracted from the previously saved file. The data is collected in a binary format and is located at the /var/log/sysstat directory. You may need to enable data collection in the /etc/default/sysstat file. When the stats collection is enabled, sar automatically collects data every 10 minutes. Sar is again available from the package sysstat. Along with the sar package, sysstat combines two utilities: command sa1 to record daily system activity data in a binary format, and command sa2 to extract that data to a human readable format. All data collected by sar can be extracted in a human readable format using the sa2 command. Check the manual pages for both commands to get more details.

There's more…

Similar to sar, one more well-known tool is collectd. It gathers and stores system statistics, which can later be used to plot graphs.

See also

Get information on your system CPU with the following command:

$ less /proc/cpuinfo

Details on /proc file system: http://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/proc.html

HTTP is a non-secure protocol commonly used to communicate over the Web. The traffic is transferred in plain text form and can be captured and interpreted by a third-party attacker. Transport Layer Security and Secure Socket Layer protocols (TLS/SSL) can be used to secure the traffic between client and server. These protocols encapsulate normal traffic in an encrypted and secure wrapper. It also validates the identity of the client and server with SSL keys, certificates, and certification authorities.

When HTTP is combined with TLS or SSL, it is abbreviated as HTTPS or HTTP secure. Port 443 is used as a standard port for secured HTTP communication. Nearly all leading web servers provide inbuilt support for enabling HTTPS. Apache has a module called mod_ssl that enables the use of HTTPS.

To set up your servers with SSL/TLS encrypted traffic, you will need an SSL certificate and a key pair that can be used to encrypt traffic. Generally, the certificate and keys are obtained from a trusted signing authority. They charge you some fees to verify your ownership of the web property and allocate the required signed certificates. You can also generate self-signed certificates for internal use. Few certification authorities provide a free SSL certificate. Recently, Mozilla has started a free and automated certificate authority named Let's Encrypt. At the time of writing, the service is in public beta and has started allocating certificates. Let's Encrypt offers a client that can be used to obtain certificates and set up automated renewal. You can also find various unofficial clients for Apache and Nginx servers.

In this recipe, we will learn how to create our own self-signed certificate and set up the Apache server to serve contents over a secure channel.

Getting ready

You will need access to a root account or an account with sudo privileges. I assume that you have the Apache server preinstalled. You will also need OpenSSL installed.

Make sure your firewall, if any, allows traffic on port 443. Check article 2, Networking, Securing network with uncomplicated firewall recipe for more details on Uncomplicated Firewall.

How to do it…

Follow these steps to secure web traffic with HTTPS:

First, we will start by creating a self-signed SSL certificate. Create a directory under /etc/apache2 to hold the certificate and key:

$ sudo mkdir /etc/apache2/ssl

Change to the new directory and enter the following command to create a certificate and SSL key:

$ cd /etc/apache2/ssl

$ sudo openssl req -x509 -nodes -days 365 \

-newkey rsa:2048 -keyout ssl.key -out ssl.crt

This will prompt you to enter some information about your company and website. Enter the respective details and press Enter for each prompt:

After you are done with it, you can check the generated certificate and key:

$ ls -l

Next, we need to configure Apache to use SSL. We will enable SSL for the previously created virtual host.

Open the Virtual Host configuration file, example.com.conf. After removing comments, it should look similar to the following:

Now, copy the entire ... tag and paste it at the end of the file.

Under the newly copied contents, change the port from 80 to 443.

Add the following lines below the DocumentRoot line. This will enable SSL and specify the path to the certificate and key:

SSLEngine on

SSLCertificateFile /etc/apache2/ssl/ssl.crt

SSLCertificateKeyFile /etc/apache2/ssl/ssl.key

The final file should look something like this:

Save the changes, exit example.com.conf, and enable the mod_ssl module on the Apache server:

$ sudo a2enmod ssl

Next, enable the Virtual Host example.com. If it's already enabled, it will return a message saying site example.com already enabled:

$ sudo a2ensite example.com.conf

Reload the Apache server for the changes to take effect:

$ sudo service apache2 reload

Now, open your browser on the client system and point it to your domain name or IP address with HTTPS at the start:

https://example.com

Your browser may return an error saying Invalid Certification Authority. This is fine as we are using a self-signed certificate. Click Advanced and then click Proceed to example.com to open a specified page:

Once the page is loaded completely, find the padlock icon in the upper right corner of the browser and click on it. The second section with the green lock icon will display the encryption status. Now your communication with the server is encrypted and secure:

How it works…

We have created a self-signed certificate to secure an HTTP communication. The key will be used to encrypt all communication with clients. Another thing to note is that we have defined a separate Virtual Host entry on port 443. This Virtual Host will be used for all requests that are received over port 443. At the same time, we have allowed non-secured HTTP communication for the same Virtual Host. To disable non-secure communication on port 80, you can simply comment out the original Virtual Host configuration. Alternatively, you can separate both configurations into two files and enable or disable with the a2ensite and a2dissite commands.

Some of the parameters used for generating a key and certificate are as follows:

- nodes specifies that we do not want to use a passphrase for a key.

- days this specifies the number of days the certificate is valid for. Our certificate is valid for 365 days, that is, a year.

- newkey rsa:2048 this option is used to generate a certificate along with a private key. rsa:2048 specifies the 2048 bit long RSA private key.

I have modified the existing Virtual Host entry to demonstrate the minimal configuration required to enable secure HTTP communication. You can always use the default secure Virtual Host configuration available under sites-available/default-ssl.conf. This file provides some additional parameters with respective comments.

The certificate created in this recipe will not be trusted over the Internet but can be used for securing local or internal communication. For production use, it is advisable to get a certificate signed from an external, well known certification authority. This will avoid the initial errors in browsers.

There's more…

To get a signed certificate from an external certification authority, you will need a CSR document.

The following are the steps to generate a CSR:

Generate a key for the CSR:

$ openssl genrsa -des3 -out server.key 2048

You will be asked to enter a passphrase for the key and then verify it. They will be generated with name server.key.

Now, remove the passphrase from the key. We don't want to enter a passphrase each time a key is used:

$ openssl rsa -in server.key -out server.key.insecure

$ mv server.key server.key.secure

$ mv server.key.insecure server.key

Next, create the CSR with the following command:

$ openssl req -new -key server.key -out server.csr

A CSR file is created with the name server.csr, and now you can submit this CSR for signing purposes.

See also

Refer to the Installing and configuring the Apache web server recipe for the installation and configuration of the Apache web server.

Check out the certificates and security in the Ubuntu server guide at https://help.ubuntu.com/lts/serverguide/certificates-and-security.html

How to set up client verification at http://askubuntu.com/questions/511149/how-to-setup-ssl-https-for-your-site-on-ubuntu-linux-two-way-ssl

Apache documentation on SSL configuration at http://httpd.apache.org/docs/2.4/ssl/ssl_howto.html

Free SSL certificate with Mozilla Let's Encrypt at https://letsencrypt.org/getting-started/

Easily generate SSL configuration for your web server at Mozilla SSL Configuration Generator at https://mozilla.github.io/server-side-tls/ssl-config-generator/

Now that we have the Git binaries installed, let's take a step forward and create our first local Git repository.

Getting ready

Make sure that you have installed Git.

How to do it…

We will take a common path by starting a new pet project, where we will simply create a new local directory, add some files to it, and then realize, Ohh I am gonna need a version control system:

So, yes, quickly create your new project:

$ mkdir mynewproject

$ touch mynewproject /index.html

$ touch mynewproject /main.js

$ touch mynewproject/main.css

Add some sample content to these files by editing them:

Now you need to create a Git repository for this project. Sure, Git covered you with the git init command.

Make sure you are in the project directory and then initialize a new repository, as follows:

$ cd mynewproject

$ git init

This will initialize a new empty repository under the project directory. A new hidden directory gets created with the name .git. This directory will contain all the metadata of your Git repository and all revisions of every single file tracked by Git.

How it works…

Here, we have used the git init command to initialize a new repository on our local system. The files created before initializing a repo are optional; you can always skip that step and directly use git init to create a new local repository. Later, when you need to push (synchronize) this repo with a remote hosted repository, you can simply use the git remote add command. We will see examples of git remote add in the next recipes.

With the git init command, you can also create a bare repository by using the --bare flag. The difference between a normal repository and a bare repository is that a bare repository does not have a working copy. You cannot use a bare repository directly to edit and commit files. Unlike a normal repository, where revision history, tags, and head information is stored in a separate .git directory, a bare repo stores all this data in the same directory. It is meant to be a central shared repository where multiple people can commit their changes. You need to clone these types of repositories to access and edit files. The changes can be pushed using the git push command from the cloned copy.

There's more…

You can also use git clone to clone existing repositories. The repository can be local or remote. The clone command will replicate the contents of a parent repository, including revision history and other details. We will see more details of git clone in the next recipes.

See also

You can read more by following these links:

Git init: https://git-scm.com/docs/git-in it

Git clone: https://git-scm.com /docs/git-clone