Skip to main content

Resources

Introduction for Streaming with Ampache in Ubuntu

This article covers the installation and configuration of the open source audio and video streaming service, Ampache. It is a web-based streaming application that allows you to upload your own audio/video contents and access them across multiple Internet-enabled devices. You can easily set up your home media server using Ampache and your old personal computer running Ubuntu. We will focus on installing Ampache on the Ubuntu server, but you can install Ampache on any Linux distribution of your choice.

Ubuntu advanced installation

Much of the Ubuntu Server installation process fulfils a majority of user needs, and it is used in the same manner by most of the users (newbie or advanced), but the part that requires much more attention and which is customized by many advanced system administrators depending on their needs is the partitioning step. In this section, we will have a look at how to perform an advanced partitioning for

a specific hard disk schema.

Using RAID

The RAID technology (an acronym for Redundant Array of Inexpensive Disks) refers to the techniques used to distribute data across multiple hard drives (creating a storage unit from several hard disks). The unit thus created has an improved fault tolerance (high availability) or a greater performance (the capacity/write speed). The distribution of data on several hard drives allows you to increase the safety and reliability of the associated services.

For more information about this technology, you can take a tour at http://www.tldp.org/HOWTO/Software-RAID-HOWTO.html . In this section, we will see how to install Ubuntu Server using two RAID1 partitions on two different hard disks, one for the root and the other for Swap:

  1. In the Partition disk step, select manual. Then, select one by one all the disks that you want and which should be a part of the RAID partitions (in our case, only two disks). For each disk, create a new empty partition table on it.
  2. Select the free space on the first drive. Then, choose Create a new partition. Set the size that you want for Swap; it should be twice the RAM size and up to 2 GB. Then, choose Primary and select Beginning. The physical volume for RAID option should be chosen in the Use as: line. Finally, select Done setting up partition.
  3. With the remaining free space, perform the same steps for the root partition. The only exception is that after setting the size, choose Continue and then select Primary. Also, select the Bootable flag: line to change the value to on.
  4. Return to the Partition Disks step and select Configure Software RAID at the beginning of the page. Then, choose yes to write the modifications to disk.
  5. In the Create MD device setup and for this example, choose RAID1, but if

you are applying another architecture, choose the one that fits your needs.

  1. Now, enter the number of hard drives that you have chosen for the array (in our case, we have two active devices). Then, select Continue.
  2. Next, enter the number of spare devices (choose the default setting for RAID1, which is 0) and then select Continue.
  3. At this step, choose the partitions that you wish to use. For the Swap partition, choose sda1 and sdb1. Select Continue to go to the next step.
  4. Repeat steps 3 to 7 to prepare the root partition, and at the end, choose sda2 and sdb2. Finally, select Finish.

We should now have a list of RAID devices and hard drives. The next remaining steps are required to format and set the mount point for the RAID devices. Deal with the RAID devices as a local hard disk; format and mount accordingly:

  1. Getting back to the initial partitioning page again, we will see our newly created RAID devices #0 and #1 partitions.
  2. Under the RAID1 device #0 partition, select #1 and press the Enter key.
  3. In the Use as: file, select swap area and then apply this partition by using the Done setting up partition entry.
  4. Under the RAID1 device #1 partition, select #1 and press the Enter key.
  5. In the Use as: file, select Ext4 journaling file system.
  6. In the Mount point field, choose / (the root file system). You can change the other options as you wish. Then, apply that partition by using the Done setting up partition entry.
  7. Finally, select Finish partitioning and write changes to disk.

The installation process will then continue normally.

Using LVM

LVM (Logical Volume Manager) allows the creation and the management of logical volume on Linux. The use of logical volumes somehow replaces the disks'

partitioning. This is a much more flexible system as it allows you, for instance, to reduce the size of a file system to expand another one regardless of their location

on disks.

For more information about this technology, you can take a tour at http://tldp.org/HOWTO/LVM-HOWTO/ . In this section, we will have a look at the partitioning step of Ubuntu Server installation process with the /srv partition mounted on a LVM volume. This section

will be divided into two parts—the first part is during the Ubuntu installation within

only one PV (physical volume), which will be a part of the VG (volume group), and the second part will be after the Ubuntu installation. While performing this process, we will demonstrate how to add a second PV to the VG:

  1. In the Partition disks screen, choose Manual.
    1. Select the hard disk. Then, choose Yes to confirm the Create a new empty partition table on this device order.
    2. [ 14 ]
  2. Next, create standard swap and / partitions with the file system that

you want.

  1. For our /srv LVM partition, create a new logical partition and change the Use as field to physical volume for LVM. Then, select Done setting up the partition.
  2. Now, choose Configure the Logical Volume Manager and select Yes.
  3. On the next screen and for LVM configuration action, select Create volume group. Name the VG (for example, vg01). Then, select the partition prepared for LVM and choose Continue.
  4. Return to the LVM configuration action page and select Create logical volume. Choose the newly created VG and name the new LV. Then, choose a size and at the end, select Finish. You will be taken to the main Partition Disks page again.
  5. Now, let's add a file system to our new LVM. To do this, select the partition under the LVM that you created and set up the file system that you want and

choose /srv as the mount point. Next, select Done setting up the partition.

9. Finally, apply the Finish partitioning and write changes to disk order.

Then, confirm the changes and continue your installation.

Now, we will continue with the second part that started after the Ubuntu Server installation and which covers adding a second hard disk, creating a PV, adding it to

the existing VG, extending LV with the srv command, and finally extending the file

system. In this part, we will use a second hard disk named /dev/sdb, and we will

use it entirely as a PV. You can configure and use as much of PV as you want:

1. First of all, create the PV using the following command:

sudo pvcreate /dev/sdb

2. Next, extend the VG (called vg01 in our example), as follows:

sudo vgextend vg01 /dev/sdb

  1. Use the vgdisplay command to find out the size that you can allocate (for free physical extents, see the Free PE / size line). We will assume in our example a free size of 511 PE (which is equivalent to 2 GB with a size of 4 MB PE), and we will use the entire available free space.
  2. We can now use the PE to extend the LV via the following command (there are other methods, but they are beyond the scope of this book):

sudo lvextend /dev/vg01/srv -l +511

  1. We used the -l option here to extend the LV using PE. If you want to extend using MB, GB, and TB, you can use the -l option.
  2. Before expanding an ext3 or ext4 file system, it is recommended that you unmount it before hand (as best practice). In case you want to reduce an LV, it will be mandatory to unmount it. The following commands unmount the partition and then check it:

sudo umount /srv sudo e2fsck -f /dev/vg01/srv

7. Finally, you can resize the file system by using the following command:

sudo resize2fs /dev/vg01/srv

8. Now, you can mount the partition and check its new size using the following command:

mount /dev/vg01/srv /srv && df -h /srv

Managing services with Juju on Ubuntu

In the previous recipe, we learned how to install the Juju service orchestration framework. Now, we will look at how to use Juju to deploy and manage a service.

Getting ready

Make sure you have installed and bootstrapped Juju.

How to do it…

We will deploy a sample WordPress installation with a load balancer. The MySQL service will be used as the database for WordPress. Both services are available in the Juju Charm store.

Follow these steps to manage services with Juju:

Let's start by deploying the WordPress service with juju deploy. This should give you the following output:

$ juju deploy wordpress

Added charm "cs:trusty/wordpress-4" to the model.

Deploying charm "cs:trusty/wordpress-4" with the charm series "trusty".

Now, deploy a MySQL service to store WordPress contents:

$ juju deploy mysql

Added charm "cs:trusty/mysql-38" to the model.

Deploying charm "cs:trusty/mysql-38" with the charm series "trusty".

Now, you can use juju status to confirm your deployed services. It should show you the deployed services, their relations, and respective machine statuses, as follows:

$ juju status

Now that both services have been deployed, we need to connect them together so that wordpress can use the database service. Juju calls this a relation, and it can be created as follows:

$ juju add-relation mysql wordpress

Finally, we need to expose our wordpress service so that it can be accessed outside our local network. By default, all charms start as unexposed and are accessible only on a local network:

$ juju expose wordpress

You can get the IP address or DNS name of the wordpress instance with the juju status command from the Machines section. Note that in a local LXD environment, you may need a forwarded port to access WordPress.

How it works…

In this example, we deployed two separate services using Juju. Juju will create two separate machines for each of them and deploy the service as per the instructions in the respective charms. These two services need to be connected with each other so that wordpress knows the existence of the MySQL database. Juju calls these connections relations. Each charm contains a set of hooks that are triggered on given events. When we create a relation between WordPress and MySQL, both services are informed about it with the database-relation-changed hook. At this point, both services can exchange the necessary details, such as MySQL ports and login credentials. The WordPress charm will set up a MySQL connection and initialize a database.

Once both services are ready, we can expose them to be accessed on a public network. Here, we do not need MySQL to be accessible by WordPress users, so we have only exposed the wordpress service. WordPress can access MySQL internally, with the help of a relation.

You can use the Juju GUI to visualize your model and add or remove charms and their relations. At this point, if you open a GUI, you should see your charms plotted on the graph and connected with each other through a small line, indicating a relation. The GUI also provides an option to set constraints on a charm and configure charm settings, if any.

Note that both charms internally contain scaling options. WordPress is installed behind an Nginx reverse proxy and can be scaled with extra units as and when required. You can add new units to the service with a single command, as follows:

$ juju add-unit mysql -n 1

There's more…

When you no longer need these services, the entire model can be destroyed with the juju destroy-model command. You can also selectively destroy particular services with the remove-service command and remove relations with remove-relations. Check out the Juju manual page for tons of commands that are not listed in the Juju help menu.

See also

How to create your own charm: https://jujucharms.com/docs/stable/authors-charm-writing

More about hooks: https://jujucharms.com/docs/stable/authors- hook-environment

Introduction of Network Storage in Ubuntu

Often we need to store a lot of data and local systems don't have enough space. Sometimes, we need to quickly share this data across multiple systems and users. Also, when you have a big network, chances are you have Linux systems as well as Windows or Mac. Centralized networked storage can help to solve these storage and sharing problems. Linux provides various options, such as Samba and NFS, to host a centralized storage server and share data across multiple computers.

In this article, we will learn how to set up a centralized storage system. We will set up the Samba server and NFS server. We will learn how to enable synchronization with Rsync and set Windows clients to access storage servers.

Installing the Zimbra mail server on Ubuntu

Until now, we have installed Postfix, Dovecot, spam-assassin, and other tools separately. In this recipe, we will learn how to install the Zimbra collaboration server, which covers all tools in a single package. The Zimbra server contains Postfix, MySQL, OpenLDAP, ClamAV, and Spam-Assassin, Calendar, and various other features. Zimbra provides a paid option as well as an open source version. We will be installing an open source version of the Zimbra server in single server mode.

Getting ready

As always, you will need access to a root account or an account with sudo privileges.

For Zimbra to work properly, you will need the following minimum configuration for your server:

At least 1.5 GHz of CPU 2 GHz recommended

Minimum 8 GB of memory

Minimum 10 GB of storage 20 GB recommended

You will need to set proper DNS and MX records for your domain.

You will also need various ports, as follows:

Postfix/LMTP 25, 7025

HTTP 80, 443

POP3 110, 995

IMAP 143, 993

LDAP 389

How to do it…

Follow these steps to install Zimbra collaboration server:

Install the dependency packages before starting with the Zimbra installation:

$ sudo apt-get update

$ sudo apt-get install libperl5.18 libaio1 unzip pax sysstat sqlite3 libgmp10

Download and extract the Zimbra open source package using the following command:

$ wget https://files.zimbra.com/downloads/8.6.0_GA/zcs- 8.6.0_GA_1153.UBUNTU14_64.20141215151116.tgz

$ tar -zxvf zcs-8.6.0_GA_1153.UBUNTU14_64.20141215151116.tgz

$ cd zcs-8.6.0_GA_1153.UBUNTU14_64.20141215151116

Make sure you have set the proper hostname and hosts entries in respective files:

$ cat /etc/hosts

127.0.0.1 localhost

119.9.107.28 mail.server.local mail

$ cat /etc/hostname

mail.server.local

Start the Zimbra installation by executing the installer:

$ sudo ./install.sh

The installation process will ask you to agree with License Agreement. Type y and press Enter to continue:

On acceptance of agreement, Zimbra will check for dependencies and then ask for the component selection. I have chosen to skip a few components. Type y when asked for confirmation:

Type y when asked for package selection confirmation.

The installation process will take some time. As installation completes, the Zimbra configuration menu will be displayed. Here, you need to set an admin account password:

On the main menu, select 6 to choose zimbra-store and then type 4 for the admin password. The new prompt will ask for the admin account password:

Then, type r to come back to the main menu and then type a to apply settings, and again press Enter to save settings:

Finally, apply all configurations when asked. Zimbra will ask you to send installation notification to Zimbra. Choose Yes by typing y to notify Zimbra:

Now you can access your Zimbra server with the domain name of your server or IP address. Your browser may prompt for a non-trusted server certificate, as shown in the following screenshot:

You can access the Inbox panel on port 7071, https://yourserver.tld:7071.

How it works…

Zimbra combines various commonly used packages in a single package and provides a web interface to work with them. It reduces the efforts required in installing and configuring all tools separately. For any additional features, you can always switch to the Zimbra collaboration server, Network Edition.

There's more…

If you are planning to use Zimbra on your local network, you will need a DNS server set up. Alternatively, you can use the tool dnsmasq. It is a small package that sets up a quick DNS environment on your local network.

See also

Zimbra open source features at https://www.zimbra.com/open-source/features

Introduction for Performance Monitoring in Ubuntu

When starting a new server, we tend to use stock images of the Ubuntu server and default installation process. The focus is on developing and improving the application code. The base operating system is not given much attention until we hit some performance issues. Once you reach the tip of application level optimizations and have collected all low-hanging fruit, the next obvious target is system monitoring and resource optimization. In this article, we will focus on various performance monitoring tools. We will learn to use various tools to track down the bottlenecks and then briefly look at possible solutions.

The article is separated in various recipes, and each covers the monitoring of a single system resource, such as the CPU and memory. At the end of the article, we will learn how to set up a performance baseline and use it to compare different configurations of system parameters.

How to Host multiple websites with virtual domain in Ubuntu server

Setting multiple domains on a single server is a very commonly asked question. In fact, it is very easy to do this with virtual host. In this recipe, we will set up two domains on a single server and set up a sub-domain as well. We will also look at IP-based virtual hosts.

Getting ready

You will need access to a root account or an account with sudo privileges.

You will need the Apache server installed and working. This recipe describes configuration for Apache version 2.4

You may need a DNS set up if you want to access configured domains over the Internet.

We will set up two domains, namely example1.dom and example2.com, and a sub-domain, dev.example1.com.

How to do it…

Follow these steps to host multiple websites with a virtual domain:

Change the directory to /var/www and create a directory structure for the required domains and sub-domain. Also create a blank index.html for each domain:

$ cd /var/www

$ sudo mkdir -p example1.com/public_html

$ sudo touch example1.com/public_html

$ sudo cp -R example1.com example2.com

$ sudo cp -R example1.com dev.example1.com

Change the directory ownership and file permissions on the newly created directories:

$ sudo chown -R ubuntu:www-data example*

$ sudo chown -R ubuntu:www-data dev.example1.com

$ chmod 750 -R example*

$ chmod 750 -R dev.example1.com

Note the use of the wildcard syntax (chmod 750 -R example*).

You can use a similar syntax with various other commands in Linux

and save some repeated typing or copy and paste work.

Edit the index.html file for each domain with the respective text:

Next, we need to create virtual host configuration for each domain. Change the directory to /etc/apache2/sites-available and copy the default virtual host file 000-default.conf:

$ cd /etc/apache2/sites-available

$ sudo cp 000-default.conf example1.com.conf

Edit the new virtual host file and set ServerName, DocumentRoot, and other variables to match your environment. The final file should look something like this:

ServerName example1.com

ServerAlias www.example1.com

DocumentRoot /var/www/example1.com/public_html

...

Now copy this virtual host file to create example2.com.conf and dev.example1.com.conf and modify the respective settings in each of them. You need to update the serverName, serverAlias, and DocumentRoot parameters.

Here, we are done with the setup and configuration part. Now enable the virtual hosts and reload the Apache server for the settings to take effect:

$ sudo a2ensite example*

$ sudo a2ensite dev.example1.com.conf

$ sudo service apache2 reload

You can check all enabled virtual hosts with the following command:

$ sudo a2query -s

Next, to test our setup, we need to configure the hosts' setup on the local system. Open and edit the /etc/hosts file and add host entries. If you have Windows as your local system, you can find the hosts file under %systemroot%\System32\drivers\etc:

Finally, try to access domains by their names. You should see text entered in the respective index.html files for each domain:

How it works…

Multiple domain hosting works with the concept of NamedVirtualHost. We have configured virtual hosts with ServerName and ServerAlias. When a client sends a request with a domain name, it sends a host name in the request headers. This host name is used by Apache to determine the actual virtual host to serve this request. If none of the available virtual hosts match the requested host header, then the default virtual host or the first virtual host will be used to serve the request.

In this example, we have used hosts file to map test domain names with local IP. With the actual domain name, you need to point DNS servers to the IP address of your web server. Generally, all popular hosting providers host their own DNS servers. You need to add these DNS servers to your domain setting with domain registrar. Then, on your hosting side, you need to set respective A records and CNAME records. An A record points to an IP address and the CNAME record is an alias for the A record used for pointing a subdomain to an A record. Your hosting provider should give you details on how to configure domains and subdomains.

In previous versions of Apache server, you might need to enable NameVirtualHost under the configuration file. Find a line similar to #NameVirtualHost 172.20.30.40 and uncomment it by removing the # symbol at the start.

You can also set up IP-based virtual hosts. If you have multiple IP addresses available on your server, you can set the virtual host to listen on a particular IP address. Use the following steps to set up an IP-based virtual host:

Get a list of the available IP addresses:

$ ifconfig | grep "inet addr"

ubuntu@ubuntu:~$ ifconfig | grep "inet addr"

inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0

inet addr:192.168.56.102 Bcast:192.168.56.255 Mask:255.255.255.0

inet addr:127.0.0.1 Mask:255.0.0.0

Edit the virtual host configuration and set it to match the following:

Listen 80

DocumentRoot /var/www/example1.com/public_html

ServerName example1.com

See also

Apache documentation at https://httpd.apache.org/docs/2.2/vhosts/examples.html

Refer to the Installing and configuring the Apache web server recipe for the installation and configuration of the Apache web server.

Installing Git on Ubuntu server

This recipe covers the installation of Git binaries on the Ubuntu server. As always, we will install the latest available Git package.

Getting ready

You will need access to a root account or an account with sudo privileges.

How to do it…

Git maintains a separate repository of the latest binaries on Launchpad. We will use PPA for this repository,to install the latest Git version:

Add PPA to the Ubuntu installation source:

$ sudo add-apt-repository ppa:git-core/ppa

Update the apt repository cache:

$ sudo apt-get update

Now, install Git with a simple apt-get install git command:

$ sudo apt-get install git -y

Once installation completes, you can check the Git version with the following command. You can cross check the version with the official Git download page:

$ git version

Now introduce yourself to Git by providing your name and email address. Git will add this information to every commit message made by you:

$ git config --global user.name "Your Name"

$ git config --global user.email "email@domain.com"

You can cross-check the configuration by using the --list parameter to git config:

$ git config --list

Use git help to get a list of the basic daily use commands:

$ git help

How it works…

Here, we have the installed the latest Git version from the repository maintained by Git developers. The Ubuntu default package repository contains the Git package, but often it is not updated. Ubuntu 14.04 still provides Git version 1.9.1.

Once the Git packages are installed, you need to identify yourself to Git. This information is used to tag the commits created by you. We have globally set the username and email with the git config command. Now, whenever you create a new commit in any repository on this system, the commit will get tagged with your username and email. This helps in tracking who did what, especially when you are working in a large group. You can get a list of configuration settings with the command git config --list, and the output should look something like the following:

$ git config --list

user.name=yourname

user.email=youremail@example.com

If you execute the same command from within a repository directory, the list will show some extra settings specific to that repository:

~/sample-repo$ git config --list

user.name=yourname

user.email=youremail@example.com

core.repositoryformatversion=0

core.filemode=true

core.bare=false

core.logallrefupdates=true

Now, if you are not already familiar with Git, you can make use of the git help command to get documentation and manual pages. The default help menu lists commonly used commands with a short description. You can get a list of all available commands with the same git help command and a flag, -a.

$ git help -a

Additionally, the installation contains some guides or manual pages to help you get started with Git. To get a list of the available guides, use:

$ git help -g

The common Git guides are as follows:

attributes: Defines attributes per path

glossary: A Git glossary

ignore: Specifies intentionally untracked files to ignore

To open a particular guide, use the git help guidename or the man git[guidename] command:

$ git help everyday # or man giteveryday

There's more…

Git has become a mainstream version control system, especially after the rise of the social coding site GitHub. There are other well-known version control systems available, such as Subversion and Mercurial. Facebook uses a modified version of Mercurial for their internal code hosting. Bazaar is another distributed version control system sponsored and developed by Canonical, the force behind Ubuntu. Bazaar provides tight integration with Launchpad, a collaborative development platform by Canonical.

You can get more details about Bazaar on their official page at http://bazaar.canonical.com/en/ .

See also

You can read more by following these links:

Git basics: https://git-scm.com/book/en/v2/Getting-Started-Git-Basics

Git book: https://git-scm.com/book/en/v2

Check out the Git interactive tutorial at: https://try.github.io and http://git.rocks/

Launchpad: https://launchpad.net/

Introduction of Ubuntu Networking

When we are talking about server systems, networking is the first and most important factor. If you are using an Ubuntu server in a cloud or virtual machine, you generally don't notice the network settings, as they are already configured with various network protocols. However, as your infrastructure grows, managing and securing the network becomes the priority.

Networking can be thought of as an umbrella term for various activities that include network configurations, file sharing and network time management, firewall settings and network proxies, and many others. In this article, we will take a closer look at the various networking services that help us set up and effectively manage our networks, be it in the cloud or a local network in your office.

Ubuntu manual installation

In this section, we will have a look at how to install Ubuntu Server manually, either from the CD or the USB key. Here, we will only cover the installation from the CD; the same procedure applies for the installation from the USB key. In the

latter case, you should configure your computer (using the BIOS interface) to boot

from USB.

There is another way to manually install Ubuntu Server, which is via network, but since we will see this procedure in depth in the next Automated installation section, we will not cover it here.

A simple installation from a CD

The procedure for installing Ubuntu from a CD is the same as that for most of the Linux distributions. So, if you are familiar with this, you can save time and move

directly to the next section. If you are a newbie in this field, please follow this section

closely, since the next sections (advanced installation and automated installation) will be based on this. Note that for Ubuntu Server edition, there is no graphical environment like the one in the desktop edition. So, you will need to use only the keyboard and not the mouse. The stepwise procedure to install Ubunto from CD is as follows:

  1. First of all, download the installation CD image that matches your CPU architecture from the download page at http://releases.ubuntu. com/15.04/ and burn it on to an empty CD.
  2. Secondly, verify that your server BIOS is configured to boot on a CD-ROM drive; if this isn't the case, take care that you change it.
  3. Insert your CD in to the CD-ROM drive and boot it.
  4. Just after the boot process starts, you will be asked to select the installation language. Choose the one that you want. Don't worry if you choose some other language by mistake. You will soon learn how to change the settings.
  5. Now, you will get the following interface:

You can see a lot of options that cater to your needs:

° If you just press the Enter key when selecting the default Install Ubuntu Server option, you will start the installation process.

° The second option, Multiple server install with MAAS, will be covered later in the article when talking about virtualization and cloud.

° Some people prefer starting with the third option, Check disk for defects, to verify that the burn process of the ISO file on the CD was done correctly.

° The fourth option, Test memory, is very helpful, especially when your server starts crashing and you suspect a RAM-related problem.

° You can use the fifth option, Boot from first hard disk, to bypass the boot from the CD in case you forgot it by mistake in the CD-ROM driver.

° Finally, the last option, Rescue a broken system, turns a CD into a rescue disk that is useful for backup and recovery.

° In most of the cases, all that you need to do is launch the installation by pressing the Enter key when selecting the default Install Ubuntu Server option, but there are some cases where you need some special options, either because of some specific hardware-related need of your machine, or because you need to customize the kernel parameters of the server for future use. For this purpose, Ubuntu gives you a lot of possibilities via the bottom menu on the boot splash screen, which can be accessed via the function keys.

° If you press F1, you will see an interactive help screen with documentation for the rest of the options.

° If you accidentally chose the wrong language at the time of booting, press F2 to change it. The boot screen will automatically choose a keyboard mapping based on your language.

° If you want a different mapping (for example, in my case, I am preparing screenshots for this book in English but I am using a French keyboard), press F3 to choose from a list of keyboard mapping options. The boot screen also has a lot of great accessibility options.

° The F4 key displays a list of installation modes from which you can choose to install Ubuntu Server by using an OEM installation, a minimal system, and a minimal virtualization guest. The OEM installation is available for manufacturers. The minimal virtualization guest gives you an easy way to install a virtualized version of Ubuntu.

° The F5 key shows an accessibility menu that allows you to choose a high-contrast screen, a screen magnifier, a screen reader, a braille terminal, keyboard modifiers, and even an on-screen keyboard.

° The real power and control over the boot process is available once you press the F6 key. Here, you can see a menu of the common arguments that help the CD boot on difficult hardware. If you press the Esc key, you will move from this menu to the boot prompt. You can type extra kernel boot parameters that you might need for your hardware, as shown in the following screenshot:

6. Just after launching the installation process, you will be asked to choose the language of the installation procedure, which will be the default language of the server that you wish to install. Select the one that you want by using the up/down arrow keys on your keyboard and then press the Enter key. You will also be asked for the location of the server (based on this, the system

will fix the server time settings). Then, you have a choice of either letting

the installer detect your keyboard layout, or you entering it manually.

7. After performing these steps, the installer starts the installation process by detecting the hardware and loading some packages. Then, it moves on to configure the network settings by using DHCP (Dynamic Host Configuration Protocol) and asking for the hostname that you would like to give to your server. If you wouldn't like to use DHCP for network configuration, choose go back or simply press the Esc button to go to the first interface of network configuration, where you will find the Configure the network manually option, as shown in the following screenshot:

By making this choice, you will be asked via different interfaces to enter your IP address, Netmask, the default Gateway, and DNS.

Note that if you enter wrong information by mistake and you realize just after pressing the Enter key, don't worry. You can go back easily to the previous interface and re-enter the correct data. All that you need to do is choose Go Back or simply press the Esc button.

Also note that this data is not a tattoo, and you can change it later. We will have a look at how to do this in the next article.

8. At this point, we have almost finished the network settings part. We will be asked for the hostname in the next screen, and then we will move on to the user settings. First of all, you will be asked about the full username. Then, you will get an interface that asks for the Ubuntu username, which will be used for authentication with a proposition derived from the full username that you entered before. You can accept the proposition or modify it. Finally, you will be asked to enter and re-enter a password for this user. Be careful in this step because the keys entered will be hidden, as shown in the following screenshot. Therefore, it is important to remember the keys that you have entered:

Next, you will be asked whether you would like to encrypt your home directory. Don't do that unless you know what you are doing. At the end of this step, you will be asked to check your time zone. Change it in case it is incorrect.

9. Now, the most critical part of the installation arrives—partitioning! If you are using a new hard disk, you will see a screen, as shown in the following screenshot: If you are using a used hard disk, you will receive an interface that is a little different. It contains in general the same four options with the options related to the reuse of the existing partitions. The first three options are the recommended ones for a newbie. In our case, we will choose the Guided—use entire disk and set up LVM option. The Manual option will be covered in the Advanced installation section.

On pressing the Enter key, you will be taken to the next screen, where you will be asked to choose the disk that you would like to partition. You will then be taken to an interface, which will ask for your permission to write changes on the disk with the No option selected. We of course need to make changes to the disk. We should just verify that we created a backup of our data on the disk before moving on. The next step is to select the Yes option and press the Enter key. Before you finish this step, you will be asked to enter the size of each volume group that you will be using for this partitioning. You can answer with a number followed by a symbol such as MB, GB, and TB, or simply use a percentage. Finally, you will get a summary of the partitioning step, which needs to be confirmed before it is applied, as shown in the following screenshot. Select the Yes option and press the Enter key:

10. At this point, the real installation process starts. During the installation, the installer needs only the CD, but if there is an Internet connection, it can be used to download software lists and install the latest updates. For that, it will ask you to either enter the proxy settings if you have one, or to keep this field empty.

After this step, the installer will start downloading the software list (we will cover this later in the next article). This step can take some time, depending on your Internet speed:

After a few minutes, the installation process starts:

During this step, you will be asked about the update policy. Personally, I recommend that you deactivate the automatic installation of updates. It is better when the administrator manages the installation of updates manually depending on the servers that are running on Ubuntu Server.

11. One of the useful facilities that are given by the Ubuntu installer is the ability to choose to install server packages such as the SSH and DNS server. You can choose one of them by pressing the space bar, and you can continue the installation process by pressing the Enter key. In our case, we will decide not to install a server at this step. We will have a look at how to do this in a later article:

12. Finally, you will be asked to confirm the installation of the GRUB boot loader to the Master Boot Recorder (MBR). Then, you will get

the following notification at the end of the installation process:

Upgrading from an old release

In case you have an existing Ubuntu Server and you would like to upgrade it to the newest release, you can use the do-release-upgrade command. This command is a part of the update-manager-core package; it does not have any graphical dependencies and is installed by default.

You can check the options list associated with this command by running the following:

do-release-upgrade --help

You will find the following options very useful:

  • do-release-upgrade --check-dist-upgrade-only: The --check-distupgrade-only option checks for a new version. If a new version is found, it is displayed as a result in the terminal. Once executed, this command performs only a verification job; no upgrade is made.
  • do-release-upgrade --sandbox: The --sandboxoption is used to test an upgrade in a protected environment. This is particularly useful if you wish to test the deployment of an upgrade prior to its application in the production environment.
  • do-release-upgrade: The do-release-upgrade tool researches and makes an upgrade to the next LTS or stable version available, if it exists.

The upgrade policy used by the do-release-upgradetool is defined in the /etc/update-manager/release-upgrades file. The prompt variable at the end of the file indicates whether

only the LTS versions will be considered or all the versions (the LTS ones as well as the regular ones) will be searched for when asking for an upgrade. The prompt variable can take as values ltsfor the LTS versions, normalfor all versions, and neverto never search for new versions.

You should only use the Prompt=ltsmode when you are deploying a version of Ubuntu that is already an LTS. Otherwise, no new version will be detected by do-release-upgrade.

Installing Juju as a service orchestration framework on Ubuntu

Up to now in this article, we have learned about virtualization and OpenStack for deploying and managing virtual servers. Now, it's time to look at a service-modeling tool, Juju. Juju is a service-modeling tool for Ubuntu. Connect it to any cloud service, model your application, and press deploy—done. Juju takes care of lower-level configuration, deployments, and scaling, and even monitors your services.

Juju is an open source tool that offers a GUI and command-line interface for modeling your service. Applications are generally deployed as collections of multiple services. For example, to deploy WordPress, you need a web server, a database system, and perhaps a load balancer. Service modeling refers to the relations between these services. Services are defined with the help of charms, which are collections of configurations and deployment instructions, such as dependencies and resource requirements. The Juju store provides more than 300 predefined and ready-to-use charms.

Once you model your application with the required charms and their relationships, these models can be stored as a bundle. A bundle represents a set of charms, their configurations, and their relationships with each other. The entire bundle can be deployed to a cloud or local system with a single command. Also, similar to charms, bundles can be shared and are available on the Juju store.

This recipe covers the installation of Juju on Ubuntu Server. With the release of Xenial, the latest Ubuntu release, Canonical has also updated the Juju platform to version 2.0.

Getting ready

You need access to the root account or an account with sudo privileges.

Make sure you have the SSH keys generated with your user account. You can generate a new key pair with the following command:

$ ssh-keygen -t rsa -b 2048

How to do it…

Juju 2.0 is available in the Ubuntu Xenial repository, so installation is quite easy. Follow these steps to install Juju, along with LXD for local deployments:

Install Juju, along with the LXD and ZFSUtils packages. On Ubuntu 16, LXD should already be installed:

$ sudo apt-get update

$ sudo apt-get install juju-2.0 lxd zfsutils-linux

The LXD installation creates a new group, lxd, and adds the current user to it. Update your group membership with newgrp so that you don't need to log out and log back in:

$ newgrp lxd

Now, we need to initialize LXD before using it with Juju. We will create a new ZFS pool for LXD and configure a local lxd bridge for container networking with NAT enabled:

$ sudo lxd init

Name of the storage backend to use (dir or zfs): zfs

Create a new ZFS pool (yes/no)? yes

Name of the new ZFS pool: lxdpool

Would you like to use an existing block device (yes/no)? no

Size in GB of the new loop device (1GB minimum): 20

Would you like LXD to be available over the network (yes/no)? no

Do you want to configure the LXD bridge (yes/no)? yes

LXD has been successfully configured.

Now that LXD has been configured, we can bootstrap Juju and create a controller node. The following command will bootstrap Juju with LXD for local deployments:

$ juju bootstrap juju-controller lxd

This command should take some time to finish as it needs to fetch the container image and the install Juju tools inside the container.

Once the bootstrap process completes, you can check the list of controllers, as follows:

$ juju list-controllers

CONTROLLER MODEL USER SERVER

local.juju-controller* default admin@local 10.155.16.114:17070

You can also check the LXD container created by Juju using the lxc list command:

$ lxc list

From Juju 2.0 onwards, every controller will install the Juju GUI by default. This is a web application to manage the controller and its models. The following command will give you the URL of the Juju GUI:

$ juju gui

...

https://10.201.217.65:17070/gui/2331544b-1e16-49ba-8ac7- 2f13ea147497/

...

You may need to use port forwarding to access the web console. Use the following command to quickly set up iptables forwarding:

$ sudo iptables -t nat -A PREROUTING -p tcp --dport 17070 -j DNAT \

--to-destination 10.201.217.65:17070

You will also need a username and password to log in to the GUI. To get these details, use the following command:

$ juju show-controller --show-passwords juju-controller

...

accounts:

admin@local:

user: admin@local

password: 8fcb8aca6e22728c6ac59b7cba322f39

When you log in to the web console, it should look something like this:

Now, you are ready to use Juju and deploy your applications either with a command line or from the web console.

How it works…

Here, we installed and configured the Juju framework with LXD as a local deployment backend. Juju is a service-modeling framework that makes it easy to compose and deploy an entire application with just a few commands. Now, we have installed and bootstrapped Juju. The bootstrap process creates a controller node on a selected cloud; in our case, it is LXD. The command provides various optional arguments to configure controller machines, as well as pass the credentials to the bootstrap process. Check out the bootstrap help menu with the juju bootstrap --help command.

We have used LXD as a local provider, which does not need any special credentials to connect and create new nodes. When using pubic cloud providers or your own cloud, you will need to provide your username and password or access keys. This can be done with the help of the add-credentials command. All added credentials are stored in a plaintext file: ~/.local/share/juju/credentials.yaml. You can view a list of available cloud credentials with the juju list-credentials command.

The controller node is a special machine created by Juju to host and manage data and models related to an environment. The container node hosts two models, namely admin and default, and the admin model runs the Juju API server and database system. Juju can use multiple cloud systems simultaneously, and each cloud can have its own controller node.

From version 2.0 onwards, every controller node installs the Juju GUI application by default. The Juju GUI is a web application that provides an easy-to-use visual interface to create and manage various Juju entities. With its simple interface, you can easily create new models, import charms, and set up relations between them. The GUI is still available as a separate charm and can be deployed separately to any machine in a Juju environment. The command-line tools are more than enough to operate Juju, and it is possible to skip the installation of the GUI component using the --no-gui option with the bootstrap command.

There's more…

In the previous example, we used LXD as a local deployment backend for Juju. With LXD, Juju can quickly create new containers to deploy applications. Along with LXD, Juju supports various other cloud providers. You can get a full list of supported cloud providers with the list-clouds option:

$ juju list-clouds

Juju also provides the option to fetch updates to a supported cloud list. With the update-clouds subcommand, you can update your local cloud with the latest developments from Juju.

Along with public clouds, Juju also supports OpenStack deployments and MaaS-based infrastructures. You can also create your own cloud configuration and add it to Juju with the juju add-cloud command. Like with LXD, you can use virtual machines or even physical machines for Juju-based deployments. As far as you can access the machine with SSH, you can use it with Juju. Check out the cloud-configuration manual for more details: https://jujucharms.com/docs/devel/clouds-manual

See also

Read more about Juju concepts at https://jujucharms.com/docs/devel/juju-concepts

Get to know Juju-supported clouds or how to add your own at https://jujucharms.com/docs/devel/clouds

The Juju GUI: https://jujucharms.com/docs/devel/controllers-gui

Juju controllers: https://jujucharms.com/docs/devel/controllers

Refer to article 8, Working with Containers for more details about LXD containers

Learn how to connect Juju to a remote LXD server: https://insights.ubuntu.com/2015/11/16/juju-and-remote-lxd-host/

Network Storage in Ubuntu

In this article, we will cover the following recipes:

  • Installing the Samba server
  • Adding users to the Samba server
  • Installing the secure FTP server
  • Synchronizing files with Rsync
  • Performance tuning the Samba server
  • Troubleshooting the Samba server
  • Installing the Network File System

Troubleshooting the mail server on Ubuntu

Sometimes you may face problems such as e-mails not being sent, delayed delivery or mail bouncing, issues while fetching e-mails, and login failures. In this recipe, we will learn how to identify the exact problem behind these issues. We will learn how to use debugging tools and read the logs of Postfix and Dovecot.

Getting ready

You will need access to a root account or an account with sudo privileges.

It is assumed that you have already installed Postfix and Dovecot servers.

How to do it…

Follow these steps to troubleshoot the mail server:

Start with checking the status of Postfix and Dovecot. If you get output that says stop/waiting or not running then the respective service is not running:

$ sudo service postfix status

$ sudo service dovecot status

Try to restart the respective services. Restarting may give you error messages. Also check for startup logs under /var/log/mail.log:

$ sudo service postfix restart

$ less /var/log/mail.log

You can use a tail command to monitor the stream of logs while the service is running. You can easily filter the output of tail by piping it to a grep command:

$ tail -f /var/log/mail.log

Use grep to only view selected logs:

$ tail -f /var/log/mail.log | grep "dovecot"

Use grep -v to filter/remove selected logs:

$ tail -f /var/log/mail.log | grep -v "dovecot"

You can check other log files such as /var/log/mail.err and /var/log/upstart/dovecot.log.

You may want to enable verbose logging to get detailed debugging information. To enable debug mode on Dovecot, edit 10-logging.conf and enable auth_verbose and mail_debug variables:

$ sudo nano /etc/dovecot/conf.d/10-logging.conf

auth_verbose = yes

mail_debug = yes

Restart Dovecot:

$ sudo service dovecot restart

To enable verbose logging on Postfix, edit master.cf file and add the -v argument:

$ sudo nano /etc/postfix/master.cf

smtp inet n - - - - smtpd -v

Restart Postfix.

Turn off chroot operations:

$ sudo nano /etc/postfix/master.cf

smtp inet n - n - - smtpd

Check user account with Dovecot:

$ doveadm username useremail@example.com

If you have set virtual users, check if they are working properly:

$ postmap -q bob@example.org mysql:/etc/postfix/mysql-virtual- maps

Check respective ports used by Postfix and Dovecot. Postfix uses ports 25, 465, 587 and Dovecot uses port 993 and 995:

$ telnet localhost 993

Check netstat to make sure services are listening:

$ sudo netstat -plutn

Check for DNS resolution and MX records:

$ host -t mx example.com

Check if spam filters and antivirus scanners are working properly.

See also

Postfix debugging - http://www.postfix.org/DEBUG_README.html

Postfix book (troubleshooting) at http://www.postfix-book.com/debugging.html

Dovecot troubleshooting at http://wiki2.dovecot.org/WhyDoesItNotWork

Serving dynamic contents with PHP in Ubuntu

In this recipe, we will learn how to install PHP and set it to work alongside the Apache web server. We will install PHP binaries and then the Apache module mod_php to support PHP-based dynamic contents.

Getting ready

You will need access to a root account or an account with sudo privileges.

The Apache web server should be installed and working properly.

How to do it…

Follow these steps to serve dynamic contents with PHP:

Install PHP7 and the Apache module for PHP support:

$ sudo apt-get update

$ sudo apt-get install -y php7.0 libapache2-mod-php7.0

Check if PHP is properly installed and which version has been installed:

$ php -v

Create index.php under the public_html directory of our site:

$ cd /var/www/example.com/public_html

$ vi index.php

Add the following contents to index.php:

Save and exit the index.php file.

Open example.com.conf from sites-available:

$ sudo vi /etc/apache2/sites-available/example.com.conf

Add the following line under the VirtualHost directive:

DirectoryIndex index.php index.html

Save the changes and reload Apache:

$ sudo service apache2 reload

Now, access your site with your browser, and you should see a page with information regarding the installed PHP:

How it works…

Here, we have installed PHP binaries on our server along with the Apache module libapache2-mod-php7.0 to support dynamic content coded in PHP. A module, mod_php, runs inside Apache process and processes PHP scripts from within Apache itself. For mod_php to work, Apache needs to run with the mpm_prefork module. PHP setup completes all these settings and restarts the Apache server:

After we have installed PHP and mod_php, we simply need to create a PHP script. We have created index.php with little code to display phpinfo. At this stage, if you have both index.html and index.php under the same directory; by default, index.html will take over and be rendered first. You will need to explicitly specify index.php to access the page as http://127.0.0.1/index.php. We have set a directive, DirectoryIndex, under Apache Virtual Host to set index.php as a default index file.

PHP settings

All PHP settings are listed under its own configuration file, php.ini. PHP comes with two sets of configurations, as follows:

/usr/lib/php/7.0/php.ini-development

The /usr/lib/php/7.0/php.ini-productionDevelopment file is customized for a development environment and enables options like display_errors. For production systems, you can use the configuration file, php.ini-production.

The preceding files can be treated as a reference configuration that ships with the PHP installation. A copy of php.ini-production can be found under /etc/php/7.0. Apache and CLI configurations are separated in respective directories. You can directly edit settings under these files or simply use default files by creating a symbolic link to the development or production file as follows:

$ cd /etc/php/7.0/apache2

$ sudo mv php.ini php.ini.orig

$ sudo ln -s /usr/lib/php/7.0/php.ini-development php.ini

There's more…

Along with PHP, Apache supports various other scripting languages for dynamic content. You can install modules for Perl, Python, Ruby, and other scripting languages.

Add Python support:

$ sudo apt-get install libapache2-mod-python

Add Perl support:

$ sudo apt-get install libapache2-mod-perl2

Add Ruby support:

$ sudo apt-get install libapache2-mod-passenger

Installing the LAMP stack

If you are interested in installing the entire LAMP stack, then Ubuntu provides a single command to do so. Use the following command to install Apache, PHP, and MySQL collectively:

$ sudo apt-get install lamp-server^

Notice the caret symbol at the end of the command. If you miss this symbol, apt will return an error saying package not found.

lamp-server is set in the Ubuntu repository as a task to install

and configure Apache, PHP, and MySQL collectively. The caret symbol

in apt-get command is used to specify the task rather than the package.

Alternatively, you can use the tasksel command as $ sudo tasksel

install lamp-server. Tasksel is a program used to ease the installation

of packages that are commonly used together.

Upgrading PHP under Ubuntu 14

As of Ubuntu 14.10, Ubuntu does not provide a package for PHP7 in its repository, but you can use a Debian package repository to upgrade your PHP version. This repository is maintained by Ondřej Surý.

Use the following commands to upgrade to PHP 7:

$ sudo apt-get install software-properties-common

$ sudo add-apt-repository ppa:ondrej/php

$ sudo apt-get update

$ sudo apt-get install php7.0

Check the PHP version after installation completes:

$ php -v

Installing and configuring Apache in Ubuntu web server

In this recipe, we will simply install the Apache web server from the Ubuntu package repository. We will also look at the basic configuration options and set up our first web page.

Getting ready

You will need access to a root account or an account with sudo privileges.

I will be using Apache to refer to the Apache web server. The Apache web server is the most popular project by the Apache Foundation and is generally known as just Apache.

How to do it…

Follow these steps to install and configure the Apache web server:

Install Apache2 from the Ubuntu package repository:

$ sudo apt-get update

$ sudo apt-get install apache2

Check if Apache2 has installed successfully. The command wget should download the index.html file:

$ wget 127.0.0.1

You can also open a browser on a local machine and point it to the server IP address. You should see a default It works! page customized for Ubuntu:

Now, let's proceed with creating our first virtual host. First create a directory structure. Change the directory to /var/www/ and create a new directory for the contents of our site:

$ cd /var/www

$ sudo mkdir example.com

Change the ownership and group of the directory example.com:

$ sudo chown ubuntu:www-data example.com

Set the file permissions to secure web contents:

$ sudo chmod 750 example.com

Create the required directories under the example.com directory:

$ cd example.com

$ mkdir public_html

Create a index.html file under the public_html directory:

$ echo 'Hello World ...' > public_html/index.html

Next, we need to set up a new virtual host under the Apache configuration.

Copy the default Virtual Host file under /etc/apache2/sites-available and use it as a starting point for our configuration:

$ cd /etc/apache2/sites-available

$ sudo cp 000-default.conf example.com.conf

Edit example.com.conf to match it with the following example. Change the parameters as per your requirements:

Save the changes and exit example.com.conf.

If you are using the same port as the default VirtualHost, do not forget to disable the default one:

$ sudo a2dissite 000-default.conf

Finally, enable our new VirtualHost with a2ensite and reload Apache:

$ sudo a2ensite example.com.conf

$ sudo service apache2 reload

Start your browser and point it to the domain or IP address of your server:

How it works…

The Apache package for Ubuntu is included in the default package repository. We need a single command to install the Apache web server. Installation creates a structure of configuration files under /etc/apache2 and a sample web page under /var/www/html.

As mentioned in the default It works! page, Apache2 does not use a single configuration file such as httpd.conf in older versions, but rather separates its configuration across multiple configuration files. These files are named after their respective uses. apache2.conf is now a main configuration file and creates a central configuration by including all other files.

conf-available, mods-available, and sites-available contain configuration snippets and other files for global configurations, modules, and virtual hosts respectively. These configurations are selectively activated under their enabled counterparts with symbolic links for each configuration to be enabled.

envvars contains all environment variables and default values for Apache to work.

ports.conf defines the ports Apache should listen on.

The default web page is created under the /var/www/html directory.

In this recipe, we have created our virtual host for the domain name example.com and hosted it under the directory /var/www/example.com. Next, we have to change the owner and default group of this directory to the user, ubuntu and group, www-data. This grants full access to the user ubuntu and allows read and execute access to the group www-data. If you have observed the contents of the envvars file, you may have noticed that the variable APACHE_RUN_GROUP is set to www-data. This means Apache process will be started as the group www-data. By setting a default group, we have allowed Apache process to read the contents of the example.com directory. We have also enabled write access to the logs directory so that Apache processes can log to this directory.

After creating the virtual host configuration and setting the respective options, all we need to do is enable a new virtual host or site. Apache2 provides the respective commands to enable or disable configurations, modules, and sites. a2ensite will be used to enable the site from options available under sites-available. Basically, this will create a symbolic link under the sites-enabled directory to a specified site configuration. Similarly, a2dissite will disable the site by removing the symbolic link from the sites-enabled directory. Similar commands are available to work with configurations and modules.

There's more…

You may want to get rid of the warning that says Could not reliably determine the server's fully qualified domain name. This warning appears because the Apache process could not find the default FQDN for this server. You can set the default FQDN simply by creating a new configuration file and then enabling this new configuration:

Create a new file under the conf-available directory:

$ sudo vi /etc/apache2/conf-available/fqdn.conf

Add a server name variable to this file:

ServerName localhost

Save the changes and enable this configuration:

$ sudo a2enconf fqdn

Reload the Apache server:

$ sudo service apache2 reload

HTTP version 2 support

If you are looking for HTTP2 support, Apache does provide a separate module for that. Apache version 2.4.17 ships with a module, mod_http2, that implements the latest HTTP version, HTTP2. It is still an experimental implementation and needs to be enabled manually. This version of Apache (2.4.17) is available with Ubuntu Xenial (16.04) in the default package repository. If you are using Ubuntu 14.04, you can use the external repository as follows:

$ sudo add-apt-repository -y ppa:ondrej/apache2

Once the required version of Apache is installed, you can enable mod_http2 as follows:

$ sudo a2enmod http2

Next, edit the specific virtual host file to enable the HTTP2 protocol for a specific site. Note that you need to configure your site to use an SSL/TLS connection:

Protocols h2 http/1.1

...

Finally, restart your Apache server:

$ sudo service apache2 restart

H2O, the new name in web servers, is developed around the HTTP2 protocol. It does support both HTTP 1.1 and a stable implementation of the HTTP2 protocol. You may want to check this out as your local or development server.

See also

You can read more by following the links:

There is a good Q and A about permissions for web directory at http://serverfault.com/questions/357108/what-permissions-should-my-website-files-folders-have-on-a-linux-webserver

You can find more details about installing the Apache web server at https://help.ubuntu.com/lts/serverguide/httpd.html

Apache official documentation - http://httpd.apache.org/docs/2.4/

Introduction on Git Hosting

In this article, we will learn how to set up a popular version control system: Git. A version control system, also known as revision control system, can be thought of as a repository of files that record every single change in a file. Every update to a file or set of files is recorded as a new version, with some metadata about that specific modification. Metadata contains details of who made the change, a small comment explaining why the change was made, details on exactly what changed in each file, and a timestamp. You can easily switch back to an older version when needed.

Version control systems are generally used to track software source code, but they can be used with virtually any type of file. It is necessary for collaborative work where two or more people are working on the same file. Everyone maintains their own local copy of each file and works on them. When a person satisfactorily completes his work, he sends the updated file to the central repo. Others can synchronize their local copies with this central repo and receive any updates. If two people happen to modify the same file at the same time, they can choose what to keep and what to remove before sending updates to the central repository. If any issue happens with the latest updates, source code can be replaced with previous known-to-work versions. This allows you to track the changes over time and find the cause of the problem.

Over time, multiple version control systems have been developed; some are centralized version control systems (CVCS) and others are distributed version control systems. Centralized systems consist of a single central server that hosts all the versions and updates. Everyone sends new changes to the central server and gets updates from it. This makes it easy to administer the repository and enable fine-grained control, but it also becomes a candidate for a single point of failure. If a central server goes down, no one can push changes or get updates. CVS and Subversion are well known centralized version control systems.

Distributed version control systems, on the other hand, overcome this problem by distributing a full copy of the repository on each participating system. If a central server goes down, a copy from any client can be sent to the server to restore it. One can even choose to promote a client as a new server. Git, Mercurial, and Bazaar are examples of distributed version control systems. Bazaar is sponsored and developed by Canonical, the developer of Ubuntu. It is primarily focused on community-supported open source software development.

In this article, we will focus on Git, a popular version control system. It was primarily developed by Linus Torvalds to support the development of the Linux kernel. Git is influenced by the lessons learned from other version control systems. It was developed with the aim to support large projects, such as the Linux kernel, and the need for a fully distributed system and high speed. Later, GitHub, a social network for code and developers, ensured the widespread adoption of Git.

In this article, we will learn how to work with Git. Starting with the basics, such as installing Git and using it locally, we will also cover some advanced features of Git. We will also set up our own Git hosting with GitLab, an open source tool.

Ubuntu Networking

In this article, we will cover the following recipes:

  • Connecting to a network with a static IP
  • Installing the DHCP server
  • Installing the DNS server
  • Hiding behind the proxy with squid
  • Being on time with NTP
  • Discussing load balancing with HAProxy
  • Tuning the TCP stack
  • Troubleshooting network connectivity
  • Securing remote access with OpenVPN
  • Securing a network with uncomplicated firewall
  • Securing against brute force attacks
  • Discussing Ubuntu security best practices

Monitoring Docker containers in Ubuntu

In this recipe, we will learn to monitor Docker containers.

How to do it…

Docker provides inbuilt monitoring with the docker stats command, which can be used to get a live stream of the resource utilization of Docker containers.

To monitor multiple containers at once using their respective IDs or names, use this command:

$ docker stats mysql f9617f4b716c

With docker logs, you can fetch logs of your application running inside a container. This can be used similarly to the tail -f command:

$ docker logs -f ubuntu

Docker also records state change events from containers. These events include start, stop, create, kill, and so on. You can get real-time events with docker events:

$ docker events

To get past events, use the --since flag with docker events:

$ docker events --since '2015-11-01'

You can also check the changes in the container filesystem with the docker diff command. This will list newly added (A), changed (C), or deleted (D) files.

$ docker diff ubuntu

Another useful command is docker top, which helps look inside a container. This commands displays the processes running inside a container:

$ docker top ubuntu

How it works…

Docker provides various inbuilt commands to monitor containers and the processes running inside them. It uses native system constructs such as namespaces and cgroups. Most of these statistics are collected from the native system. Logs are directly collected from running processes.

Need something more, possibly a tool with graphical output? There are various such tools available. One well-known tool is cAdvisor by Google. You can run the tool itself as a Docker container, as follows:

docker run -d -p 8080:8080 --name cadvisor \

--volume=/:/rootfs:ro \

--volume=/var/run:/var/run:rw \

--volume=/sys:/sys:ro \

--volume=/var/lib/docker/:/var/lib/docker:ro \

google/cadvisor:latest

Once the container has been started, you can access the UI at your server domain or IP on port 8080 or any other port that you use. cAdvisor is able to monitor both LXC and Docker containers. In addition, it can report host system resources.

Set a proper firewall on your host system. Ubuntu comes preinstalled with UFW; you simply need to add the necessary rules and enable the firewall. Refer to article 2, Networking for more details on UFW configuration.

On Ubuntu systems, Docker ships with the AppArmor profile. This profile is installed and enforced with a Docker installation. Make sure you have AppArmor installed and working properly. AppArmor will provide better security against unknown vulnerabilities:

$ sudo apparmor_status

Next, we will move on to configure the Docker daemon. You can get a list of all available options with the docker daemon --help command:

$ docker daemon --help

You can configure these settings in the Docker configuration file at /etc/default/docker, or start the Docker daemon with all required settings from the command line.

Edit the Docker configuration and add the following settings to the DOCKER_OPTS section:

$ sudo nano /etc/default/docker

Turn off inter-container communication:

--icc=false

Set default ulimit restrictions:

--default-ulimitnproc=512:1024 --default-ulimitnofile=50:100

Set the default storage driver to overlayfs:

---storage-driver=overlay

Once you have configured all these settings, restart the Docker daemon:

$ sudo service docker restart

Now, you can use the security bench script provided by Docker. This script checks for common security best practices and gives you a list of all the things that need to be improved.

Clone the script from the Docker GitHub repository:

$ git clone https://github.com/docker/docker-bench- security.git

Execute the script:

$ cd docker-bench-security

$ sh docker-bench-security.sh

Try to fix the issues reported by this script.

Now, we will look at Docker container configurations.

The most important part of a Docker container is its image. Make sure that you download or pull the images from a trusted repository. You can get most of the images from the official Docker repository, Docker Hub.

Alternatively, you can build the images on your own server. Dockerfiles for the most popular images are quite easily available and you can easily build images after verifying their contents and making any changes if required.

When building your own images, make sure you don't add the root user:

RUN group add -r user && user add -r -g user user

USER user

When creating a new container, make sure that you configure CPU and memory limits as per the containers requirements. You can also pass container-specific ulimit settings when creating containers:

$ docker run --cpu-shares1024 --memory 512 --cpuset-cpus 1

Whenever possible, set your containers to read-only:

$ docker run --read-only

Use read-only volumes:

$ docker run -v /shared/path:/container/path:ro ubuntu

Try not to publish application ports. Use a private Docker network or Docker links when possible. For example, when setting up WordPress in the previous recipe, we used a Docker network and connected WordPress and MySQL without exposing MySQL ports.