Skip to main content

Ubuntu

Monitoring Docker containers in Ubuntu

In this recipe, we will learn to monitor Docker containers.

How to do it…

Docker provides inbuilt monitoring with the docker stats command, which can be used to get a live stream of the resource utilization of Docker containers.

To monitor multiple containers at once using their respective IDs or names, use this command:

$ docker stats mysql f9617f4b716c

With docker logs, you can fetch logs of your application running inside a container. This can be used similarly to the tail -f command:

$ docker logs -f ubuntu

Docker also records state change events from containers. These events include start, stop, create, kill, and so on. You can get real-time events with docker events:

$ docker events

To get past events, use the --since flag with docker events:

$ docker events --since '2015-11-01'

You can also check the changes in the container filesystem with the docker diff command. This will list newly added (A), changed (C), or deleted (D) files.

$ docker diff ubuntu

Another useful command is docker top, which helps look inside a container. This commands displays the processes running inside a container:

$ docker top ubuntu

How it works…

Docker provides various inbuilt commands to monitor containers and the processes running inside them. It uses native system constructs such as namespaces and cgroups. Most of these statistics are collected from the native system. Logs are directly collected from running processes.

Need something more, possibly a tool with graphical output? There are various such tools available. One well-known tool is cAdvisor by Google. You can run the tool itself as a Docker container, as follows:

docker run -d -p 8080:8080 --name cadvisor \

--volume=/:/rootfs:ro \

--volume=/var/run:/var/run:rw \

--volume=/sys:/sys:ro \

--volume=/var/lib/docker/:/var/lib/docker:ro \

google/cadvisor:latest

Once the container has been started, you can access the UI at your server domain or IP on port 8080 or any other port that you use. cAdvisor is able to monitor both LXC and Docker containers. In addition, it can report host system resources.

Set a proper firewall on your host system. Ubuntu comes preinstalled with UFW; you simply need to add the necessary rules and enable the firewall. Refer to article 2Networking for more details on UFW configuration.

On Ubuntu systems, Docker ships with the AppArmor profile. This profile is installed and enforced with a Docker installation. Make sure you have AppArmor installed and working properly. AppArmor will provide better security against unknown vulnerabilities:

$ sudo apparmor_status

Next, we will move on to configure the Docker daemon. You can get a list of all available options with the docker daemon --help command:

$ docker daemon --help

You can configure these settings in the Docker configuration file at /etc/default/docker, or start the Docker daemon with all required settings from the command line.

Edit the Docker configuration and add the following settings to the DOCKER_OPTS section:

$ sudo nano /etc/default/docker

Turn off inter-container communication:

--icc=false

Set default ulimit restrictions:

--default-ulimitnproc=512:1024 --default-ulimitnofile=50:100

Set the default storage driver to overlayfs:

---storage-driver=overlay

Once you have configured all these settings, restart the Docker daemon:

$ sudo service docker restart

Now, you can use the security bench script provided by Docker. This script checks for common security best practices and gives you a list of all the things that need to be improved.

Clone the script from the Docker GitHub repository:

$ git clone https://github.com/docker/docker-bench- security.git

Execute the script:

$ cd docker-bench-security

$ sh docker-bench-security.sh

Try to fix the issues reported by this script.

Now, we will look at Docker container configurations.

The most important part of a Docker container is its image. Make sure that you download or pull the images from a trusted repository. You can get most of the images from the official Docker repository, Docker Hub.

Alternatively, you can build the images on your own server. Dockerfiles for the most popular images are quite easily available and you can easily build images after verifying their contents and making any changes if required.

When building your own images, make sure you don't add the root user:

RUN group add -r user && user add -r -g user user

USER user

When creating a new container, make sure that you configure CPU and memory limits as per the containers requirements. You can also pass container-specific ulimit settings when creating containers:

$ docker run --cpu-shares1024 --memory 512 --cpuset-cpus 1

Whenever possible, set your containers to read-only:

$ docker run --read-only

Use read-only volumes:

$ docker run -v /shared/path:/container/path:ro ubuntu

Try not to publish application ports. Use a private Docker network or Docker links when possible. For example, when setting up WordPress in the previous recipe, we used a Docker network and connected WordPress and MySQL without exposing MySQL ports.

Preparation before Ubuntu installation

In this section, we will take a quick look at the latest Ubuntu Server release news and then, we will make a list of all the system requirements.

The latest Ubuntu release

Canonical, the company that produces Ubuntu, releases a new version every 6 months. Each release has a code with a YY.ZZ pattern, where YY is the year and ZZ is the month.

I started writing this book just after Ubuntu 15.04 (Vivid Vervet) was released on April 23, 2015. Currently, there are two major releases—the LTS one that was released last year (LTS stands for long-term support), which is version 14.04, and the latest version 15.04. It is not a big deal if you choose either of these two versions to perform the tasks in the coming articles, since it will make no difference. So, we decided to use the latest version as a reference for our samples, especially because the next LTS release will be based on it. (Note that only the LTS releases are supported for 5 years by Canonical, and the non-LTS releases have a support of only 9 months. That's why we normally choose the LTS versions for Ubuntu Server deployments.) When there is a notable difference between these two versions, we will mention it.

Now, let's take a look at some information related to the latest version:

  • It uses Linux kernel 3.19, which brings a lot of improvements in terms of performance as well as network facilities for both servers and cloud.
  • 15.04 is the first Ubuntu version that features LXD.
  • It uses the latest versions of OpenStack, LXC (Linux Containers), LXD, Juju, libvirt, QEMU, Open vSwitch, Ceph, cloud-init, Docker, and HA-related package updates.
  • It replaced the service manager and the standard boot upstart with systemd.

The upstartboot still exists under Ubuntu. You can use it by opening the GRUB boot menu, choosing Advanced options for Ubuntu, and then clicking on Ubuntu, with Linux (upstart).

If you would like to switch permanently to the upstartboot, you can install the upstart-sysvpackage, which will remove ubuntu-standardand systemd-sysv.

System requirements

System requirements depend on the services that may need to be deployed in the future and installed on the server. For demonstration/test purposes, we need a

minimal configuration of 300 MHz CPU, 192 MB of RAM, and a 1.5 GB hard disk. This light configuration allows us to deploy Ubuntu Server on an old computer or even

a little virtual machine. This limited footprint is basically due to the absence of the X Windows System (graphic interface), which is not needed in a server environment.

In a production environment, you should be careful about your actual needs in terms

of resources (the CPU, RAM, and hard disk) and the estimation growth of those needs. To do this, you need to make a good measure of dimensions based on the services that you are going to deploy.

In the case of the samples in this book, we will use the 64-bit version of Ubuntu Server 15.04, and we will install it on a virtual box machine that has 1 GB of RAM and 2 TB of hard disk.

Note that if you are using a used computer/server, you should back up your data before installing or upgrading Ubuntu. Partitioning tools used in the installation process are reliable and can be used for many years without any problems in general, but sometimes, they can perform catastrophic actions.

Additional resources

This book comprises only the essentials. It contains exactly what you need to know

to perform a specific task. If you need more information about and an in-depth understanding of Ubuntu, you can have a look at the official documentation by

visiting https://help.ubuntu.com .

You should download the CD image according to your system architecture. The whole list exists at http://releases.ubuntu.com/15.04/ .

OpenStack Virtual instance in Ubuntu

Now that we have OpenStack installed and have set our desired operating system image, we are ready to launch our first instance in a self-hosted cloud.

Getting ready

You will need credentials to access the OpenStack dashboard.

Uploading your own image is not necessary; you can use the default Cirros image to launch the test instance.

Log in to the OpenStack dashboard and set the SSH key pair in the Access & Security tab available under the Projects menu. Here, you can generate a new key pair or import your existing public key.

How to do it…

OpenStack instances are the same virtual machines that we launch from the command line or desktop tools. OpenStack give you a web interface to launch your virtual machines from. Follow these steps to create and start a new instance:

Select the Instance option under the Projects menu and then click on the Launch Instance button on the right-hand side. This should open a modal box with various options, which will look something like this:

Now, start filling in the necessary details. All fields that are marked with * are required fields. Let's start by naming our instance. Enter the name in the Instance Name field.

Set the value of Count to the number of instances you want to launch. We will leave it at the default value of 1.

Next, click on the Source tab. Here, we need to configure the source image for our instance. Set Select Boot Source to Image and select No for Create New Volume. Then, from the Available Images section, search the desired image and click on the button with the + sign to select the image. The list should contain our recently uploaded image. The final screen should look something like this:

Next, on the Flavor tab, we need to select the desired resources for our instance. Select the desired flavor by clicking on the + button. Make sure that the selected row does not contain any warning signs.

Now, from the Key Pair tab, select the SSH key pair that we just created. This is required to log in to your instance.

Finally, click on the Launch Instance button from the bottom of the modal box. A new instance should be created and listed under the instances list. It will take some time to start; wait for the Status column to show Active:

You are now ready to access your virtual instance. Log in to your host console and try to ping the IP address of your instance. Then, open an SSH session with the following command:

$ ssh -i your_key ubuntu@instance_ip

This should give you a shell inside your new cloud instance. Try to ping an external server, such as an OpenDNS server, from within an instance to ensure connectivity.

To make this instance available on your local network, you will need to assign a floating IP address to it. Click on the drop-down arrow from the Actions column and select Associate Floating IP. This should add one more IP address to your instance and make it available on your local network.

How it works…

OpenStack instances are the same as the virtual machines that we build and operate with common virtualization tools such as VirtualBox and Qemu. OpenStack provides a central console for deploying and managing thousands of such machines on multiple hosts. Under the hood, OpenStack uses the same virtualization tools as the others. The preferred hypervisor is KVM, and if hardware acceleration is not available, Qemu emulation is used. OpenStack supports various other hypervisors, including VMware, XEN, Hyper-V, and Docker. In addition, a lightervisor, LXD, is on its way to a stable release. Other than virtualization, OpenStack adds various other improvements, such as image management, block storage, object storage, and various network configurations.

In the previous example, we set various parameters before launching a new instance; these include the instance name, resource constraints, operating system image, and login credentials. All these parameters will be passed to the underlying hypervisor to create and start the new virtual machine. A few other options that we have not used are volumes and networks. As we have installed a very basic OpenStack instance, new developments in network configurations are not available for use. You can update your DevStack configuration and install the OpenStack networking component Neutron.

Volumes, on the other hand, are available and can be used to obtain disk images of the desired size and format. You can also attach multiple volumes to a single machine, providing extended storage capacity. Volumes can be created separately and do not depend on the instance. You can reuse an existing volume with a new instance, and all data stored on it will be available to the new instance.

Here, we have used a cloud image to start a new instance. You can also choose a previously stored instance snapshot, create a new volume, or use a volume snapshot. The volume can be a permanent volume, which has its life cycle separate from the instance, or an ephemeral volume, which gets deleted along with the instance. Volumes can also be attached at instance runtime or even removed from an instance, provided they are not a boot source.

Other options include configuration and metadata. The configuration tab provides an option to add initialization scripts that are executed at first boot. This is very similar to cloud-init data. The following is a short example of a cloud-init script:

#cloud-config

package_update: true

package_upgrade: true

password: password

chpasswd: { expire: False }

ssh_pwauth: True

ssh_authorized_keys:

- your-ssh-public-key-contents

This script will set a password for the default user (ubuntu in the case of Ubuntu images), enable password logins, add an SSH key to authorize keys, and update and upgrade packages.

The metadata section adds arbitrary data to instances in the form of key-value pairs. This data can be used to identify an instance from a group and automate certain tasks.

Once an instance has been started, you have various management options from the Actions menu available on the instance list. From this menu, you can create instance snapshots; start, stop, or pause instances; edit security groups; get the VNC console; and so on.

There's more…

Similar to the glance command-line client, a compute client is available as well and is named after the compute component. The nova command can be used to create and manage cloud instances from the command line. You can get detailed parameters and options with the nova help command or, to get help with a specific subcommand, nova help .

See also

The cloud-init official documentation: https://cloudinit.readthedocs.io/en/latest/

More on cloud-init: https://help.ubuntu.com/community/CloudInit

OpenStack instance guide: http://docs.openstack.org/user-guide/dashboard_launch_instances.html

Command-line cheat sheet: http://docs.openstack.org/user-guide/cli_cheat_sheet.html#compute-nova

Storing and retrieving data with MongoDB in Ubuntu

In this recipe, we will look at basic CRUD operations with MongoDB. We will learn how to create databases, store, retrieve, and update stored data. This is a recipe to get started with MongoDB.

Getting ready

Make sure that you have installed and configured MongoDB. You can also use the MongoDB installation on a remote server.

How to do it…

Follow these steps to store and retrieve data with MongoDB:

Open a shell to interact with the Mongo server:

$ mongo

To open a shell on a remote server, use the command given. Replace server_ip and port with the respective values:

$ mongo server_ip:port/db

To create and start using a new database, type use dbname. Since schemas in MongoDB are dynamic, you do not need to create a database before using it:

> use testdb

You can type help in Mongo shell to get a list of available commands and help regarding a specific command:

> help: Let’s insert our first document:

> db.users.insert({‘name’:’ubuntu’,’uid’:1001})

To view the created database and collection, use the following commands:

> show dbs

> show collections

You can also insert multiple values for a key, for example, which groups a user belongs to:

> db.users.insert({‘name’:’root’,’uid’:1010, ‘gid’:[1010, 1000, 1111]})

Check whether a document is successfully inserted:

> db.users.find()

To get a single record, use findOne():

> db.users.findOne({uid:1010})

To update an existing record, use the update command as follows:

> db.users.update({name:’ubuntu’}, {$set:{uid:2222}})

To remove a record, use the remove command. This will remove all records with a name equal to ubuntu:

> db.users.remove({‘name’:’ubuntu’})

To drop an entire collection, use the drop() command:

> db.users.drop()

To drop a database, use the dropDatabase() command:

> db.users.dropDatabase()

How it works…

The preceding examples show very basic CRUD operations with the MongoDB shell interface. MongoDB shell is also a JavaScript shell. You can execute all JS commands in a MongoDB shell. You can also modify the shell with the configuration file, ~/.mongorc.js. Similar to shell, MongoDB provides language-specific drivers, for example, MongoDB PHP drivers to access MongoDB from PHP.

MongoDB works on the concept of collections and documents. A collection is similar to a table in MySQL and a document is a set of key value stores where a key is similar to a column in a MySQL table. MongoDB does not require any schema definitions and accepts any pair of keys and values in a document. Schemas are dynamically created. In addition, you do not need to explicitly create the collection. Simply type a collection name in a command and it will be created if it does not already exist. In the preceding example, users is a collection we used to store all data. To explicitly create a collection, use the following command:

> use testdb

> db.createCollection(‘users’)

You may be missing the where clause in MySQL queries. We have already used that with the findOne() command:

> db.users.findOne({uid:1010})

You can use $lt for less than, $lte for less than or equal to, $gt for greater than, $gte for greater than or equal to, and $ne for not equal:

> db.users.findOne({uid:{$gt:1000}})

In the preceding example, we have used the where clause with the equality condition uid=1010. You can add one more condition as follows:

> db.users.findOne({uid:1010, name:’root’})

To use the or condition, you need to modify the command as follows:

> db.users.find ({$or:[{name:’ubuntu’}, {name:’root’}]})

You can also extract a single key (column) from the entire document. The find command accepts a second optional parameter where you can specify a select criteria. You can use values 1 or 0. Use 1 to extract a specific key and 0 otherwise:

> db.users.findOne({uid:1010}, {name:1})

> db.users.findOne({uid:1010}, {name:0})

There’s more…

You can install a web interface to manage the MongoDB installation. There are various open source web interfaces listed on Mongo documentation at http://docs.mongodb.org/ecosystem/tools/administration-interfaces/ .

When you start a mongo shell for the first time, you may see a warning message regarding transperent_hugepage and defrag. To remove those warnings, add the following lines to /etc/init/mongod.conf, below the $DAEMONUSER /var/run/mongodb.pid line:

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then

echo never > /sys/kernel/mm/transparent_hugepage/enabled

fi

if test -f /sys/kernel/mm/transparent_hugepage/defrag; then

echo never > /sys/kernel/mm/transparent_hugepage/defrag

fi

Find more details on this Stack Overflow post at http://stackoverflow.com/questions/28911634/how-to-avoid-transparent-hugepage-defrag-warning-from-mongodb

See also

Mongo CRUD tutorial at https://docs.mongodb.org/manual/applications/crud/

MongoDB query documents at https://docs.mongodb.org/manual/tutorial/query-documents/

Mail filtering with spam-assassin in Ubuntu

In this recipe, we will learn how to install and set up a well-known e-mail filtering program, spam-assassin.

Getting ready

You will need access to a root account or an account with sudo privileges.

You need to have Postfix installed and working.

How to do it…

Follow these steps to filter mail with spam-assassin:

Install spam-assassin with the following command:

$ sudo apt-get update

$ sudo apt-get install spamassassin spamc

Create a user account and group for spam-assassin:

$ sudo groupadd spamd

$ sudo useradd -g spamd -s /usr/bin/nologin \

-d /var/log/spamassassin -m spamd

Change the default settings for the spam daemon. Open /etc/default/spamassassin and update the following lines:

ENABLED=1

SAHOME="/var/log/spamassassin/"

OPTIONS="--create-prefs --max-children 5 --username spamd - -helper-home-dir ${SAHOME} -s ${SAHOME}spamd.log"

PIDFILE="${SAHOME}spamd.pid"

CRON=1

Optionally, configure spam rules by changing values in /etc/spamassassin/local.cf:

trusted_networks 10.0.2. # set your trusted network

required_score 3.0 # 3 + will be marked as spam

Next, we need to change the Postfix settings to pass e-mails through spam- assassin. Open /etc/postfix/master.cf and find the following line:

smtp inet n - - - - smtpd

Add the content filtering option:

-o content_filter=spamassassin

Define the content filter block by adding the following lines to the end of the file:

spamassassin unix - n n - - pipe

user=spamd argv=/usr/bin/spamc -f -e

/usr/sbin/sendmail -oi -f ${sender} ${recipient}

Finally, restart spam-assassin and Postfix:

$ sudo service spamassassin start

$ sudo service postfix reload

You can check spam-assassin and mail logs to verify that spam-assassin is working properly:

$ less /var/log/spamassassin/spamd.log

$ less /var/log/mail.log

How it works…

Spam filtering works with the help of a piping mechanism provided by Postfix. We have created a new Unix pipe which will be used to filter e-mails. Postfix will pass all e-mails through this pipe, which will be then scanned through spam-assassin to determine the spam score. If given e-mail scores below the configured threshold, then it passes the filter without any modification; otherwise, spam-assassin adds a spam header to the e-mail.

Spam-assassin works with a Bayesian classifier to classify e-mails as spam or not spam. Basically, it checks the content of the e-mail and determines the score based on content.

There's more…

You can train spam-assassin's Bayesian classifier to get more accurate spam detections.

The following command will train spam-assassin with spam contents (--spam):

$ sudo sa-learn --spam -u spamd --dir ~/Maildir/.Junk/* -D

To train with non-spam content, use the following command (--ham):

$ sudo sa-learn --ham -u spamd --dir ~/Maildir/.INBOX/* -D

If you are using the mbox format, replace --dir ~/Maildir/.Junk/* with the option --mbox.

See also

Sa-learn - train SpamAssassin's Bayesian classifier at https://spamassassin.apache.org/full/3.2.x/doc/sa-learn.html and https://wiki.apache.org/spamassassin/BayesInSpamAssassin

Learn about Bayesian classification at https://en.wikipedia.org/wiki/Naive_Bayes_classifier

Installing OwnCloud, self-hosted cloud storage

OwnCloud is a self-hosted file storage and synchronization service. It provides client tools to upload and sync all your files to a central storage server. You can access all your data through a well-designed web interface, which can be accessed on any device of your choice. In addition to a simple contact service, OwnCloud supports contacts, email, and calendar synchronization. Plus, all your data is stored on your own server, making it a more secure option.

In this recipe, we will learn how to install the OwnCloud service on the Ubuntu server. We will be working with a basic OwnCloud setup that includes file sharing and storage. Later, you can add separate plugins to extend the capability of your OwnCloud installation.

Getting ready

You will need access to an account with sudo privileges.

How to do it…

OwnCloud is a PHP-based web application. Its dependencies include a web server, PHP runtime, and a database server. We will use the installation package provided by OwnCloud. The package takes care of all dependencies, plus it will help in updating our installation whenever a new version is available. We will install the latest stable version of OwnCloud. As of writing this, OwnCloud does not provide any packages for Ubuntu 16.04. I have used the package for Ubuntu 15.10:

Add the OwnCloud repository public key to your Ubuntu server:

$ wget https://download.owncloud.org/download/repositories/stable/Ubu ntu_15.10/Release.key -O owncloud.key

$ sudo apt-key add - owncloud.key

Next, add the OwnCloud repository to installation sources. Create a new source list:

$ sudo touch /etc/apt/sources.list.d/owncloud.list

Add an installation path to the newly created source list:

$ sudo nano /etc/apt/sources.list.d/owncloud.list

deb http://download.owncloud.org/download/repositories/stable/Ubun tu_15.10/ /

Update installation sources with the apt-get update command:

$ sudo apt-get update

Install the OwnCloud package. This will download and install all dependencies, download the OwnCloud package, and set up the Apache web server virtual host configuration. By default, OwnCloud use SQLite as a default database. This can be changed at the signup page:

$ sudo apt-get install owncloud

Once installed, you can access your OwnCloud installation at http://your_server/owncloud . This will open the registration page for an admin account. Enter the admin username and password for a new account. The first user to register will be marked as the admin of the OwnCloud instance.

The same page contains a warning saying the default database is SQLite. Click the configure database link; this will show you the option to enter database connection details. Enter all the required details and click submit.

Once registration completes, you will be redirected to the OwnCloud homepage. If you need any help, this page contains the OwnCloud user manual. You can start uploading content or create new text files right from the homepage.

Optionally, install OwnCloud desktop and mobile applications to sync files across all your devices.

How it works…

OwnCloud is a web application that enables you to synchronize and share files across the web. Store a backup of all your files on a central OwnCloud server, or use it as a central place to send and receive files. OwnCloud also provides native applications for all platforms so that you can easily replicate the necessary data across all your devices. Once you have logged in to your account, OwnCloud will list the default directory structure with a PDF file for the user manual. The screen should look similar to the following:

With the recent updates, OwnCloud has removed various default packages and reduced the overall binary size. For now, the default installation contains a file browser, an activity monitor, and a gallery. The file browser supports the uploading, viewing, and sharing of files. You can create new text files and open PDF files right from the browser:

Default features can be extended from the Apps submenu accessible from the Files link at the top, left of the screen. It gives you a list of installed and enabled or disabled apps. Plus, you can search for apps across categories such as Multimedia, Productivity, Games, and Tools. Choose your desired category, scroll to the desired app and click enable to install a new component:

OwnCloud also allows flexible user management. When logged in as an admin user, you can access the Users menu from the top-right login section of the screen. Under users, you can create a new user, assign them to a group, create a new group, and even set the disk quota allowed:

Next is the admin section, which is again accessible to users from the admin group at the top-right of the screen. This section lists all the administrative settings relating to the core OwnCloud setup, as well as for installed apps. Each section contains a link to detailed documentation. The important part of the settings is the email server setup. By default, OwnCloud uses default PHP-based emails. It is recommended you set up an SMTP service. You can use external SMTP service providers, such as MailChimp, or set up your own SMTP server. At the bottom of the admin settings page, you can see some links to improve your OwnCloud experience. This includes performance tuning the OwnCloud setup, security guidelines, theme support, and so on.

See also

OwnCloud repositories: https://download.owncloud.org/download/repositories/stable/owncloud/

OwnCloud admin manual: https://doc.owncloud.org/server/8.2/admin_manual/

Introduction for Working with Ubuntu Web Servers

A web server is a tool that publishes documents on a network, generally the Internet. HTTP is called a language of the Internet and web servers, apart from browsers, are native speakers of HTTP. Web servers generally listen on one or multiple ports for requests from clients and accept requests in the form of URLs and HTTP headers. On receiving a request, web servers look for the availability of the requested resource and return the contents to the client. The term web server can refer to one or multiple physical servers or a software package, or both of them working together.

Some well known web servers include the Apache web server, Microsoft IIS, and Nginx. Apache web server is the most popular web server package available across platforms such as Windows and Linux. It is an open source project and freely available for commercial use. Nginx, which is again an open source web server project, started to overcome the problems in a high-load environment. Because of its lightweight resource utilization and ability to scale even on minimal hardware, Nginx quickly became a well known name. Nginx offers a free community edition as well as a paid commercial version with added support and extra features. Lastly, Microsoft IIS is a web server specifically designed for Windows servers. Apache still has the major share in the web server market, with Nginx rapidly taking over with some other notable alternatives such as lighttpd and H2O.

Apache is a modularized web server that can be extended by dynamically loading extra modules as and when required. This provides the flexibility to run a bare minimum web server or a fully featured box with modules to support compression, SSL, redirects, language modules, and more. Apache provides multiple connection processing algorithms called multi-processing modules (MPM). It provides an option to create a separate single threaded process for each new request (mpm_prefork), a multi-threaded process that can handle multiple concurrent requests (mpm_worker), or the latest development of mpm_event, which separates the active and idle connections.

Nginx can be considered the next generation of web servers. Its development started to solve the C10k problem, that is, handling ten thousand connections at a time. Apache, being a process-driven model, has some limitations when handling multiple concurrent connections. Nginx took advantage of the event-driven approach with asynchronous, non-blocking connection handling algorithms. A new connection request is handled by a worker process and placed in an event loop where they are continuously checked for events. The events are processed asynchronously. This approach enables Nginx to run with a much lower memory footprint and lower CPU use. It also eliminates the overload of starting a new process for a new connection. A single worker process started by Nginx can handle thousands of concurrent connections.

Both Apache and Nginx can be configured to process dynamic contents. Apache provides respective language processors such as mod_php and mod_python to process dynamic contents within the worker process itself. Nginx depends on external processors and uses CGI protocols to communicate with external processors. Apache can also be configured to use an external language processor over CGI, but the choice depends on performance and security considerations.

While both Apache and Nginx provide various similar features, they are not entirely interchangeable. Each one has its own pros and cons. Where Nginx excels at serving static contents, Apache performs much better processing dynamic contents. Many web administrators prefer to use Apache and Nginx together.

Nginx is commonly used as a frontend caching/reverse proxy handling client

requests and serving static contents, while Apache is used as a backend

server processing dynamic contents.

Nginx handles a large number of connections and passes limited requests of dynamic contents to backend Apache servers. This configuration also allows users to scale horizontally by adding multiple backend servers and setting Nginx as a load balancer.

In this article, we will be working with both Apache and Nginx servers. We will learn how to set up Apache with PHP as a language for dynamic contents. We will look at some important configurations of Apache. Later, we will set up Nginx with an optional PHP processor, PHP_FPM, and configure Nginx to work as a reverse proxy and load balancer. We will also look at performance and security configurations for both the servers.

Git Hosting

In this article, we will cover the following recipes:

  • Installing Git
  • Creating a local repository with Git CLI
  • Storing file revisions with Git commit
  • Synchronizing the repository with a remote server
  • Receiving updates with Git pull
  • Creating repository clones
  • Installing GitLab, your own Git hosting
  • Adding users to the GitLab server
  • Creating a repository with GitLab
  • Automating common tasks with Git hooks

How to secure Ubuntu user accounts

In this recipe, we will look at ways to make user profiles more secure.

How to do it...

Follow these steps to secure the user account:

Set a strong password policy with the following steps:

Open the /etc/pam.d/common-password file with GNU nano:

$ sudo nano /etc/pam.d/common-password

Find the line similar to this:

password [success=1 default=ignore] pam_unix.so obscure sha512

Add minlen to the end of this line:

password [success=1 default=ignore] pam_unix.so obscure sha512 minlen=8

Add this line to enforce alphanumeric passwords:

password requisite pam_cracklib.so ucredit=-1 lcredit=-1 dcredit=-1 ocredit=-1

Save changes and exit GNU nano editor.

Press Ctrl + O to save changes.

Press Ctrl + X to exit GNU nano editor.

Secure the home directory with the following steps:

Check home directory permissions with the following command:

$ ls -ld /home/username

Restrict permissions to user and group with the following command:

$ chmod 750 /home/username

Change adduser default permissions by editing /etc/adduser.conf. Find DIR_MODE=0755 and change it to DIR_MODE=0750.

Disable SSH access to root user with the following step:

Open /etc/ssh/sshd_config and add or edit PermitRootLogin to PermitRootLogin no

Disable password authentication with the following step:

Open /etc/ssh/sshd_config and add or edit PasswordAuthentication no

Install fail2ban with sudo apt-get install fail2ban.

How it works…

This recipe discussed a few important steps to make user accounts more secure.

A password is the most important aspect in securing user accounts. A weak password can be easily broken with brute force attacks and dictionary attacks. It is always a good idea to avoid password-based authentication, but if you are still using it, then make sure you enforce a strong password policy.

Password authentication is controlled by the PAM module pam_unix, and all settings associated with login are listed at /etc/pam.d/login. An additional configuration file /etc/pam.d/common-password includes values that control password checks.

The following line in the primary block of common-password file defines the rules for password complexity:

password [success=1 default=ignore] pam_unix.so obscure sha512

The default setting already defines some basic rules on passwords. The parameter obscure defines some extra checks on password strength. It includes the following:

Palindrome check

Case change only

Similar check

Rotated check

The other parameter, sha512, states that the new password will be encrypted with the sha512 algorithm. We have set another option, minlen=8, on the same line, adding minimum length complexity to passwords.

For all settings of the pam_unix module, 

refer to the manual pages with the command man pam_unix.

Additionally, we have set alphanumeric checks for new passwords with the PAM module pam_cracklib:

password requisite pam_cracklib.so ucredit=-1 lcredit=-1 dcredit=- 1 ocredit=-1

The preceding line adds requirement of one uppercase letter, one lowercase letter, one digit (dcredit), and one special character (ocredit)

There are other PAM modules available, and you can search them with the following command:

$ apt-cache search limpam-

You might also want to secure the home directory of users. The default permissions on Ubuntu allow read and execute access to everyone. You can limit the access on the home directory by changing permission on the home directory as required. In the preceding example, we changed permissions to 750. This allows full access to the user, and allows read and execute access to the user's primary group.

You can also change the default permissions on the user's home directory by changing settings for the adduser command. These values are located at /etc/adduser.conf. We have changed default permissions to 750, which limits access to the user and the group only.

Additionally, you can disable remote login for the root account as well as disable password-based authentication. Public key authentication is always more secure than passwords, unless you can secure your private keys. Before disabling password authentication, ensure that you have properly enabled public key authentication and you are able to log in with your keys. Otherwise, you will lock yourself out of the server.

You might want to install a tool like fail2ban to watch and block repeated failed actions. It scans through access logs and automatically blocks repeated failed login attempts. This can be a handy tool to provide a security against brute force attacks.

Deploying WordPress using a Docker network in Ubuntu

In this recipe, we will learn to use a Docker network to set up a WordPress server. We will create two containers, one for MySQL and the other for WordPress. Additionally, we will set up a private network for both MySQL and WordPress.

How to do it…

Let's start by creating a separate network for WordPress and the MySQL containers:

A new network can be created with the following command:

$ docker network create wpnet

Check whether the network has been created successfully with docker network ls:

$ docker network ls

You can get details of the new network with the docker network inspect command:

$ docker network inspect wpnet

Next, start a new MySQL container and set it to use wpnet:

$ docker run --name mysql -d \

-e MYSQL_ROOT_PASSWORD=password \

--net wpnet mysql

Now, create a container for WordPress. Make sure the WORDPRESS_DB_HOST argument matches the name given to the MySQL container:

$ docker run --name wordpress -d -p 80:80 \

--net wpnet\

-e WORDPRESS_DB_HOST=mysql\

-e WORDPRESS_DB_PASSWORD=password wordpress

Inspect wpnet again. This time, it should list two containers:

Now, you can access the WordPress installation at your host domain name or IP address.

How it works…

Docker introduced the container networking model (CNM) with Docker version 1.9. CNM enables users to create small, private networks for a group of containers. Now, you can set up a new software-assisted network with a simple docker network create command. The Docker network supports bridge and overlay drivers for networks out of the box. You can use plugins to add other network drivers. The bridge network is a default driver used by a Docker network. It provides a network similar to the default Docker network, whereas an overlay network enables multihost networking for Docker clusters.

This recipe covers the use of a bridge network for wordpress containers. We have created a simple, isolated bridge network using the docker network command. Once the network has been created, you can set containers to use this network with the --net flag to docker run command. If your containers are already running, you can add a new network interface to them with the docker network connect command, as follows:

$ # docker network connect network_name container_name

$ docker network connect wpnet mysql

Similarly, you can use docker network disconnect to disconnect or remove a container from a specific network. Additionally, this network provides an inbuilt discovery feature. With discovery enabled, we can communicate with other containers using their names. We used this feature while connecting the MySQL container to the wordpress container. For the WORDPRESS_DB_HOST parameter, we used the container name rather than the IP address or FQDN.

If you've noticed, we have not mentioned any port mapping for the mysql container. With this new wpnet network, we need not create any port mapping on the MySQL container. The default MySQL port is exposed by the mysql container and the service is accessible only to containers running on the wpnet network. The only port available to the outside world is port 80 from the wordpress container. We can easily hide the WordPress service behind a load balancer and use multiple wordpress containers with just the load balancer exposed to the outside world.

There's more…

Docker also supports links to create secure communication links between two or more containers. You can set up a WordPress site using linked containers as follows:

First, create a mysql container:

$ docker run --name mysql -d \

-e MYSQL_ROOT_PASSWORD=password mysql

Now, create a wordpress container and link it with the mysql container:

$ docker run --name wordpress -d -p 80:80 --link mysql:mysql

And you are done. All arguments for wordpress, such as DB_HOST and ROOT_PASSWORD, will be taken from the linked mysql container.

The other option to set up WordPress is to set up both WordPress and MySQL in a single container. This needs process management tools such as supervisord to run two or more processes in a single container. Docker allows only one process per container by default.

See also

You can find the respective Dockerfiles for MySQL and WordPress containers at the following addresses:

Docker Hub WordPress: https://hub.docker.com/_/wordpress/

Docker Hub MySQL: https://hub.docker.com/_/mysql/

Docker networking: https://blog.docker.com/2015/11/docker-multi-host-networking-ga/

Networking for containers using libnetwork: https://github.com/docker/libnetwork