Resources

In this section, we will take a quick look at the latest Ubuntu Server release news and then, we will make a list of all the system requirements.

Now that we have OpenStack installed and have set our desired operating system image, we are ready to launch our first instance in a self-hosted cloud.

Getting ready

You will need credentials to access the OpenStack dashboard.

Uploading your own image is not necessary; you can use the default Cirros image to launch the test instance.

Log in to the OpenStack dashboard and set the SSH key pair in the Access & Security tab available under the Projects menu. Here, you can generate a new key pair or import your existing public key.

How to do it…

OpenStack instances are the same virtual machines that we launch from the command line or desktop tools. OpenStack give you a web interface to launch your virtual machines from. Follow these steps to create and start a new instance:

Select the Instance option under the Projects menu and then click on the Launch Instance button on the right-hand side. This should open a modal box with various options, which will look something like this:

Now, start filling in the necessary details. All fields that are marked with * are required fields. Let's start by naming our instance. Enter the name in the Instance Name field.

Set the value of Count to the number of instances you want to launch. We will leave it at the default value of 1.

Next, click on the Source tab. Here, we need to configure the source image for our instance. Set Select Boot Source to Image and select No for Create New Volume. Then, from the Available Images section, search the desired image and click on the button with the + sign to select the image. The list should contain our recently uploaded image. The final screen should look something like this:

Next, on the Flavor tab, we need to select the desired resources for our instance. Select the desired flavor by clicking on the + button. Make sure that the selected row does not contain any warning signs.

Now, from the Key Pair tab, select the SSH key pair that we just created. This is required to log in to your instance.

Finally, click on the Launch Instance button from the bottom of the modal box. A new instance should be created and listed under the instances list. It will take some time to start; wait for the Status column to show Active:

You are now ready to access your virtual instance. Log in to your host console and try to ping the IP address of your instance. Then, open an SSH session with the following command:

$ ssh -i your_key ubuntu@instance_ip

This should give you a shell inside your new cloud instance. Try to ping an external server, such as an OpenDNS server, from within an instance to ensure connectivity.

To make this instance available on your local network, you will need to assign a floating IP address to it. Click on the drop-down arrow from the Actions column and select Associate Floating IP. This should add one more IP address to your instance and make it available on your local network.

How it works…

OpenStack instances are the same as the virtual machines that we build and operate with common virtualization tools such as VirtualBox and Qemu. OpenStack provides a central console for deploying and managing thousands of such machines on multiple hosts. Under the hood, OpenStack uses the same virtualization tools as the others. The preferred hypervisor is KVM, and if hardware acceleration is not available, Qemu emulation is used. OpenStack supports various other hypervisors, including VMware, XEN, Hyper-V, and Docker. In addition, a lightervisor, LXD, is on its way to a stable release. Other than virtualization, OpenStack adds various other improvements, such as image management, block storage, object storage, and various network configurations.

In the previous example, we set various parameters before launching a new instance; these include the instance name, resource constraints, operating system image, and login credentials. All these parameters will be passed to the underlying hypervisor to create and start the new virtual machine. A few other options that we have not used are volumes and networks. As we have installed a very basic OpenStack instance, new developments in network configurations are not available for use. You can update your DevStack configuration and install the OpenStack networking component Neutron.

Volumes, on the other hand, are available and can be used to obtain disk images of the desired size and format. You can also attach multiple volumes to a single machine, providing extended storage capacity. Volumes can be created separately and do not depend on the instance. You can reuse an existing volume with a new instance, and all data stored on it will be available to the new instance.

Here, we have used a cloud image to start a new instance. You can also choose a previously stored instance snapshot, create a new volume, or use a volume snapshot. The volume can be a permanent volume, which has its life cycle separate from the instance, or an ephemeral volume, which gets deleted along with the instance. Volumes can also be attached at instance runtime or even removed from an instance, provided they are not a boot source.

Other options include configuration and metadata. The configuration tab provides an option to add initialization scripts that are executed at first boot. This is very similar to cloud-init data. The following is a short example of a cloud-init script:

#cloud-config

package_update: true

package_upgrade: true

password: password

chpasswd: { expire: False }

ssh_pwauth: True

ssh_authorized_keys:

- your-ssh-public-key-contents

This script will set a password for the default user (ubuntu in the case of Ubuntu images), enable password logins, add an SSH key to authorize keys, and update and upgrade packages.

The metadata section adds arbitrary data to instances in the form of key-value pairs. This data can be used to identify an instance from a group and automate certain tasks.

Once an instance has been started, you have various management options from the Actions menu available on the instance list. From this menu, you can create instance snapshots; start, stop, or pause instances; edit security groups; get the VNC console; and so on.

There's more…

Similar to the glance command-line client, a compute client is available as well and is named after the compute component. The nova command can be used to create and manage cloud instances from the command line. You can get detailed parameters and options with the nova help command or, to get help with a specific subcommand, nova help .

See also

The cloud-init official documentation: https://cloudinit.readthedocs.io/en/latest/

More on cloud-init: https://help.ubuntu.com/community/CloudInit

OpenStack instance guide: http://docs.openstack.org/user-guide/dashboard_launch_instances.html

Command-line cheat sheet: http://docs.openstack.org/user-guide/cli_cheat_sheet.html#compute-nova

In this recipe, we will look at basic CRUD operations with MongoDB. We will learn how to create databases, store, retrieve, and update stored data. This is a recipe to get started with MongoDB.

Getting ready

Make sure that you have installed and configured MongoDB. You can also use the MongoDB installation on a remote server.

How to do it…

Follow these steps to store and retrieve data with MongoDB:

Open a shell to interact with the Mongo server:

$ mongo

To open a shell on a remote server, use the command given. Replace server_ip and port with the respective values:

$ mongo server_ip:port/db

To create and start using a new database, type use dbname. Since schemas in MongoDB are dynamic, you do not need to create a database before using it:

> use testdb

You can type help in Mongo shell to get a list of available commands and help regarding a specific command:

> help: Let’s insert our first document:

> db.users.insert({‘name’:’ubuntu’,’uid’:1001})

To view the created database and collection, use the following commands:

> show dbs

> show collections

You can also insert multiple values for a key, for example, which groups a user belongs to:

> db.users.insert({‘name’:’root’,’uid’:1010, ‘gid’:[1010, 1000, 1111]})

Check whether a document is successfully inserted:

> db.users.find()

To get a single record, use findOne():

> db.users.findOne({uid:1010})

To update an existing record, use the update command as follows:

> db.users.update({name:’ubuntu’}, {$set:{uid:2222}})

To remove a record, use the remove command. This will remove all records with a name equal to ubuntu:

> db.users.remove({‘name’:’ubuntu’})

To drop an entire collection, use the drop() command:

> db.users.drop()

To drop a database, use the dropDatabase() command:

> db.users.dropDatabase()

How it works…

The preceding examples show very basic CRUD operations with the MongoDB shell interface. MongoDB shell is also a JavaScript shell. You can execute all JS commands in a MongoDB shell. You can also modify the shell with the configuration file, ~/.mongorc.js. Similar to shell, MongoDB provides language-specific drivers, for example, MongoDB PHP drivers to access MongoDB from PHP.

MongoDB works on the concept of collections and documents. A collection is similar to a table in MySQL and a document is a set of key value stores where a key is similar to a column in a MySQL table. MongoDB does not require any schema definitions and accepts any pair of keys and values in a document. Schemas are dynamically created. In addition, you do not need to explicitly create the collection. Simply type a collection name in a command and it will be created if it does not already exist. In the preceding example, users is a collection we used to store all data. To explicitly create a collection, use the following command:

> use testdb

> db.createCollection(‘users’)

You may be missing the where clause in MySQL queries. We have already used that with the findOne() command:

> db.users.findOne({uid:1010})

You can use $lt for less than, $lte for less than or equal to, $gt for greater than, $gte for greater than or equal to, and $ne for not equal:

> db.users.findOne({uid:{$gt:1000}})

In the preceding example, we have used the where clause with the equality condition uid=1010. You can add one more condition as follows:

> db.users.findOne({uid:1010, name:’root’})

To use the or condition, you need to modify the command as follows:

> db.users.find ({$or:[{name:’ubuntu’}, {name:’root’}]})

You can also extract a single key (column) from the entire document. The find command accepts a second optional parameter where you can specify a select criteria. You can use values 1 or 0. Use 1 to extract a specific key and 0 otherwise:

> db.users.findOne({uid:1010}, {name:1})

> db.users.findOne({uid:1010}, {name:0})

There’s more…

You can install a web interface to manage the MongoDB installation. There are various open source web interfaces listed on Mongo documentation at http://docs.mongodb.org/ecosystem/tools/administration-interfaces/ .

When you start a mongo shell for the first time, you may see a warning message regarding transperent_hugepage and defrag. To remove those warnings, add the following lines to /etc/init/mongod.conf, below the $DAEMONUSER /var/run/mongodb.pid line:

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then

echo never > /sys/kernel/mm/transparent_hugepage/enabled

fi

if test -f /sys/kernel/mm/transparent_hugepage/defrag; then

echo never > /sys/kernel/mm/transparent_hugepage/defrag

fi

Find more details on this Stack Overflow post at http://stackoverflow.com/questions/28911634/how-to-avoid-transparent-hugepage-defrag-warning-from-mongodb

See also

Mongo CRUD tutorial at https://docs.mongodb.org/manual/applications/crud/

MongoDB query documents at https://docs.mongodb.org/manual/tutorial/query-documents/

In this recipe, we will learn how to install and set up a well-known e-mail filtering program, spam-assassin.

Getting ready

You will need access to a root account or an account with sudo privileges.

You need to have Postfix installed and working.

How to do it…

Follow these steps to filter mail with spam-assassin:

Install spam-assassin with the following command:

$ sudo apt-get update

$ sudo apt-get install spamassassin spamc

Create a user account and group for spam-assassin:

$ sudo groupadd spamd

$ sudo useradd -g spamd -s /usr/bin/nologin \

-d /var/log/spamassassin -m spamd

Change the default settings for the spam daemon. Open /etc/default/spamassassin and update the following lines:

ENABLED=1

SAHOME="/var/log/spamassassin/"

OPTIONS="--create-prefs --max-children 5 --username spamd - -helper-home-dir ${SAHOME} -s ${SAHOME}spamd.log"

PIDFILE="${SAHOME}spamd.pid"

CRON=1

Optionally, configure spam rules by changing values in /etc/spamassassin/local.cf:

trusted_networks 10.0.2. # set your trusted network

required_score 3.0 # 3 + will be marked as spam

Next, we need to change the Postfix settings to pass e-mails through spam- assassin. Open /etc/postfix/master.cf and find the following line:

smtp inet n - - - - smtpd

Add the content filtering option:

-o content_filter=spamassassin

Define the content filter block by adding the following lines to the end of the file:

spamassassin unix - n n - - pipe

user=spamd argv=/usr/bin/spamc -f -e

/usr/sbin/sendmail -oi -f ${sender} ${recipient}

Finally, restart spam-assassin and Postfix:

$ sudo service spamassassin start

$ sudo service postfix reload

You can check spam-assassin and mail logs to verify that spam-assassin is working properly:

$ less /var/log/spamassassin/spamd.log

$ less /var/log/mail.log

How it works…

Spam filtering works with the help of a piping mechanism provided by Postfix. We have created a new Unix pipe which will be used to filter e-mails. Postfix will pass all e-mails through this pipe, which will be then scanned through spam-assassin to determine the spam score. If given e-mail scores below the configured threshold, then it passes the filter without any modification; otherwise, spam-assassin adds a spam header to the e-mail.

Spam-assassin works with a Bayesian classifier to classify e-mails as spam or not spam. Basically, it checks the content of the e-mail and determines the score based on content.

There's more…

You can train spam-assassin's Bayesian classifier to get more accurate spam detections.

The following command will train spam-assassin with spam contents (--spam):

$ sudo sa-learn --spam -u spamd --dir ~/Maildir/.Junk/* -D

To train with non-spam content, use the following command (--ham):

$ sudo sa-learn --ham -u spamd --dir ~/Maildir/.INBOX/* -D

If you are using the mbox format, replace --dir ~/Maildir/.Junk/* with the option --mbox.

See also

Sa-learn - train SpamAssassin's Bayesian classifier at https://spamassassin.apache.org/full/3.2.x/doc/sa-learn.html and https://wiki.apache.org/spamassassin/BayesInSpamAssassin

Learn about Bayesian classification at https://en.wikipedia.org/wiki/Naive_Bayes_classifier

OwnCloud is a self-hosted file storage and synchronization service. It provides client tools to upload and sync all your files to a central storage server. You can access all your data through a well-designed web interface, which can be accessed on any device of your choice. In addition to a simple contact service, OwnCloud supports contacts, email, and calendar synchronization. Plus, all your data is stored on your own server, making it a more secure option.

In this recipe, we will learn how to install the OwnCloud service on the Ubuntu server. We will be working with a basic OwnCloud setup that includes file sharing and storage. Later, you can add separate plugins to extend the capability of your OwnCloud installation.

Getting ready

You will need access to an account with sudo privileges.

How to do it…

OwnCloud is a PHP-based web application. Its dependencies include a web server, PHP runtime, and a database server. We will use the installation package provided by OwnCloud. The package takes care of all dependencies, plus it will help in updating our installation whenever a new version is available. We will install the latest stable version of OwnCloud. As of writing this, OwnCloud does not provide any packages for Ubuntu 16.04. I have used the package for Ubuntu 15.10:

Add the OwnCloud repository public key to your Ubuntu server:

$ wget https://download.owncloud.org/download/repositories/stable/Ubu ntu_15.10/Release.key -O owncloud.key

$ sudo apt-key add - owncloud.key

Next, add the OwnCloud repository to installation sources. Create a new source list:

$ sudo touch /etc/apt/sources.list.d/owncloud.list

Add an installation path to the newly created source list:

$ sudo nano /etc/apt/sources.list.d/owncloud.list

deb http://download.owncloud.org/download/repositories/stable/Ubun tu_15.10/ /

Update installation sources with the apt-get update command:

$ sudo apt-get update

Install the OwnCloud package. This will download and install all dependencies, download the OwnCloud package, and set up the Apache web server virtual host configuration. By default, OwnCloud use SQLite as a default database. This can be changed at the signup page:

$ sudo apt-get install owncloud

Once installed, you can access your OwnCloud installation at http://your_server/owncloud . This will open the registration page for an admin account. Enter the admin username and password for a new account. The first user to register will be marked as the admin of the OwnCloud instance.

The same page contains a warning saying the default database is SQLite. Click the configure database link; this will show you the option to enter database connection details. Enter all the required details and click submit.

Once registration completes, you will be redirected to the OwnCloud homepage. If you need any help, this page contains the OwnCloud user manual. You can start uploading content or create new text files right from the homepage.

Optionally, install OwnCloud desktop and mobile applications to sync files across all your devices.

How it works…

OwnCloud is a web application that enables you to synchronize and share files across the web. Store a backup of all your files on a central OwnCloud server, or use it as a central place to send and receive files. OwnCloud also provides native applications for all platforms so that you can easily replicate the necessary data across all your devices. Once you have logged in to your account, OwnCloud will list the default directory structure with a PDF file for the user manual. The screen should look similar to the following:

With the recent updates, OwnCloud has removed various default packages and reduced the overall binary size. For now, the default installation contains a file browser, an activity monitor, and a gallery. The file browser supports the uploading, viewing, and sharing of files. You can create new text files and open PDF files right from the browser:

Default features can be extended from the Apps submenu accessible from the Files link at the top, left of the screen. It gives you a list of installed and enabled or disabled apps. Plus, you can search for apps across categories such as Multimedia, Productivity, Games, and Tools. Choose your desired category, scroll to the desired app and click enable to install a new component:

OwnCloud also allows flexible user management. When logged in as an admin user, you can access the Users menu from the top-right login section of the screen. Under users, you can create a new user, assign them to a group, create a new group, and even set the disk quota allowed:

Next is the admin section, which is again accessible to users from the admin group at the top-right of the screen. This section lists all the administrative settings relating to the core OwnCloud setup, as well as for installed apps. Each section contains a link to detailed documentation. The important part of the settings is the email server setup. By default, OwnCloud uses default PHP-based emails. It is recommended you set up an SMTP service. You can use external SMTP service providers, such as MailChimp, or set up your own SMTP server. At the bottom of the admin settings page, you can see some links to improve your OwnCloud experience. This includes performance tuning the OwnCloud setup, security guidelines, theme support, and so on.

See also

OwnCloud repositories: https://download.owncloud.org/download/repositories/stable/owncloud/

OwnCloud admin manual: https://doc.owncloud.org/server/8.2/admin_manual/

A web server is a tool that publishes documents on a network, generally the Internet. HTTP is called a language of the Internet and web servers, apart from browsers, are native speakers of HTTP. Web servers generally listen on one or multiple ports for requests from clients and accept requests in the form of URLs and HTTP headers. On receiving a request, web servers look for the availability of the requested resource and return the contents to the client. The term web server can refer to one or multiple physical servers or a software package, or both of them working together.

Some well known web servers include the Apache web server, Microsoft IIS, and Nginx. Apache web server is the most popular web server package available across platforms such as Windows and Linux. It is an open source project and freely available for commercial use. Nginx, which is again an open source web server project, started to overcome the problems in a high-load environment. Because of its lightweight resource utilization and ability to scale even on minimal hardware, Nginx quickly became a well known name. Nginx offers a free community edition as well as a paid commercial version with added support and extra features. Lastly, Microsoft IIS is a web server specifically designed for Windows servers. Apache still has the major share in the web server market, with Nginx rapidly taking over with some other notable alternatives such as lighttpd and H2O.

Apache is a modularized web server that can be extended by dynamically loading extra modules as and when required. This provides the flexibility to run a bare minimum web server or a fully featured box with modules to support compression, SSL, redirects, language modules, and more. Apache provides multiple connection processing algorithms called multi-processing modules (MPM). It provides an option to create a separate single threaded process for each new request (mpm_prefork), a multi-threaded process that can handle multiple concurrent requests (mpm_worker), or the latest development of mpm_event, which separates the active and idle connections.

Nginx can be considered the next generation of web servers. Its development started to solve the C10k problem, that is, handling ten thousand connections at a time. Apache, being a process-driven model, has some limitations when handling multiple concurrent connections. Nginx took advantage of the event-driven approach with asynchronous, non-blocking connection handling algorithms. A new connection request is handled by a worker process and placed in an event loop where they are continuously checked for events. The events are processed asynchronously. This approach enables Nginx to run with a much lower memory footprint and lower CPU use. It also eliminates the overload of starting a new process for a new connection. A single worker process started by Nginx can handle thousands of concurrent connections.

Both Apache and Nginx can be configured to process dynamic contents. Apache provides respective language processors such as mod_php and mod_python to process dynamic contents within the worker process itself. Nginx depends on external processors and uses CGI protocols to communicate with external processors. Apache can also be configured to use an external language processor over CGI, but the choice depends on performance and security considerations.

While both Apache and Nginx provide various similar features, they are not entirely interchangeable. Each one has its own pros and cons. Where Nginx excels at serving static contents, Apache performs much better processing dynamic contents. Many web administrators prefer to use Apache and Nginx together.

Nginx handles a large number of connections and passes limited requests of dynamic contents to backend Apache servers. This configuration also allows users to scale horizontally by adding multiple backend servers and setting Nginx as a load balancer.

In this article, we will be working with both Apache and Nginx servers. We will learn how to set up Apache with PHP as a language for dynamic contents. We will look at some important configurations of Apache. Later, we will set up Nginx with an optional PHP processor, PHP_FPM, and configure Nginx to work as a reverse proxy and load balancer. We will also look at performance and security configurations for both the servers.

In this article, we will cover the following recipes:

• Installing Git
• Creating a local repository with Git CLI
• Storing file revisions with Git commit
• Synchronizing the repository with a remote server
• Receiving updates with Git pull
• Creating repository clones
• Installing GitLab, your own Git hosting
• Adding users to the GitLab server
• Creating a repository with GitLab
• Automating common tasks with Git hooks

In this recipe, we will look at ways to make user profiles more secure.

How to do it...

Follow these steps to secure the user account:

Set a strong password policy with the following steps:

Open the /etc/pam.d/common-password file with GNU nano:

$ sudo nano /etc/pam.d/common-password

Find the line similar to this:

password [success=1 default=ignore] pam_unix.so obscure sha512

Add minlen to the end of this line:

password [success=1 default=ignore] pam_unix.so obscure sha512 minlen=8

Add this line to enforce alphanumeric passwords:

password requisite pam_cracklib.so ucredit=-1 lcredit=-1 dcredit=-1 ocredit=-1

Save changes and exit GNU nano editor.

Press Ctrl + O to save changes.

Press Ctrl + X to exit GNU nano editor.

Secure the home directory with the following steps:

Check home directory permissions with the following command:

$ ls -ld /home/username

Restrict permissions to user and group with the following command:

$ chmod 750 /home/username

Change adduser default permissions by editing /etc/adduser.conf. Find DIR_MODE=0755 and change it to DIR_MODE=0750.

Disable SSH access to root user with the following step:

Open /etc/ssh/sshd_config and add or edit PermitRootLogin to PermitRootLogin no

Disable password authentication with the following step:

Open /etc/ssh/sshd_config and add or edit PasswordAuthentication no

Install fail2ban with sudo apt-get install fail2ban.

How it works…

This recipe discussed a few important steps to make user accounts more secure.

A password is the most important aspect in securing user accounts. A weak password can be easily broken with brute force attacks and dictionary attacks. It is always a good idea to avoid password-based authentication, but if you are still using it, then make sure you enforce a strong password policy.

Password authentication is controlled by the PAM module pam_unix, and all settings associated with login are listed at /etc/pam.d/login. An additional configuration file /etc/pam.d/common-password includes values that control password checks.

The following line in the primary block of common-password file defines the rules for password complexity:

password [success=1 default=ignore] pam_unix.so obscure sha512

The default setting already defines some basic rules on passwords. The parameter obscure defines some extra checks on password strength. It includes the following:

Palindrome check

Case change only

Similar check

Rotated check

The other parameter, sha512, states that the new password will be encrypted with the sha512 algorithm. We have set another option, minlen=8, on the same line, adding minimum length complexity to passwords.

Additionally, we have set alphanumeric checks for new passwords with the PAM module pam_cracklib:

password requisite pam_cracklib.so ucredit=-1 lcredit=-1 dcredit=- 1 ocredit=-1

The preceding line adds requirement of one uppercase letter, one lowercase letter, one digit (dcredit), and one special character (ocredit)

There are other PAM modules available, and you can search them with the following command:

$ apt-cache search limpam-

You might also want to secure the home directory of users. The default permissions on Ubuntu allow read and execute access to everyone. You can limit the access on the home directory by changing permission on the home directory as required. In the preceding example, we changed permissions to 750. This allows full access to the user, and allows read and execute access to the user's primary group.

You can also change the default permissions on the user's home directory by changing settings for the adduser command. These values are located at /etc/adduser.conf. We have changed default permissions to 750, which limits access to the user and the group only.

Additionally, you can disable remote login for the root account as well as disable password-based authentication. Public key authentication is always more secure than passwords, unless you can secure your private keys. Before disabling password authentication, ensure that you have properly enabled public key authentication and you are able to log in with your keys. Otherwise, you will lock yourself out of the server.

You might want to install a tool like fail2ban to watch and block repeated failed actions. It scans through access logs and automatically blocks repeated failed login attempts. This can be a handy tool to provide a security against brute force attacks.

In this recipe, we will learn to use a Docker network to set up a WordPress server. We will create two containers, one for MySQL and the other for WordPress. Additionally, we will set up a private network for both MySQL and WordPress.

How to do it…

Let's start by creating a separate network for WordPress and the MySQL containers:

A new network can be created with the following command:

$ docker network create wpnet

Check whether the network has been created successfully with docker network ls:

$ docker network ls

You can get details of the new network with the docker network inspect command:

$ docker network inspect wpnet

Next, start a new MySQL container and set it to use wpnet:

$ docker run --name mysql -d \

-e MYSQL_ROOT_PASSWORD=password \

--net wpnet mysql

Now, create a container for WordPress. Make sure the WORDPRESS_DB_HOST argument matches the name given to the MySQL container:

$ docker run --name wordpress -d -p 80:80 \

--net wpnet\

-e WORDPRESS_DB_HOST=mysql\

-e WORDPRESS_DB_PASSWORD=password wordpress

Inspect wpnet again. This time, it should list two containers:

Now, you can access the WordPress installation at your host domain name or IP address.

How it works…

Docker introduced the container networking model (CNM) with Docker version 1.9. CNM enables users to create small, private networks for a group of containers. Now, you can set up a new software-assisted network with a simple docker network create command. The Docker network supports bridge and overlay drivers for networks out of the box. You can use plugins to add other network drivers. The bridge network is a default driver used by a Docker network. It provides a network similar to the default Docker network, whereas an overlay network enables multihost networking for Docker clusters.

This recipe covers the use of a bridge network for wordpress containers. We have created a simple, isolated bridge network using the docker network command. Once the network has been created, you can set containers to use this network with the --net flag to docker run command. If your containers are already running, you can add a new network interface to them with the docker network connect command, as follows:

$ # docker network connect network_name container_name

$ docker network connect wpnet mysql

Similarly, you can use docker network disconnect to disconnect or remove a container from a specific network. Additionally, this network provides an inbuilt discovery feature. With discovery enabled, we can communicate with other containers using their names. We used this feature while connecting the MySQL container to the wordpress container. For the WORDPRESS_DB_HOST parameter, we used the container name rather than the IP address or FQDN.

If you've noticed, we have not mentioned any port mapping for the mysql container. With this new wpnet network, we need not create any port mapping on the MySQL container. The default MySQL port is exposed by the mysql container and the service is accessible only to containers running on the wpnet network. The only port available to the outside world is port 80 from the wordpress container. We can easily hide the WordPress service behind a load balancer and use multiple wordpress containers with just the load balancer exposed to the outside world.

There's more…

Docker also supports links to create secure communication links between two or more containers. You can set up a WordPress site using linked containers as follows:

First, create a mysql container:

$ docker run --name mysql -d \

-e MYSQL_ROOT_PASSWORD=password mysql

Now, create a wordpress container and link it with the mysql container:

$ docker run --name wordpress -d -p 80:80 --link mysql:mysql

And you are done. All arguments for wordpress, such as DB_HOST and ROOT_PASSWORD, will be taken from the linked mysql container.

The other option to set up WordPress is to set up both WordPress and MySQL in a single container. This needs process management tools such as supervisord to run two or more processes in a single container. Docker allows only one process per container by default.

See also

You can find the respective Dockerfiles for MySQL and WordPress containers at the following addresses:

Docker Hub WordPress: https://hub.docker.com/_/wordpress/

Docker Hub MySQL: https://hub.docker.com/_/mysql/

Docker networking: https://blog.docker.com/2015/11/docker-multi-host-networking-ga/

Networking for containers using libnetwork: https://github.com/docker/libnetwork

This article will cover how to install Ubuntu Server by using different methods. We will have a look at how to install Ubuntu Server in the manual and automated mode by using a simple or an advanced installation.

To reach this goal, we will follow this plan:

• Preparation before the installation
• The manual installation
• The automated installation

In the previous recipe, we installed and configured OpenStack. Now, to start using the service, we need to upload virtual machine images. The OpenStack installation uploads a test image named Cirros. This is a small Linux distribution designed to be used as a test image in the cloud. We will upload prebuilt cloud images available from Ubuntu.

Getting ready

Make sure you have installed the OpenStack environment and you can access the OpenStack dashboard with valid credentials. It is not necessary to have an admin account to create and upload images.

Select the cloud image of your choice and get its download URL. Here, we will use the Trusty Ubuntu Server image. The selected image format is QCOW2, though OpenStack support various other image formats. The following is the URL for the selected image:

https://cloud-images.ubuntu.com/trusty/current/trusty-server- cloudimg-amd64-disk1.img

How to do it…

The OpenStack dashboard provides a separate section for image management. You can see the images that are already available and add or remove your own images. Follow these steps to create your own image:

Log in to your OpenStack dashboard. On successful login, you should get an Overview page for your account.

Now, from the left-hand side Project menu, under the Compute submenu, click on Images:

This should show you a list of all publicly available images—something like this:

Click on the Create Image button to add a new image. This should open a popup box with various details. Here, you can choose to add an image URL or enter an image path if you have downloaded the image to your local machine.

Fill in the name and other required details. Under Image Source, select the image location, and in the next box, Image Location, enter the URL for the Ubuntu Cloud image.

Under Format, select the image format of your selected image. In this case, it's QCOW2.

Enter amd64 under Architecture. Make sure you match this with your selected image.

Enter the minimum disk and RAM size. As we have selected an Ubuntu image, the minimum disk size should be 5 GB and minimum RAM 256 MB. These values will affect the selection of instance flavors while creating a new instance.

Finally, click on the Create Image button to save the details and add the image to OpenStack. This will download the image from the source URL and save it in the image repository. The resulting image will be listed under the Project tab, as follows:

Now, the image is ready can be used to launch new cloud instances.

How it works…

OpenStack is a cloud virtualization platform and needs operating system images to launch virtual machines in the cloud. The Glance OpenStack imaging service provides the image-management service. It supports various types of image, including Qemu format, raw disk files, ISO images, and images from other virtualization platforms, as well as Docker images. Like every other thing in OpenStack, image management works with the help of APIs provided by Glance.

OpenStack, being a cloud platform, is expected to have ready-to-use images that can be used to quickly start a virtual instance. It is possible to upload the operating system installation disk and install the OS to a virtual instance, but that would be a waste of resources. Instead, it is preferable to have prebuilt cloud images. Various popular operating systems provide their respective cloud images, which can be imported to cloud systems. In the previous example, we used the Ubuntu Cloud image for the Ubuntu Trusty release.

We imported the image by specifying its source URI. Local image files can also be uploaded by selecting the image file as an image source. You can also build your own images and upload them to the image store to be used in the cloud. Along with the image source, we need to provide a few more parameters, which include the type of the image being uploaded and the minimum resource requirements of that image. Once the image has been uploaded, it can be used to launch a new instance in the cloud. Also, the image can be marked as public so that it is accessible to all OpenStack users. You will need specific rights for your OpenStack account to create public images.

There's more…

OpenStack images can also be managed from the command line with the client called glance. To access the respective APIs from the command line, you need to authenticate with the Glance server. Use the following steps to use glance from the command line:

First, add authentication parameters to the environment:

export OS_USERNAME=demo

export OS_PASSWORD=password

export OS_AUTH_URL=http://10.196.69.158/identity

export OS_TENANT_ID=8fe52bb13ca44981aa15d9b62e9133f4

Now, use the following command to obtain the image list for the specified user:

$ glance image-list

You can get a list of available command-line options with glance help.

See also

Read more about OpenStack image management: http://docs.openstack.org/image-guide/

Command-line image management: http://docs.openstack.org/user- guide/common/cli_manage_images.html

Dashboard image management: http://docs.openstack.org/user- guide/dashboard_manage_images.html

Glance documentation: http://docs.openstack.org/developer/glance/

Until now, we have worked with the relational database server, MySQL. In this recipe, we will learn how to install and configure MongoDB, which is a not only SQL (NoSQL) document storage server.

Getting ready

You will need access to a root account or an account with sudo privileges.

How to do it…

To get the latest version of MongoDB, we need to add the MongoDB source to Ubuntu installation sources:

First, import the MongoDB GPG public key:

$ sudo apt-key adv \

--keyserver hkp://keyserver.ubuntu.com:80 \

--recv 7F0CEB10

Create a list file and add an install source to it:

$ echo “deb http://repo.mongodb.org/apt/ubuntu “$(lsb_release -sc)”/mongodb-org/3.0 multiverse” | sudo tee /etc/apt/sources.list.d/mongodb-org-3.0.list

Update the apt repository sources and install the MongoDB server:

$ sudo apt-get update

$ sudo apt-get install -y mongodb-org

After installation completes, check the status of the MongoDB server:

$ sudo service mongod status

Now you can start using the MongoDB server. To access the Mongo shell, use the following command:

$ mongo

How it works…

We have installed the MongoDB server from the MongoDB official repository. The Ubuntu package repository includes the MongoDB package in it, but it is not up to date with the latest release of MongoDB. With GPG keys, Ubuntu ensures the authenticity of the packages being installed. After importing the GPG key, we have created a list file that contains the installation source of the MongoDB server.

After installation, the MongoDB service should start automatically. You can check logs at /var/log/mongodb/mongod.log.

See also

MongoDB installation guide at http://docs.mongodb.org/manual/tutorial/install-mongodb-on-ubuntu/

In this recipe, we will learn how to add e-mail accounts to Postfix. The easiest way to add a new e-mail account to Postfix is to add a new user account on your server. Postfix will check for user accounts and deliver e-mails to respective users. We will create a virtual user setup so that we do not need to create user accounts for each e-mail user.

Getting ready

You will need access to a root account or an account with sudo privileges.

I assume that you have completed your basic Postfix setup and that it is working properly.

How to do it…

Follow these steps to add e-mail account:

Create a new user account:

$ useradd -s /usr/bin/nologin -m vmail

Get the UID and GID for this account:

$ grep vmail /etc/passwd

vmail:x:1001:1001::/home/vmail:/usr/bin/nologin

Create a base directory layout for domains and users:

$ sudo mkdir -p /home/vmail/example.org/bob

$ sudo mkdir -p /home/vmail/example.net/alice

Allow only the user vmail to access these files:

$ sudo chown -R vmail:vmail /home/vmail

$ chmod -R 700 /home/vmail

Next, configure Postfix. Edit /etc/postfix/main.cf and add the following lines:

virtual_mailbox_base = /home/vmail

virtual_mailbox_domains = /etc/postfix/virtual_domains

virtual_mailbox_maps = hash:/etc/postfix/virtual_maps

virtual_alias_maps = hash:/etc/postfix/virtual_alias

virtual_uid_maps = static:1001 # user ID for user vmail

virtual_gid_maps = static:1001 # group ID for user vmail

Create the file virtual_domains under /etc/postfix:

$ sudo nano /etc/postfix/virtual_domains

example.org

example.net

Create the virtual_maps file:

$ sudo nano /etc/postfix/virtual_maps

bob@example.org example.org/bob/

alice@example.org example.org/alice/

@example.org example.org/catchall/ # catch all address

Create the virtual_alias file and optionally set a redirect:

$ sudo nano /etc/postfix/virtual_alias

# redirect emails for tim to bob

tim@example.org bob@example.org

Now generate database of virtual maps and aliases by hashing respective files:

$ sudo postmap /etc/postfix/virtual_maps

$ sudo postmap /etc/postfix/virtual_alias

Reload Postfix and send an e-mail to the newly created address:

$ sudo postfix reload

$ sendmail bob@example.org

How it works…

Here, we have created a virtual mailbox setup to enable our Postfix server to serve multiple domains as well as add e-mail users without creating user accounts on the server. All e-mails received by virtual users will be stored under the home directory of the vmail user (virtual_mailbox_base in Postfix configuration). When you need to add a new e-mail account, simply add the e-mail address with its respective domain to the virtual_maps file. In case you need to support a new domain, you can easily add it to the virtual_domains file.

The third file we used is virtual_alias. You can set e-mail forwarding in this file. It is handy when you need to create a new alias for an e-mail address or forward e-mails to one or multiple accounts. We have set a catchall entry in the virtual_alias file; this setting will redirect all e-mails received on nonexistent accounts to catchall@example.org, which can be checked by the domain administrator.

There's more…

Using files for virtual users and domains is good for getting started with setup. But once you need to add more and more user accounts and domains it is a good idea to move the users and domains to a database server. This can be easily done by changing the lookup table type. Postfix supports a variety of lookup table types, which include LDAP, MySQL, PGSQL, memcache, SQLite, and many others.

To use MySQL as a backend database, complete the following steps:

Create respective tables for virtual_domain, virtual_maps, and virtual_alias.

Change the Postfix configuration to use MySQL as a lookup table:

virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-domains

virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-maps

virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias

Add the respective details to each file using the following commands:

$ sudo nano /etc/postfix/mysql-virtual-domains

user = mysql_user

password = mysql_password

hosts = 127.0.0.1

dbname = mysql_db_name

query = SELECT 1 FROM virtual_domains WHERE name='%s'

$ sudo nano /etc/postfix/mysql-virtual-maps

...

query = SELECT 1 FROM virtual_users WHERE email='%s'

$ sudo nano /etc/postfix/mysql-virtual-alias

...

query = SELECT destination FROM virtual_aliases WHERE source='%s'

You can test your mapping with the following command. This should output 1 as a result:

$ postmap -q bob@example.org mysql:/etc/postfix/mysql-virtual-maps

Finally, restart the Postfix daemon.

Web console for virtual mailbox administration

The Vimbadmin package provides a web console for virtual mailbox administration. It is a PHP-based open source package. You can get source code and installation instructions at https://github.com/opensolutions/ViMbAdmin .

See also

Postfix guide at http://www.postfix.org/VIRTUAL_README.html

Postfix lookup table types at http://www.postfix.org/DATABASE_README.html#types

This recipe covers another open source collaboration tool, Mattermost. Mattermost is a modern communication tool that includes one-to-one chat, group chat IRC-like channels, file sharing, and a super-fast search functionality. It can be thought of as a modern IRC tool. Mattermost is well known as an open source Slack alternative, but the Mattermost website says it is not limited to being a Slack alternative. You can find a list of features at http://www.mattermost.org/features .

The GitHub repository contains a step-by-step guide for installing Mattermost on production servers. We will use the same guide as our base.

Getting ready

You will need a 64-bit Ubuntu server and access to an account with sudo privileges. Mattermost prebuilt binaries are available only on a 64-bit platform. If you are running 32-bit Ubuntu, you will need to compile Mattermost from source. We will use MySQL as a database for Mattermost. I will use the same server for the database and Mattermost. You may want to separate these services on two different servers for better performance.

Create a separate MySQL user account and database for Mattermost. I will use the lowercase mattermost as a database name as well as a username.

Additionally, you will need a proxy if you are planning to load balance multiple Mattermost instances or have a secure setup with SSL enabled.

You will need a separate storage directory for shared multimedia contents. You should use a separate large volume specifically assigned for this purpose. Make sure that the directory is owned by the current user. To keep things simple, I will use a data directory under the current user's home, that is, /home/ubuntu/mattermost-data.

How to do it…

Mattermost is based on Golang as a backend and React, a JavaScript framework, for the frontend. Golang is capable of creating self-sufficient independent binaries. We will download the prebuilt package available on GitHub. As of writing this, the latest stable version is 1.3.0:

Download the Mattermost archive with the following command:

$ wget https://github.com/mattermost/platform/releases/download/v1.3. 0/mattermost.tar.gz

Extract content from the archive. This will create a new directory named mattermost:

$ tar -xf mattermost.tar.gz

Next, edit the Mattermost configuration file located under the config directory:

$ cd mattermost

$ vi config/config.json

It is already configured to use MySQL as a data source. We need to set our username and password details for the database. Search for the SqlSettings section and replace the content of the DataSource parameter with the following line:

"DataSource": "mattermost: password@tcp(localhost:3306)/mattermost?charset=utf8mb4,utf 8"

Next, search for the FileSettings section and set the Directory parameter to the directory we created for multimedia content:

"Directory":"/home/ubuntu/mattermost-data/"

Now, run the Mattermost server with the following command, and wait for the server to start listening:

$./bin/platform

Now you can access the Mattermost service at the hostname of your server at http://server_ip_or_host:8065 . However, the service is still running from the console and will be terminated when we close the terminal.

Terminate this process by pressing Ctrl + C and set a startup daemon so that we can start Mattermost in the backend and automatically start the service on system reboot.

Create a new upstart configuration under the /etc/init directory:

$ sudo nano /etc/init/mattermost.conf

Add the following content to the newly created file:

start on runlevel [2345]

stop on runlevel [016]

respawn

chdir /home/ubuntu/mattermost

setuid ubuntu

exec bin/platform

Now you can start Mattermost with any of the following commands:

$ sudo start mattermost

Or

$ sudo service mattermost start

Optionally, if you want to load balance the Mattermost service using Nginx or HAProxy in front of it, please refer to article 3, Working with Web Servers, for detail on how to do so. The use of a load balancer will also give you an option to enable SSL security for all communication.

Once you start the Mattermost service and access the homepage, you will be asked to sign up. Create an account with an email address and you can start using your own Mattermost instance. You can access the server at http://yourserver:8065.

How it works…

Mattermost is all about team communication and collaboration. When you access the Mattermost server for the first time and sign up with your email address, you will get an option to create a new team or join existing teams.:

To join an existing team, you need to submit your email address and Mattermost will reply with links to the team page where you are a member. If you have not yet created a team, simply proceed with signup. On signup, after you have entered your email address, you will be asked to select a team name and URI or a web address for your team page. Enter a good name for your team and click Next:

On the next page, you will be asked to choose a URL for your team page. The box should be pre-filled with a suggested URL. Feel free to change it if you have a better idea:

Once you are done with signup, you will be greeted with a welcome message and a simple walkthrough of the Mattermost service. Once you are done with the introduction, you will land on the Town Square channel. This is a prebuilt public channel accessible to all users. There's one more prebuilt channel named Off-Topic listed on the left side menu. You can create your own public channel, create a Private Group, or have a one-to-one chat through Direct Messages.

Before you start using the service, invite some more users to your team. Click on the Invite others to this team link or click on your username at the top left and then select the Invite New Member link. Here, you can enter the email and name of a single member to invite them. Optionally, you can get a team invite link, which can be shared with a group:

The username menu on the left gives you some more options. You can update team settings, manage team members, and even create a new team altogether. You will need to be a team admin to access these options. If you are part of multiple teams, then you can see an option to switch to a different team.

The team members will receive all communication in public channels. A user can decide to be a part of a channel or leave it and not receive any communication from a specific channel. Other options are Private group and Direct messages. In private groups, you can communicate and share with selected people and not the entire team, whereas in a direct message, as the name suggests, it is a one-to-one chat.

Up to now, this article has covered XMPP and its usages. It is a good, mature protocol with multiple servers developed around it. Sometimes, however, you may need to set up a quick application that uses a simple message transfer, or develop a small chat application for your team. For such projects, XMPP servers may turn out to be overkill. You may not use all the features of XMPP and waste resources, even for a basic setup. Plus, developing an XMPP application is a time consuming process.

In this case, you can quickly start using Node.js-based socket communication. Node.js has gained popularity in the developer community. It is a framework developed in a commonly known language, JavaScript. In this recipe, we will learn how to develop a message passing application using Node.js sockets. We will use Socket.io, a popular Node.js library, to work with sockets and a demo app provided by Socket.io.

Getting ready

You will need access to a root account or an account with sudo privileges.

How to do it…

We are going to set up a Node.js-based application, so we need to install Node.js on our Ubuntu server.

Follow these steps to install Node.js:

Install Node.js with the following command:

$ sudo apt-get update

$ sudo apt-get install nodejs

Optionally, check your Node.js version:

$ node -v

Next, download the sample application from the Socket.io GitHub repo:

$ wget https://github.com/rauchg/chat- example/archive/master.zip

Unzip the downloaded contents. This will create a new directory named chat-sample-master:

$ unzip master.zip

Change the path to the newly created directory:

$ cd chat-sample-master

Next, we will need to install the dependencies for this sample application. Use the following Node.js command to install all dependencies.

$ npm install

This will fetch all dependencies and install them in the node_modules directory under chat-sample-master. Once the install command completes, you can start your application with the following command:

$ node index.js

ubuntu: ~/chat-example-master $ node index.js listening on *:3000

This will start an inbuilt HTTP server and set it to listen on default port 3000. Now you can access the app at http://server-ip:3000. The screen will look similar to the following image:

Open another instance in a separate browser window and start sending your messages.

How it works…

We have set up a very simple application that listens on a given Node.js socket. To send a message, we have used the socket.emit() function, which writes the data from text box to socket:

$('form').submit(function(){

socket.emit('chat message', $('#m').val());

...

});

When this message is received on the server side, the server writes it to all connected sockets, resulting in a group chat scenario:

io.on('connection', function(socket){

socket.on('chat message', function(msg){

io.emit('chat message', msg);

});

});

Similarly, to receive a message, we keep listening on the socket, and when an event chat message happens, we write the received data to an HTML page as a message:

socket.on('chat message', function(msg){

$('#messages').append($('

});

This is very basic application and can be extended easily to implement one-on-one chat. All we need is a unique ID for all clients and a little modification to the interface to separate messages. Right now, the message is sent as it is; you can collect the message and create a JSON object to contain sender and receiver IDs, plus any additional information.

The advantage of using NodeJS is quick and easy development. JavaScript is a commonly used language and you can easily get support from the large community. You can always develop the application as per your requirements. The disadvantage is regarding scaling; you will need to code the clustering mechanism on your own, whereas for XMPP, clustering is implemented by nearly all leading servers.

There's more…

The Node.js setup available with the Ubuntu repository is not the latest one. You can download the latest version from the node official download page.

Download NodeJS binaries for Linux. Choose your desired version by visiting the NodeJS download page. As of writing this, the latest stable version is 5.1:

$ wget https://nodejs.org/download/release/v5.1.0/node-v5.1.0-linux- x64.tar.xz

Extract binaries and move it to /use/local so that it is accessible globally:

$ tar Jxv --strip=1 -C /usr/local/

Check the node version with the following command:

$ node -v

See also

Node.js download page: https://nodejs.org/en/download

Node: how to install: https://github.com/nodejs/help/issues/41

Sample chat application on GitHub: https://github.com/rauchg/chat-example

In this recipe, you will see how to set up secure public key authentication.

Getting ready

You might need root privileges for certain tasks.

How to do it...

Follow these steps to set up public key authentication:

Add a new user. You can skip this step if you have already created a user:

$sudo adduser john

Log in as john and change to the home directory with cd ~/:

Create a .ssh directory if it doesn't already exist:

$ mkdir .ssh

Create a file named authorized_keys under the .ssh directory:

$ touch .ssh/authorized_keys

Set permissions on the .ssh directory to 700:

$chmod 700 .ssh

Set permissions for authorized_keys to 600:

$ chmod 600 .ssh/authorized_keys

Generate public key pair on your local system with the following command:

$ ssh-keygen

Copy the generated public key from the .ssh/id_rsa.pub file to the authorized_keys file on the server.

Now, open an ssh connection from local to server with the following command:

$ ssh john@server

If asked for confirmation, type yes and press the Enter key to continue:

How it works…

Logging in with SSH supports different authentication methods. Public key authentication and password-based authentication are two common methods. To log in with public key authentication, we need a public private key pair. We generate this key pair with the ssh-keygen command. This command creates two files under the .ssh directory in the user's home:

id_rsa: This is the private key file

id_rsa.pub: This is the public key file

You can view the contents of the files with $cat id_rsa.pub. It should start with something like ssh-rsa AAAA...(except for the trailing dots).

We then copy the contents of public key to the server's authorized_keys file. Ensure that all contents are listed on single line in the authorized_keys file.

Also, ensure the permissions are properly set for the .ssh directory, and ensure that the authorized_keys file and directory are owned by the user. The permissions for the .ssh directory limits read, write, and execute permissions to the owner of the file. Similarly, for authorized_keys file, permissions are limited to read and write for owner only. This ensures that no other user can modify the data in the .ssh directory. If these permissions are not properly set, the SSH daemon will raise the warning Permission denied?.

Working of SSH authentication

When the SSH client initiates a connection with the server, the server sends public key identification of server to client. If a client is connecting to the server for the first time, it shows a warning and asks for user confirmation to store the server key in the known_hosts file under the .shh directory. After receiving the identity, the client authenticates server to ensure that it is really the intended server.

After server authentication, the server sends a list of possible authentication methods. The client selects the authentication method and selection to the server. After receiving the authentication method, the server sends a challenge string encrypted with client's private key. The client has to decrypt this string and send it back to server along with previously shared session key. If the response from the client matches the response generated by the server, then client authentication succeeds.

There's more…

You might be searching for a secure option to install key on server. Here's one way!

If your local system has the ssh-copy-id tool installed, you can directly add your public key to the server's authorized_keys file with a single command:

$ ssh-copy-id john@serverdomain

After providing the password, your local public key will be added to the authorized_keys file under the .ssh directory of the user john.

Troubleshooting SSH connections

Most of the connection issues are related with configuration problems. If you happen to face any such issue, read the error message in detail. It is descriptive enough to understand the mistake. You can also go through following checklist:

Check if the SSH daemon is running. Check the port in use and port conflicts, if any

Check whether the firewall configuration allows SSH ports

Check the list of configuration methods that are enabled

Check permissions for your private keys on your local system

Check authorized_keys file for your public key on the server

Check for any entry with the old address of the server in known_hosts on the local system

Additionally, you can use the verbose flag (-v or -vvv) with the ssh command to get details of every step taken by the SSH client. Also, check SSH daemon logs on server.

SSH tools for the Windows platform

If your local system runs Windows, then you can use tools provided by puTTYto generate new keys and connect to the server:

putty.exe: This is the SSH client on Windows

puttygen.exe: This tool generates public or private keys

pscp.exe: This is the SCP client for secure file transfer

When using public key generated by the puttygen.exe tool, make sure that you convert the key to OpenSSH key format. Remove all comments and prepend ssh-rsa. Additionally, the entire key should be listed on a single line.

Another easy option is to use puttygen.exe. Load your private key in PuTTYgen and then copy the public key from the Key section of the PuTTYgen window.

See also

For more information on the full working of SSH authentication, visit http://www.slashroot.in/secure-shell-how-does-ssh-work

One of the most common questions seen on Docker forums is how to separate data from containers. This is because any data created inside containers is lost when the container gets deleted. Using docker commit to store data inside Docker images is not a good idea. To solve this problem, Docker provides an option called data volumes. Data volumes are special shared directories that can be used by one or more Docker containers. These volumes persist even when the container is deleted. These directories are created on the host file system, usually under the /var/lib/docker/ directory.

In this recipe, we will learn to use Docker volumes, share host directories with Docker containers, and learn basic backup and restore tricks that can be used with containers.

Getting ready

Make sure that you have the Docker daemon installed and running. We will need two or more containers.

You may need sudo privileges to access the /var/lib/docker directory.

How to do it…

Follow these steps to understand Docker volumes:

To add a data volume to a container, use the -v flag with the docker run command, like so:

$ docker run -dP -v /var/lib/mysql --name mysql\

-e MYSQL_ROOT_PASSWORD= passwdmysql:latest

This will create a new MySQL container with a volume created at /var/lib/mysql inside the container. If the directory already exists on the volume path, the volume will overlay the directory contents.

Once the container has been started, you can get the host-specific path of the volume with the docker inspect command. Look for the Mounts section in the output of docker inspect:

$ docker inspect mysql

To mount a specific directory from the host system as a data volume, use the following syntax:

$ mkdir ~/mkdir

$ docker run -dP -v ~/mysql:/var/lib/mysql \

--name mysql mysql:latest

This will create a new directory named mysql at the home path and mount it as a volume inside a container at /var/lib/mysql.

To share a volume between multiple containers, you can use named volume containers.

First, create a container with a volume attached to it. The following command will create a container with its name set to mysql:

$ docker run -dP -v /var/lib/mysql --name mysql\

-e MYSQL_ROOT_PASSWORD= passwd mysql:latest

Now, create a new container using the volume exposed by the mysql container and list all the files available in the container:

$ docker run --rm --volumes-from mysql ubuntu ls -l /var/lib/mysql

To back up data from the mysql container, use the following command:

$ docker run --rm--volumes-from mysql -v ~/backup:/backup \

$ tar cvf /backup/mysql.tar /var/lib/mysql

Docker volumes are not deleted when containers are removed. To delete volumes along with a container, you need to use the -v flag with the docker rm command:

$ dockerrm -v mysql

How it works…

Docker volumes are designed to provide persistent storage, separate from the containers' life cycles. Even if the container gets deleted, the volume still persists unless it's explicitly specified to delete the volume with the container. Volumes can be attached while creating a container using the docker create or docker run commands. Both commands support the -v flag, which accepts volume arguments. You can add multiple volumes by repeatedly using the volume flag. Volumes can also be created in a Dockerfile using the VOLUME instruction.

When the -v flag is followed by a simple directory path, Docker creates a new directory inside a container as a data volume. This data volume will be mapped to a directory on the host filesystem under the /var/lib/docker directory. Docker volumes are read-write enabled by default, but you can mark a volume to be read-only using the following syntax:

$ docker run -dP -v /var/lib/mysql:ro --name mysql mysql:latest

Once a container has been created, you can get the details of all the volumes used by it, as well as its host-specific path, with the docker inspect command. The Mounts section from the output of docker inspect lists all volumes with their respective names and paths on the host system and path inside a container.

Rather than using a random location as a data volume, you can also specify a particular directory on the host to be used as a data volume. Add a host directory along with the volume argument, and Docker will map the volume to that directory:

$ docker run -dP -v ~/mysql:/var/lib/mysql \

--name mysql mysql:latest

In this case, /var/lib/mysql from the container will be mapped to the mysql directory located at the user's home address.

Need to share a single file from a host system with a container? Sure, Docker supports that too. Use docker run -v and specify the file source on the host and destination inside the container. Check out following example command:

$ docker run --rmd -v ~/.bash_history:/.bash_history ubuntu

The other option is to create a named data volume container or data-only container. You can create a named container with attached volumes and then use those volumes inside other containers using the docker run --volumes-from command. The data volumes container need not be running to access volumes attached to it. These volumes can be shared by multiple containers, plus you can create temporary, throwaway application containers by separating persistent data storage. Even if you delete a temporary container using a named volume, your data is still safe with a volume container.

From Docker version 1.9 onwards, a separate command, docker volume, is available to manage volumes. With this update, you can create and manage volumes separately from containers. Docker volumes support various backend drivers, including AUFS, OverlayFS, BtrFS, and ZFS. A simple command to create a new volume will be as follows:

$ docker volume create --name=myvolume

$ docker run -v myvolume:/opt alpine sh

See also

The Docker volumes guide: http://docs.docker.com/engine/userguide/dockervolumes/

Clean up orphaned volumes with this script: https://github.com/chadoe/docker-cleanup-volumes

We have already seen how to create virtual machines with KVM and Qemu, and how to manage them with tools such as virsh and virt-manager. This approach works when you need to work with a handful of machines and manage few hosts. To operate on a larger scale, you need a tool to manage host machines, VM configurations, images, network, and storage, and monitor the entire environment. OpenStack is an open source initiative to create and manage a large pool of virtual machines (or containers). It is a collection of various tools to deploy IaaS clouds. The official site defines OpenStack as an operating system to control a large pool of compute, network, and storage resources, all managed through a dashboard.

OpenStack was primarily developed and open-sourced by Rackspace, a leading cloud service provider. With its thirteenth release, Mitaka, OpenStack provides tons of tools to manage various components of your infrastructure. A few important components of OpenStack are as follows:

Nova: Compute controller

Neutron: OpenStack networking

Keystone: Identity service

Glance: OpenStack image service

Horizon: OpenStack dashboard

Cinder: Block storage service

Swift: Object store

Heat: Orchestration program

OpenStack in itself is quite a big deployment. You need to decide the required components, plan their deployment, and install and configure them to work in sync. The installation itself can be a good topic for a separate book. However, the OpenStack community has developed a set of scripts known as DevStack to support development with faster deployments. In this recipe, we will use the DevStack script to quickly install OpenStack and get an overview of its workings. The official OpenStack documentation provides detailed documents for the Ubuntu based installation and configuration of various components. If you are planning a serious production environment, you should read it thoroughly.

Getting ready

You will need a non-root account with sudo privileges. The default account named ubuntu should work.

The system should have at least two CPU cores with at least 4 GB of RAM and 60 GB of disk space. A static IP address is preferred. If possible, use the minimal installation of Ubuntu.

DevStack scripts are available on GitHub. Clone the repository or download and extract it to your installation server. Use the following command to clone:

$ git clone https://git.openstack.org/openstack-dev/devstack \

-b stable/mitaka --depth 1

$ cd devstack

You can choose to get the latest release by selecting the master branch. Just skip the -b stable/mitaka option from the previous command.

How to do it…

Once you obtain the DevStack source, it's as easy as executing an installation script. Before that, we will create a minimal configuration file for passwords and basic network configuration:

Copy the sample configuration to the root of the devstack directory:

$ cp samples/local.conf

Edit local.conf and update passwords:

ADMIN_PASSWORD=password

DATABASE_PASSWORD=password

RABBIT_PASSWORD=password

SERVICE_PASSWORD=$ADMIN_PASSWORD

Add basic network configuration as follows. Update IP address range as per your local network configuration and set FLAT_INTERFACE to your primary Ethernet interface:

FLOATING_RANGE=192.168.1.224/27

FIXED_RANGE=10.11.12.0/24

FIXED_NETWORK_SIZE=256

FLAT_INTERFACE=eth0

Save the changes to the configuration file.

Now, start the installation with the following command. As the Mitaka stable branch has not been tested with Ubuntu Xenial (16.04), we need to use the FORCE variable. If you are using the master branch of DevStack or an older version of Ubuntu, you can start the installation with the ./stack.sh command:

$ FORCE=yes ./stack.sh

The installation should take some time to complete, mostly depending on your network speed. Once the installation completes, the script should output the dashboard URL, keystone API endpoint, and the admin password:

Now, access the OpenStack dashboard and log in with the given username and password. The admin account will give you an admin interface. The login screen looks like this:

Once you log in, your admin interface should look something like this:

Now, from this screen, you can deploy new virtual instances, set up different cloud images, and configure instance flavors.

How it works…

We used DevStack, an unattended installation script, to install and configure basic OpenStack deployment. This will install OpenStack with the bare minimum components for deploying virtual machines with OpenStack. By default, DevStack installs the identity service, Nova network, compute service, and image service. The installation process creates two user accounts, namely admin and dummy. The admin account gives you administrative access to the OpenStack installation and the dummy account gives you the end user interface. The DevStack installation also adds a Cirros image to the image store. This is a basic lightweight Linux distribution and a good candidate to test OpenStack installation.

The default installation creates a basic flat network. You can also configure DevStack to enable Neutron support, by setting the required options in the configuration. Check out the DevStack documentation for more details.

There's more…

Ubuntu provides its own easy-to-use OpenStack installer. It provides options to install OpenStack, along with LXD support and OpenStack Autopilot, an enterprise offering by Canonical. You can choose to install on your local machine (all-in-one installation) or choose a Metal as a Service (MAAS) setup for a multinode deployment. The single-machine setup will install OpenStack on multiple LXC containers, deployed and managed through Juju. You will need at least 12 GB of main memory and an 8-CPU server. Use the following commands to get started with the Ubuntu OpenStack installer:

$ sudo apt-get update

$ sudo apt-get install conjure-up

$ conjure-up openstack

While DevStack installs a development-focused minimal installation of OpenStack, various other scripts support the automation of the OpenStack installation process. A notable project is OpenStack Ansible. This is an official OpenStack project and provides production-grade deployments. A quick GitHub search should give you a lot more options.

A step-by-step detailed guide to installing various OpenStack components on Ubuntu server: http://docs.openstack.org/mitaka/install-guide-ubuntu/

DevStack Neutron configuration: http://docs.openstack.org/developer/devstack/guides/neutron.html

OpenStack Ansible: https://github.com/openstack/openstack-ansible

A list of OpenStack resources: https://github.com/ramitsurana/awesome-openstack

Ubuntu MaaS: http://www.ubuntu.com/cloud/maas

Ubuntu Juju: http://www.ubuntu.com/cloud/juju

Read more about LXD and LXC in article 8, Working with Containers