Skip to main content

Ubuntu

Sending e-mails with Postfix in Ubuntu

In this recipe, we will set up Postfix Mail Transfer Agent (MTA). This will be a very basic setup which will enable us to send and receive e-mails from our server. Postfix is an open source MTA which routes e-mails to their destination. It is a default MTA for Ubuntu and is available in Ubuntu's main package repository.

Getting ready

You will need access to a root account or an account with sudo privileges.

A domain name (FQDN) is required while configuring Postfix. You can configure your local server for testing, but make sure that you set the proper host entries and hostname.

How to do it…

Follow these steps to send e-mails with Postfix:

Install Postfix and mailutils with the following commands:

$ sudo apt-get update

$ sudo apt-get install postfix mailutils -y

The installation process will prompt you to enter some basic configuration details. When asked for General type of mail configuration:, select Internet Site and then click on :

On the next screen, enter your domain name, for example, mail.example.com, and answer the other questions. You can leave them with default values:

After installation completes, we need to modify the Postfix configuration under /etc/postfix/main.cf:

$ sudo nano /etc/postfix/main.cf

Set myhostname to point to your domain name:

myhostname = mail.example.com

Ensure mynetworks is set to the local network. This will secure your server from spammers:

mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128

Also check mydestination. It should contain your domain name:

mydestination = example.com, ubuntu, localhost.localdomain, localhost

Change the mail storage format to Maildir from the default mbox. Search and uncomment the following line:

home_mailbox = Maildir/

Optionally, you can change the TLS keys used by Postfix. Find the TLS parameters section and point the variables to your key path:

Save the configuration file and exit.

Now, reload Postfix for the changes to take effect:

$ sudo service postfix reload

Test if everything is working as expected. Open a telnet connection to the mail server:

$ telnet localhost 25

You should see an output similar to the following screenshot:

Now, send your first e-mail from this server. Type sendmail user@domain and press Enter. Then, type your message, and when done with that press Ctrl + D to send an e-mail.

To read your e-mails, log in with the user you send e-mails to. Start the mail program with the command mail. This should show you a list of e-mails received by this user account. The output should look similar to following screenshot:

To read any e-mail, type in the mail number and press Enter. Type q followed by Enter to quit the mail reader.

How it works…

Postfix installation is quite a simple task; you need to be sure that you have configured the proper settings and then you are up and running in minutes. The Postfix installation process itself prompts for basic settings.

Other parameters include mynetworks and mydestination. With mynetwork, we have restricted the uses of the mail server to the local network. Only users on the local network can use this server to send and receive e-mails. The parameter mydestination specifies the domain names that Postfix is going to serve. For all other domains that are not listed under mydestination, Postfix will simply act as a forwarder.

We have configured Postfix to use the Maildir format for storing e-mails. This is a new storage format and provides various improvements over the default format, mbox. Also, Maildir is used by various IMAP and POP servers. With Maildir, each new message is stored in a separate file. This avoids file locking when working with messages and provides protection against mailbox corruption.

Now if you send an e-mail to a local domain, it will be delivered to the inbox of the respective user, which can be read with mail command. If you send e-mails to an external mail server, such as Gmail, chances are your mail gets delivered to spam. You need to include a number of different parameters in your e-mail headers and then make sure that your server IP is not blacklisted. It would be a good idea to use an external mail server such as Mail Chimp or Gmail for sending e-mails.

See also

An article by Jeff Atwood on sending e-mails through code. This may help you get your e-mails out of spam: http://blog.codinghorror.com/so-youd-like-to-send-some-email-through-code/

Mailbox formats: http://wiki.dovecot.org/MailboxFormat

The difference between port 465 and 587: http://stackoverflow.com/questions/15796530/what-is-the-difference-between-ports-465-and-587

Installing VNC server

VNC (Virtual Network Computing) enables us to access the GUI of a remote system over a secured network. The VNC client installed on a local system captures the input events of a mouse and keyboard and transfers them to the remote VNC server. Those events are executed on a remote system and the output is sent back to the client. VNC is a desktop sharing tool and is generally used to access the desktop system for remote administration and technical support.

With Ubuntu server, we rarely need a desktop environment. However, if you are a newbie administrator or quite unfamiliar with the command line environment, then GUI becomes a handy tool for you. Plus, you may want to deploy a shared remote desktop environment where people can collaborate with each other. This recipe covers the installation of the VNC server on Ubuntu Server 14.04. We will install a GUI component that is required by VNC and then install and configure the VNC server.

Getting ready

You will need access to a root account or an account with sudo privileges.

How to do it…

The Ubuntu server and cloud editions generally ship with a minimal installation footprint and do not contain GUI components. We will use Gnome-core as our desktop component. Gnome-core is a part of an open source desktop environment.

Access the server shell and use the following command to install gnome-core:

$ sudo apt-get update

$ sudo apt-get install gnome-core -y

This will take some time as the command needs to download a bunch of components and install them.

Once Gnome is installed, we can proceed with VNC server installation using the following command:

$ sudo apt-get install vnc4server -y

When installation completes, start a new VNC session by using the following command:

$ vncserver

As this is the first time we have started VNC, you will be prompted to set up a password. This session will also create a few configuration files required for VNC. Your screen should look similar to the screenshot below:

Next, we will edit the default configuration files created by our first session, kill the VNC process, and then edit the configuration file:

$ vncserver -kill :1

Killing Xvnc4 process ID 2118

Edit the default configuration file and set it to use the Gnome session. Open ~/.vnc/xstartup and uncomment or add the following line to it:

$ nano ~/.vnc/xstartup

#!/bin/sh

# Uncomment the following two lines for normal desktop:

unset SESSION_MANAGER

# exec /etc/X11/xinit/xinitrc

#[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup

#[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources

#xsetroot -solid grey

#vncconfig -iconic &

#x-terminal-emulator -geometry 80x24+10+10 -ls -title "$VNCDESKTOP Desktop" &

#x-window-manager &

metacity &

gnome-settings-daemon &

gnome-panel &

Optionally, disable the Gnome startup script. This will stop Gnome from starting with a system boot and you will see a CLI login instead of the new Gnome-based graphical login screen. Open /etc/init/gdm.conf and comment out the following lines:

$ sudo nano /etc/init/gdm.conf

#start on ((filesystem

# and runlevel [!06]

# and started dbus

# and plymouth-ready)

# or runlevel PREVLEVEL=S)

Save all modifications in configuration files and start a new VNC session. This time, we will add screen resolution and color depth options:

$ vncserver -geometry 1366x768 -depth 24

Next, from your local system, install the VNC client software and open it. I have used the TightVNC client. Enter your server IP address and a VNC desktop number to be connected. Here, we have created a single session to a sample IP address, which will be 192.168.0.1:1:

Click Connect; you will be prompted for a password to authenticate your session:

Enter the password that we created while starting the first session. You should see a desktop screen with a basic Gnome theme. The following is the scaled screenshot of the VNC viewer:

How it works…

VNC works with a client-server model. We have installed the VNC server daemon on our Ubuntu Server and a client on the local system. The server daemon communicates with the GUI buffer or frame buffer on the server side and transfers that buffer data to the client. The client renders that buffer in specially designed software called the VNC viewer. In addition to rendering the remote buffer, the VNC client or viewer captures mouse and keyboard (input) events happening over the client window. Those events are then sent to the VNC server, which applies them to the current graphics frame and any updates are sent back to client.

The pevious example uses simple Gnome-core components. This is a basic graphics suite which contains graphics drives, plus some other tools such as the Firefox browser and an instant messaging client. You can even choose to have a limited setup and install selective, required selected required Gnome packages as follows:

$ sudo apt-get install gnome-panel gnome-settings-daemon \

metacity nautilus gnome-terminal

This GUI does not match the one provided by Ubuntu Desktop. If you prefer to have the same experience as Ubuntu Desktop, you can separately install a package, ubuntu-desktop:

$ sudo apt-get install ubuntu-desktop

VNC does support multiple sessions to a single server. You may have noticed in the connection address used previously that we used :1 to represent the first session or display. This is shorthand for the full port number, which is 5901 for the first session, 5092 for the second, and so on. You can use the full port or just the last digit to refer to a session. Notice the change in desktop number when we start multiple VNC sessions:

Additionally, you can start a new VNC session for different users with its own password. Simply log in or switch (su user1) to the user account, start vncserver, set the password, and you are done.

See also

How VNC works on Stack Overflow - http://stackoverflow.com/questions/4833152/how-realvnc-works

Securing Ubuntu server against brute force attacks

So you have installed minimal setup of Ubuntu, you have setup SSH with public key authentication and disabled password authentication, and you have also allowed only single non-root user to access the server. You also configured a firewall, spending an entire night understanding the rules, and blocked everything except a few required ports. Now does this mean that your server is secured and you are free to take a nice sound sleep? Nope.

Servers are exposed to the public network, and the SSH daemon itself, which is probably the only service open, and can be vulnerable to attacks. If you monitor the application logs and access logs, you can find repeated systematic login attempts that represent brute force attacks.

Fail2ban is a service that can help you monitor logs in real time and modify iptables rules to block suspected IP addresses. It is an intrusion-prevention framework written in Python. It can be set to monitor logs for SSH daemon and web servers. In this recipe, we will discuss how to install and configure fail2ban.

Getting ready

You will need access to a root account or an account with similar privileges.

How to do it…

Follow these steps to secure against brute force attacks:

Fail2ban is available in the Ubuntu package repository, so we can install it with a single command, as follows:

$ sudo apt-get update

$ sudo apt-get install fail2ban

Create a copy of the fail2ban configuration file for local modifications:

$ sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Open a new configuration file in your favorite editor:

$ sudo nano /etc/fail2ban/jail.local

You may want to modify the settings listed under the [DEFAULT] section:

Add your IP address to the ignore IP list.

Next, set your e-mail address if you wish to receive e-mail notifications of the ban action:

destemail = you@provider.com

sendername = Fail2Ban

mta = sendmail

Set the required value for the action parameter:

action = $(action_mwl)s

Enable services you want to be monitored by setting enable=true for each service. SSH service is enabled by default:

[ssh]

enable = true

Set other parameters if you want to override the default settings.

Fail2ban provides default configuration options for various applications. These configurations are disabled by default. You can enable them depending on your requirement.

Restart the fail2ban service:

$ sudo service fail2ban restart

Check iptables for the rules created by fail2ban:

$ sudo iptables -S

Try some failed SSH login attempts, preferably from some other system.

Check iptables again. You should find new rules that reject the IP address with failed login attempts:

How it works…

Fail2ban works by monitoring the specified log files as they are modified with new log entries. It uses regular expressions called filters to detect log entries that match specific criteria, such as failed login attempts. Default installation of fail2ban provides various filters that can be found in the /etc/fail2ban/filter.d directory. You can always create your own filters and use them to detect log entries that match your criteria.

Once it detects multiple logs matching with the configured filters within the specified timeout, fail2ban adjusts the firewall settings to reject the matching IP address for configured time period.

There's more…

Check out the article about defending against brute force attacks at http://www.la-samhna.de/library/brutessh.html .

The preceding articles shows multiple options to defend against SSH brute force attacks. As mentioned in the article, you can use iptables to slow down brute force attacks by blocking IP addresses:

$ iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent --set --name SSH -j ACCEPT

$ iptables -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j LOG --log-prefix "SSH_brute_force "

$ iptables -A INPUT -p tcp --dport 22 -m recent --update --seconds 60 --hitcount 4 --rttl --name SSH -j DROP

These commands will create an iptables rule to permit only three SSH login attempts per minute. After three attempts, whether they are successful or not, the attempting IP address will be blocked for another 60 seconds.

Configuring Ejabberd installation in Ubuntu

Ejabberd comes with various default settings that make it easy to get started. We can install Ejabberd and start using it as soon as installation completes. This works when we are testing our setup, but when we need a production server, we need to make a number of changes to the default installation. Ejabberd provides a central configuration file through which we can easily configure our XMPP installation.

This recipe covers the basic configuration of the Ejabberd server.

Getting ready

Make sure that you have installed the Ejabberd server.

You will need access to a root account or an account with sudo privileges.

How to do it…

Ejabberd configuration files are located under the conf directory in the Ejabberd installation. On the Ubuntu server, it should be /opt/ejabberd-version/conf.

Follow these steps to configure the Ejabberd installation:

Open the ejabberd.yml file. It contains configuration settings in the YML format.

Let us start by setting the domain for our XMPP service. This is located under the SERVED HOSTNAMES section in the configuration file. The default setting uses the server hostname as a host for the XMPP service.

Add a fully qualified domain name under the hosts section. You can choose to keep the default host entry or remove it:

Next, you may want to change the default ports for XMPP connections. Search for the LISTENING PORTS section in ejabberd.yml and change the respective ports. I will use the default port configuration. The following is the configuration snippet listing port 5222:

The LISTENING PORTS section contains different port configurations, each serving a separate service. Three of them are enabled by default and serve a client to server connection (5222), server to server connection (5269), and HTTP module for admin console and http_bind service (5280).

The same section contains the parameter named certfile, which specifies the SSL certificate file to be used while creating client connections. The default settings point to a certificate created by the Ejabberd installation process. You can change it to your own signed certificate.

Also note the shaper and access settings. These settings specify the connection throttling and access control settings used for the client to server connections respectively.

At the end of the LISTENING PORTS section, there is a configuration for BOSH (port 5280) connections, as well as the web admin panel. This section also enables web socket connections with the ejabberd_http_ws module.

Under the AUTHENTICATION section, you can configure the authentication mechanism to be used. By default, Ejabberd uses internal authentication but it can be set to use external scripts, system-level authentication, external databases, or even a centralized LDAP service. The following is the list of all supported options:

Default internal authentication works well enough and we will proceed with it. If you are planning to use a different authentication mechanism, make sure that you comment out internal authentication.

You can also enable anonymous login support, where clients can open an XMPP connection without a username and password. Simply uncomment the respective settings from Anonymous login support:

Next, under the DATABASE SETUP section, you can set Ejabberd to use an external database system. Ejabberd supports all leading relational database systems, including SQLite. The following is the list of all supported database systems:

The default database settings use an inbuilt database server known as Mnesia. It provides in-memory and disk-based storage and can be easily replicated across Ejaberd nodes. Mnesia works well even for very busy XMPP operations.

To define an admin user, search for the ACCESS CONTROL LISTS section and add your desired username and hostname under the admin users list:

This same section includes a list of blocked users.

You can also define your own access control lists, which can be used to restrict permissions to specific hostnames or users. The Access Rules section define the rules applicable to listed ACLs.

Finally, under the modules section, you can configure the modules to be used by Ejabberd. Modules are plugins to extend the functionality of the Ejabberd server. Comment out the modules that you are not planning to use. You can also enable or disable any module in runtime from the web admin panel. The following is the partial list of modules:

Each module is named after respective XEPs (XMPP extensions). You can get details of the functionality of any module by looking for the related XEP. Also check the Ejabberd documentation to find out the dependencies between modules.

Once you are done with all the configuration, you can restart the Ejabberd server with ejabberdctl restart or reload configuration changes with the ejabberdctl reload_config command:

$ sudo bin/ejabberdctl reload_config

How it works…

Most of the core settings of Ejabberd are controlled through the configuration file, ejabberd.yml. Alternatively, you can change settings with the ejabberdctl command, but those settings will not persist after restart. If you need the settings to be permanent, change them in the configuration file. You can always reload the configuration file changes without restarting the server.

While editing the configuration file, make sure that you follow the indentation and spacing as shown in examples. Ejabberd configuration follows the YML format and any change in spacing will leave that setting undefined. The good news is that the latest version of Ejabberd will prompt you about any mistakes in configuration.

There's another file named ejabberdctl.cfg that contains Erlang runtime settings. You may need to update those parameters while performance tuning the Ejabberd server.

Strophe is a collection of libraries that can be used to communicate with the XMPP server. It contains libstrophe, which is a C-based implementation of XMPP client functionalities, and Strophe.js, which is a JavaScript implementation. Strophe provides core XMPP client functionality and can be extended with custom modules. The community has contributed various extensions to support additional XMPP functions.

With a limit on page count, I will focus on a simple demo of Strophe.js where we download the code and modify an example to connect with our XMPP server.

Getting ready

You will need the XMPP server installed and running. You can also use public XMPP servers, but make sure that you register with them and obtain your username (JID) and password.

You will need at least two user accounts to communicate with each other.

As we are using a web-based connection, it needs a Bidirectional-streams Over Synchronous HTTP (BOSH) extension enabled on the XMPP server. Ejabberd supports this functionality with mod_http_bind and it should be enabled by default.

Download and extract the latest source achieve from the Strophe.js site: http://strophe.im/strophejs/ .

Optionally, you will need a web server set up to access a web client.

How to do it…

I assume the source code is located in the StropheJS directory. We will use one of the examples shipped with the StropheJS source:

Change the directory to examples under the extracted StropheJS code. This directory contains multiple examples, demonstrating different features of StropheJS. We will use echobot.js and echobot.html as our starting point.

Open echobot.js and change the BOSH_SERVICE URL on the first line, as follows:

var BOSH_SERVICE = 'http://hostname:5280/http-bind';

Replace the hostname with your XMPP domain or XMPP server IP address. For example, if your XMPP server is available at xmpp.mysrv.com, then the BOSH_SERVICE URL will be as follows:

var BOSH_SERVICE = 'http://xmpp.mysrv.com:5280/http-bind';

Optionally, you can enable debug logging to watch actual data exchanged between client and server. Find the $(document).ready() section and uncomment the following lines:

// uncomment the following lines to spy on the wire traffic.

connection.rawInput = function (data) { log('RECV: ' + data); };

connection.rawOutput = function (data) { log('SEND: ' + data); };

Save the changes to echobot.js and open echobot.html in your browser. You should see a page with two text fields, one for JID and another for Password:

Enter your JID (XMPP username) and respective password and click connect.

Now, Strophe.js will try to open an XMPP connection and log in with the given details. If the connection is successful, you should see the following screen:

The last line includes your JID, with a unique identifier for the current session appended at the end. This form of JID is also called full JID.

Open a separate client connection with, say, PSI, log in with some other user, and send a message on your given JID. This should print your message on the web page and the same message will be echoed back to the sender. Your web page should look similar to the following screenshot:

How it works…

Strophe.js is a JavaScript-based XMPP client library that makes it easy to write your own web-based XMPP clients. Strophe handles all actual communication parts, such as the encoding and decoding of XML stanzas, the connection procedure, and so on. You can use simple APIs provided by Strophe to create your client. Strophe.js uses jQuery to work with the HTML DOM, so if you are familiar with jQuery you will feel at home when working with Strophe.

If you browse through the code in echobot.js, you will see two main event handlers: onConnect and onMessage. These event handlers are attached to specific events and are executed when that event occurs. The onConnect handler is attached to a connection object to capture any change in connection state, and onMessage is attached as a handler for message events. It will be triggered when our client receives any message from the server.

If you are interested in the syntax for the addHandler function, it is as follows:

addHandler: function (handler,ns,name,type,id,from,options)

The handler parameter is the actual function to manipulate an incoming message object; ns is the XMPP namespace and can be used to receive packets only from a certain namespace. It defaults to jabber:client, the name parameter, which is the name of an element to act upon—in our case, it is message. You can use iq or presence to receive respective data types. Other parameters add more filtering options, where you can specify a specific ID for the message, type of the message packet (chat or normal or group, defaults to chat) and other options.

The handler function onMessage gets triggered whenever a connection object receives a new message from the server. Then, it parses the received data and extracts all required information. As it is an echo bot, it simply reads the message and echoes it back to the sender. The new message packet is generated with the following lines:

var reply = $msg({to: from, from: to, type: 'chat'})

.cnode(Strophe.copyElement(body));

The message is passed to a connection object with the following lines, which in turn sends it to the server:

connection.send(reply.tree());

The last section initiates the Strophe client on page load (ready). When we click on the connect button, a click handler in this section gets triggered and opens a new connection with the XMPP server. The same button is changed to disconnect so that we can send a proper disconnect request to the server.

There's more…

Strophe.js supports WebSocket-based XMPP connections, and the latest version of Ejabberd has also added support for WebSockets. WebSockets provides noticeable performance improvements and reduces connection time over BOSH connections. In the preceding example, we have used the BOSH protocol, which can be replaced with WebSocket simply by changing the BOSH_SERVICE URL as follows:

var BOSH_SERVICE = 'ws:// hostname:5280/websocket';

If you need a secure WebSocket connection, use the wss protocol instead of:

wsvar BOSH_SERVICE = 'wss:// hostname:5280/websocket';

You should check other examples, mainly prebind and restore. Both demonstrate connection features that can help in reducing connection delay.

See also

StropheJS official page at http://strophe.im/strophejs/

StropheJS GitHub repo at https://github.com/strophe/strophejs

StropheJS API documentation at http://strophe.im/strophejs/doc/1.1.3/files/strophe-js.html

StropheJS plugins at https://github.com/metajack/strophejs-plugins

Getting root privileges with sudo in Ubuntu server

When you create a new Ubuntu server in the cloud, by default you get the root account. This account has full system access with no restrictions at all and should only be used for administrative tasks. You can always create a new user account with fewer privileges. But there are times when you need extra root privileges to add a new user or change some system setting. You can use the sudo command to temporarily get extra privileges for a single command. In this recipe, you will see how to grant sudo privileges to a newly created user.

Getting ready

You will need a root account or an account with root privileges.

How to do it...

Follow these steps to get the root privileges with sudo:

  1. Add new user if required:

    $sudo adduser john

  2. Make john a member of sudo group with the following command:

$sudo adduser username sudo

How it works…

All sudo access rules are configured in a file located at /etc/sudoers. This file contains a list of users and groups that are allowed to use the sudo command:

alan ALL=(ALL:ALL)ALL // allow sudo access to user alan

%sudo ALL=(ALL) ALL // allow sudo access to members of sudo

The line alan ALL=(ALL:ALL) ALL specifies that the user alan can run any command as any user and optionally set any group (taken from man pages for sudoers: man sudoers).

The entry %sudo ALL=(ALL) ALL specifies that any member of system group sudo can run any command as any user.

All we have to do is add a new user to the group sudo and that user will automatically get sudo privileges. After getting the membership of the sudo group, user needs to log out and log back in for the changes to take effect. Basically, the user shell needs to be restarted with new privileges. Optionally, you can always go and change the sudoers file for a specific condition.

Make sure that you use the visudo tool to make any changes to sudoers file.

There's more…

Here, we will discuss how to set a password-less sudo and some additional benefits of sudo.

Setting password less sudo

sudo is a useful and handy tool for temporary root privileges, but you need to enter your password every time. This creates problems especially for users with no password set. This problem can be solved by setting the NOPASSWD flag in the sudoers file. Make sure you use the visudo tool to edit the sudoers file:

Open the sudoers file with the visudo command:

$sudo visudo

Select the line for user or group you want to allow password-less sudo access.

Add NOPASSWD after closing the bracket:

%sudo ALL=(ALL:ALL) NOPASSWD: ALL

Press Ctrl + O and then confirm with the Enter key to save the changes.

Press Ctrl + X to exit visudo.

Now, the users of the group sudo should be able to use the sudo command without providing a password. Alternatively, you can add a separate entry to limit password-less access to a specific user.

Note that the sudoers program performs cache authentication for a small time (default is 15 minutes). When repeated within timeout, you may notice password-less sudo without setting the NOPASSWD flag.

Other uses of sudo

In addition to running a single command with sudo, you might want to execute a list of commands with the sudo privileges. Then, you can open a shell with root access (# prompt) with the command $sudo -s. The shell environment remains same as original user, but now you can execute commands as a root user.

Alternatively, you can switch user to root with the command $sudo su -. This command will open a new shell as a root user.

See also

Check manual pages for sudo with $man sudo

For more details on adduser, check the Creating user account recipe

Starting and managing Docker containers in Ubuntu

So, we have installed the latest Docker binary. In this recipe, we will start a new container with Docker. We will see some basic Docker commands to start and manage Docker containers.

Getting ready

Make sure that you have installed Docker and set your user as a member of the Docker group.

You may need sudo privileges for some commands.

How to do it…

Let's create a new Docker container and start it. With Docker, you can quickly start a container with the docker run command:

Start a new Docker container with the following command:

$ docker run -it --name dc1 ubuntu /bin/bash

Unable to find image 'ubuntu:trusty' locally

trusty: Pulling from library/ubuntu

6599cadaf950: Pull complete

23eda618d451: Pull complete

...

Status: Downloaded newer image for ubuntu:trusty

root@bd8c99397e52:/#

Once a container has been started, it will drop you in a new shell running inside it. From here, you can execute limited Ubuntu or general Linux commands, which will be executed inside the container.

When you are done with the container, you can exit from the shell by typing exit or pressing Ctrl + D. This will terminate your shell and stop the container as well.

Use the docker ps command to list all the containers and check the status of your last container:

$ docker ps -a

By default, docker ps lists all running containers. As our container is no longer running, we need to use the -a flag to list all available containers.

To start the container again, you can use the docker start command. You can use the container name or ID to specify the container to be started:

$ docker start -ia dc1

The -i flag will start the container in interactive mode and the -a flag will attach to a terminal inside the container. To start a container in detached mode, use the start command without any flags. This will start the container in the background and return to the host shell:

$ docker start dc1

You can open a terminal inside a detached container with docker attach:

$ docker attach dc1

Now, to detach a terminal and keep the container running, you need the key combinations Ctrl + P and Ctrl + Q. Alternatively, you can type exit or press Ctrl + C to exit the terminal and stop the container.

To get all the details of a container, use the docker inspect command with the name or ID of the container:

$ docker inspect dc1 | less

This command will list all the details of the container, including container status, network status and address, and container configuration files.

To execute a command inside a container, use docker exec. For example, the following command gets the environment variables from the dc1 container:

$ docker exec dc1 env

This one gets the IP address of a container:

$ docker exec dc1 ifconfig

To get the processes running inside a container, use the docker top command:

$ docker top dc1

Finally, to stop the container, use docker stop, which will gracefully stop the container after stopping processes running inside it:

$ docker stop dc1

When you no longer need the container, you can use docker rm to remove/delete it:

$ docker rm dc1

How it works…

We started our first Docker container with the docker run command. With this command, we instructed the Docker daemon to start a new container with an image called Ubuntu, start an interactive session (-i), and allocate a terminal (-t). We also elected to name our container with the --name flag and execute the /bin/bash command inside a container once it started.

The Docker daemon will search for Ubuntu images in the local cache or download the image from Docker Hub if the specified image is not available in the local cache. Docker Hub is a central Docker image repository. It will take some time to download and extract all the layers of the images. Docker maintains container images in the form of multiple layers. These layers can be shared across multiple container images. For example, if you have Ubuntu running on a server and you need to download the Apache container based on Ubuntu, Docker will only download the additional layer for Apache as it already has Ubuntu in the local cache, which can be reused.

Docker provides various other commands to manage containers and images. We have already used a few of them in the previous example. You can get the full list of all available commands from the command prompt itself, by typing docker followed by the Enter key. All commands are listed with their basic descriptions. To get more details on any specific subcommand, use its help menu, as follows:

$ docker rmi --help

There's more…

Docker images can be used to quickly create runc containers, as follows:

$ sudo apt-get install runc

$ mkdir -p runc/rootfs && cd runc

$ docker run --name alpine alpine sh

$ docker export alpine > alpine.tar

$ tar -xf alpine.tar -C rootfs

$ runc spec

$ sudo runc start alpine

See also

Docker run documentation: http://docs.docker.com/engine/reference/commandline/run/

Check manual entries for any Docker command: $ man docker create

Creating virtual machine with KVM in Ubuntu

Ubuntu server gives you various options for your virtualization needs. You can choose from KVM, XEN, QEMU, VirtualBox, and various other proprietary and open source tools. KVM, or Kernel virtual machine, is the default hypervisor on Ubuntu. In this recipe, we will set up a virtual machine with the help of KVM. Ubuntu, being a popular cloud distribution provides prebuilt cloud images that can be used to start virtual machines in the cloud. We will use one of these prebuilt images to build our own local virtual machine.

Getting ready

As always, you will need access to the root account or an account with sudo privileges.

How to do it…

Follows these steps to install KVM and launch a virtual machine using cloud image:

To get started, install the required packages:

$ sudo apt-get install kvm cloud-utils \

genisoimage bridge-utils

Before using KVM, you need to check whether your CPU supports hardware

virtualization, which is required by KVM. Check CPU support with the

following command:

$ kvm-ok

You should see output like this:

INFO: /dev/kvm exists

KVM acceleration can be used.

Next, download the cloud images from the Ubuntu servers. I have selected the Ubuntu 14.04 Trusty image:

$ wget http://cloud- images.ubuntu.com/releases/trusty/release/ubuntu-14.04-server- cloudimg-amd64-disk1.img -O trusty.img.dist

This image is in a compressed format and needs to be converted into an uncompressed format. This is not strictly necessary but should save on-demand decompression when an image is used. Use the following command to convert the image:

$ qemu-img convert -O qcow2 trusty.img.dist trusty.img.orig

Create a copy-on-write image to protect your original image from modifications:

$ qemu-img create -f qcow2 -b trusty.img.orig trusty.img

Now that our image is ready, we need a cloud-config disk to initialize this image and set the necessary user details. Create a new file called user-data and add the following data to it:

$ sudo vi user-data

#cloud-config

password: password

chpasswd: { expire: False }

ssh_pwauth: True

This file will set a password for the default user, ubuntu, and enable password authentication in the SSH configuration.

Create a disk with this configuration written on it:

$ cloud-localds my-seed.img user-data

Next, create a network bridge to be used by virtual machines. Edit /etc/network/interfaces as follows:

auto eth0

iface eth0 inet manual

auto br0

iface br0 inet dhcp

bridge_ports eth0

On Ubuntu 16.04, you will need to edit files under the /etc/network/interfaces.d

directory. Edit the file for eth0 or your default network interface, and

create a new file for br0. All files are merged under /etc/network/interfaces.

Restart the networking service for the changes to take effect. If you are on an SSH connection, your session will get disconnected:

$ sudo service networking restart

Now that we have all the required data, let's start our image with KVM, as follows:

$ sudo kvm -netdev bridge,id=net0,br=br0 \

-net user -m 256 -nographic \

-hda trusty.img -hdb my-seed.img

This should start a virtual machine and route all input and output to your console. The first boot with cloud-init should take a while. Once the boot process completes, you will get a login prompt. Log in with the username ubuntu and the password specified in user-data.

Once you get access to the shell, set a new password for the user ubuntu:

$ sudo passwd ubuntu

After that, uninstall the cloud-init tool to stop it running on the next boot:

$ sudo apt-get remove cloud-init

Your virtual machine is now ready to use. The next time you start the machine, you can skip the second disk with the cloud-init details and route the system console to VNC, as follows:

$ sudo kvm -netdev bridge,id=net0,br=br0 \

-hda trusty.img \

-m 256 -vnc 0.0.0.0:1 -daemonize

How it works…

Ubuntu provides various options to create and manage virtual machines. The previous recipe covers basic virtualization with KVM and prebuilt Ubuntu Cloud images. KVM is very similar to desktop virtualization tools such as VirtualBox and VMware. It comes as a part of the Qemu emulator and uses hardware acceleration features from the host CPU to boost the performance of virtual machines. Without hardware support, the machines need to run inside the Qemu emulator.

After installing KVM, we have used Ubuntu cloud image as our pre-installed boot disk. Cloud images are prebuilt operating system images that do not contain any user data or system configuration. These images need to be initialized before being used. Recent Ubuntu releases contain a program called cloud-init, which is used to initialize the image at first boot. The cloud-init program looks for the metadata service on the network and queries user-data once the service is found. In our case, we have used a secondary disk to pass user data and initialize the cloud image.

We downloaded the prebuilt image from the Ubuntu image server and converted it to uncompressed format. Then, we created a new snapshot with the backing image set to the original prebuilt image. This should protect our original image from any modifications so that it can be used to create more copies. Whenever you need to restore a machine to its original state, just delete the newly created snapshot images and recreate it. Note that you will need to use the cloud-init process again during such restores.

This recipe uses prebuilt images, but you can also install the entire operating system on virtual machines. You will need to download the required installation medium and attach a blank hard disk to the VM. For installation, make sure you set the VNC connection to follow the installation steps.

There's more…

Ubuntu also provides the virt-manager graphical interface to create and manage KVM virtual machines from a GUI. You can install it as follows:

$ sudo apt-get install virt-manager

Alternatively, you can also install Oracle VirtualBox on Ubuntu. Download the .deb file for your Ubuntu version and install it with dpkg -i, or install it from the package manager as follows:

Add the Oracle repository to your installation sources. Make sure to substitute xenial with the correct Ubuntu version:

$ sudo vi /etc/apt/sources.list

deb http://download.virtualbox.org/virtualbox/debian xenial contrib

Add the Oracle public keys:

wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -

Install VirtualBox:

$ sudo apt-get update && sudo apt-get install virtualbox-5.0

See also

VirtualBox downloads: https://www.virtualbox.org/wiki/Linux_Downloads

Ubuntu Cloud images on a local hypervisor: https://help.ubuntu.com/community/UEC/Images#line-105

The Ubuntu community page for KVM: https://help.ubuntu.com/community/KVM

Optimizing MySQL performance – configuration in Ubuntu

MySQL has hundreds of settings that can be configured. Version 5.7 ships with many improvements in default configuration values and requires far fewer changes. In this recipe, we will look at some of the most important parameters for tuning MySQL performance.

Getting ready

You will need access to a root account or an account with sudo privileges.

You will need access to a root account on the MySQL server.

How to do it…

Follow these steps to improve MySQL configuration:

First, create a backup of the original configuration file:

$ cd /etc/mysql/mysql.conf.d

$ sudo cp mysqld.cnf mysqld.cnf.bkp

Now open my.cnf for changes:

$ sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf

Adjust the following settings for your InnoDB tables:

innodb_buffer_pool_size = 512M # around 70% of total ram

innodb_log_file_size = 64M

innodb_file_per_table = 1

innodb_log_buffer_size = 4M

If you are using MyISAM tables, set the key buffer size:

key_buffer_size = 64M

Enable the slow query log:

slow_query_log = 1

slow_query_log_file = /var/lib/mysql/mysql-slow.log

long_query_time = 2

Disable the query cache:

query_cache_size = 0

Set the maximum connections as per your requirements:

max_connections = 300

Increase the temporary table size:

tmp_table_size = 32M

Increase max_allowed_packet to increase the maximum packet size:

max_allowed_packet = 32M

Enable binary logging for easy recovery and replication:

log_bin = /var/log/mysql/mysql-bin.log

Additionally, you can use mysqltuner.pl, which gives general recommendations about the MySQL best practices:

$ wget http://mysqltuner.pl/ -O mysqltuner.pl

$ perl mysqltuner.pl

How it works…

The preceding example shows some important settings for MySQL performance tuning. Ensure that you change one setting at a time and assess its results. There is no silver bullet that works for all, and similarly, some of these settings may or may not work for you. Secondly, most settings can be changed at runtime with a SET statement. You can test settings in runtime and easily reverse them if they do not work as expected. Once you are sure that settings work as expected, you can move them to the configuration file.

The following are details on the preceding settings:

innodb_buffer_pool_size: the size of the cache where InnoDB data and indexes are cached. The larger the buffer pool, the more data can be cached in it. You can set this to around 70% of available physical memory as MySQL uses extra memory beyond this buffer. It is assumed that MySQL is the only service running on server.

log_file_size: the size of the redo logs. These logs are helpful in faster writes and crash recovery.

innodb_file_per_table: This determines whether to use shared table space or separate files for each table. MySQL 5.7 defaults this setting to ON.

key_buffer_size: determines the key buffer for MyISAM tables.

slow_query_log and long_query_time enable slow query logging and set slow query time respectively. Slow query logging can be useful for identifying repeated slow queries.

Query_cache_size caches the result of a query. It is identified as a bottleneck for concurrent queries and MySQL 5.6 disables it by default.

max_connections sets the number of maximum concurrent connections allowed. Set this value as per your application's requirements. Higher values may result in higher memory consumption and an unresponsive server. Use connection pooling in the application if possible.

max_allowed_packet sets the size of the packet size that MySQL can send at a time. Increase this value if your server runs queries with large result sets. mysqld set it to 16M and mysqldump set it to 24M. You can also set this as a command-line parameter.

log_bin enables binary logging, which can be used for replication and also for crash recovery. Make sure that you set proper rotation values to avoid large dump files.

There’s more…

MySQL performance tuning primer script: This script takes information from show status and show variables statements. It gives recommendations for various settings such as slow query log, max connections, query cache, key buffers, and many others. This shell script is available at http://day32.com/MySQL .

You can download and use this script as follows:

$ wget http://day32.com/MySQL/tuning-primer.sh

$ sh tuning-primer.sh

Percona configuration wizard

Percona systems provide a developer-friendly, web-based configuration wizard to create a configuration file for your MySQL server. The wizard is available at http://tools.percona.com

MySQL table compression

Depending on the type of data, you can opt for compressed tables. Compression is useful for tables with long textual contents and read-intensive workloads. Data and indexes are stored in a compressed format, resulting in reduced I/O and a smaller database size, though it needs more CPU cycles to compress and uncompress data. To enable compression, you need an InnoDB storage engine with innodb_file_per_table enabled and the file format set to Barracuda. Check MySQL documents for more details on InnoDB compression at https://dev.mysql.com/doc/innodb/1.1/en/innodb-compression.html .

See also

MySQL tuner script at https://github.com/major/MySQLTuner-perl

MySQL docs at https://dev.mysql.com/doc/refman/5.7/en/optimization.html

InnoDB table compression at https://dev.mysql.com/doc/refman/5.7/en/innodb-table-compression.html

Authenticating Ejabberd users with LDAP in Ubuntu

In this recipe, we will learn to set up the Ejabberd server to authenticate the user with our LDAP server. Until now, we have set up the LDAP server and used it to log in to the Ubuntu server with a user account created on the LDAP server. This recipe covers the configuration of an external service to work with our LDAP installation.

The Ejabberd server provides built-in support for LDAP-based authentication. You can use LDAP for user authentication as well as vCard storage. As stated in the Ejabberd admin guide, Ejabberd use LDAP as a read-only data source. We cannot create new user accounts in the LDAP directory, but we can change passwords if the mod_register module is enabled.

Getting ready

You will need the Ejabberd service installed and running. Go through article 10Communication Server with XMPP, for details on the installation and configuration of the Ejabberd server.

Create a user account on the LDAP server to be used with Ejabberd.

How to do it…

As Ejabberd provides inbuilt support for LDAP-based authentication, we simply need to edit configurations and set the auth method to LDAP. If you have used a Debian package for the Ejabberd installation, your Ejabberd should be installed in /opt/ejabberd-version directory and the configuration can be found at /etc/ejabberd-version/conf. If you have installed Ejabberd from source, all configuration files are located in the /etc/ejabberd directory:

Open ejabberd.yml from your Ejabberd configuration directory and search for Authentication. With the default settings, it should contain the following line indicating internal authentication:

auth_method: internal

Comment out that line by changing it as follows:

## auth_method: internal

Next, find Authentication using LDAP. This section contains a few parameters and configures communication with the LDAP server. Search and update the following parameters:

ldap_servers:

- "domain/IP of LDAP server"

ldap_port: 389

ldap_rootdn: "cn=admin,dc=example,dc=com"

ldap_password: "password"

ldap_base: "ou=ejabberd,dc=example,dc=com"

I have used a default admin account to authenticate with the LDAP server itself. In a production environment, you should change it to a different account. With a default LDAP setup, you can skip the ldap_rootdn and ldap_password settings to enable anonymous connection.

Next, under the ldap_base parameter, I have restricted users to the Organizational Unit named Ejabberd. Only the user accounts that are configured under the Ejabberd unit can log in with the Ejabberd server.

Now, save the configuration file changes and close the file, and then restart the Ejabberd server with the following command:

$ sudo /opt/ejabberd-version/bin/ejabberdctl restart

If the server fails to restart, check the log files for any configuration errors. Alternatively, you can use the reload_config option to ejabberdctl to update the in-memory configuration without restarting:

$ sudo /opt/ejabberd-version/bin/ejabberdctl reload_config

Once the server has started, you can log in with your LDAP accounts. You will need a JID to log in with Ejabberd, which is a combination of a UID from the LDAP server and any host configured on Ejabberd, for instance, uday@cookbook.com, where uday is the UID on LDAP and cookbook.com is the host served by Ejabberd server. The domain entries on the LDAP server and Ejabberd need not match.

The following is the default host entry for my Ejabberd installation:

Now you can log in to Ejabberd with your LDAP username. Here is the account set up in my chat client with the JID uday@ubuntu, where uday is my LDAP user and ubuntu is the Ejabberd host:

Once all things are set up, you should be able to connect to the Ejabberd server using your LDAP user account.

How it works…

Here, we have set up Ejabberd as an example of LDAP-based authentication. Similar to Ejabberd, various other systems support centralized authentication through LDAP with either built-in support or with a plug-in module. Make sure that you create a proper directory structure with organizational units, roles, and separate users in proper groups. Also use a separate user account for authenticating with the LDAP server itself. You need to set the respective LDAP credentials in the Ejabberd configuration file. If somehow your Ejabberd server gets compromised, then the LDAP server credentials are readily available to an attacker. To limit the risk, using separate and limited accounts is a good idea. Ejabberd also supports anonymous authentication with the LDAP server and mostly uses it as a read-only database. So, even if you skip the authentication details (depending on the LDAP configuration), Ejabberd should work well and authenticate your users.

Ejabberd also provides good enough debug logging, where you can see the actual communication with the LDAP server. You will need to set logging to debug mode in the Ejabberd configuration. The log files are located in the /opt/ejabberd-version/logs directory or the /var/log/ejabberd directory, depending on the source of the Ejabberd installation.