Ubuntu

This article covers various collaboration tools. Collaboration enables people to share thoughts and solve problems collectively. With the help of the Internet, we can communicate quickly and more effectively. Tools such as WhatsApp and Slack have changed the way we communicate personally, as well as in corporate life. Services such as Google Docs hosts our documents in the cloud, which can then be shared with multiple people and simultaneously modified by them. Need a comment on your latest edit? Click that chat button and send your request. Need to discuss face to face? Click another button to start video call. Need to send a long detailed message? Yes, we've got e-mail services.

Most of these services are hosted by Internet giants and available as SAAS (Software as a Service) products. Simply choose subscription plans and start using them. Many of these services even offer free basic plans. The only problem with these services is you've got to trust a service provider with your data. All your messages, emails, photos, and important documents are hosted with some third party.

In this article, we will learn to how set up various open source tools on our own servers. We have already installed an email and instant messaging service, central Git hosting, and a file server. This article will focus on more advanced collaboration tools. We will cover the VNC server to share your desktop, the OwnCloud server for document and file sharing, and Mattermost, an open source Slack alternative.

It is said that the best way to improve server security is to reduce the attack surface. Network communication in any system happens with the help of logical network ports, be it TCP ports or UDP ports. One part of the attack surface is the number of open ports that are waiting for connection to be established. It is always a good idea to block all unrequired ports. Any traffic coming to these ports can be filtered, that is, allowed or blocked with the help of a filtering system.

The Linux kernel provides a built-in packet filtering mechanism called netfilter, which is used to filter the traffic coming in or going out of the system. All modern Linux firewall systems use netfilter under the hood. Iptables is a well-known and popular user interface to set up and manage filtering rules for netfilter. It is a complete firewall solution that is highly configurable and highly flexible. However, iptables need effort on the user's part to master the firewall setup. Various frontend tools have been developed to simplify the configuration of iptables. UFW is among the most popular frontend solutions to manage iptables.

Uncomplicated firewall (UFW) provides easy-to-use interface for people unfamiliar with firewall concepts. It provides a framework for managing netfilter as well as the command-line interface to manipulate the firewall. With its small command set and plain English parameters, UFW makes it quick and easy to understand and set up firewall rules. At the same time, you can use UFW to configure most of the rules possible with iptables. UFW comes preinstalled with all Ubuntu installations after version 8.04 LTS.

In this recipe, we will secure our Ubuntu server with the help of UFW and also look at some advance configurations possible with UFW.

Getting ready

You will need an access to a root account or an account with root privileges.

How to do it…

Follow these steps to secure network with uncomplicated firewall:

UFW comes preinstalled on Ubuntu systems. If it's not, you can install it with the following commands:

$ sudo apt-get udpate

$ sudo apt-get install UFW

Check the status of UFW:

$ sudo ufw status

Add a new rule to allow SSH:

$ sudo ufw allow ssh

Alternatively, you can use a port number to open a particular port:

$ sudo ufw allow 22

Allow only TCP traffic over HTTP (port 80):

$ sudo ufw allow http/tcp

Deny incoming FTP traffic:

$ sudo ufw deny ftp

Check all added rules before starting the firewall:

$ sudo ufw show added

Now enable the firewall:

$ sudo ufw enable

Check the ufw status, the verbose parameter is optional:

$ sudo ufw status verbose

Get a numbered list of added rules:

$ sudo ufw status numbered

You can also allow all ports in a range by specifying a port range:

$ sudo ufw allow 1050:5000/tcp

If you want to open all ports for a particular IP address, use the following command:

$ sudo ufw allow from 10.0.2.100

Alternatively, you can allow an entire subnet, as follows:

$ sudo ufw allow from 10.0.2.0/24

You can also allow or deny a specific port for a given IP address:

$ sudo ufw allow from 10.0.2.100 to any port 2222

$ sudo ufw deny from 10.0.2.100 to any port 5223

To specify a protocol in the preceding rule, use the following command:

$ sudo ufw deny from 10.0.2.100 proto tcp to any port 5223

Deleting rules:

$ sudo ufw delete allow ftp

Delete rules by specifying their numbers:

$ sudo ufw status numbered

$ sudo ufw delete 2

Add a new rule at a specific number:

$ sudo ufw insert 1 allow 5222/tcp # Inserts a rule at number 1

19.

If you want to reject outgoing FTP connections, you can use the following command:

$ sudo ufw reject out ftp

UFW also supports application profiles. To view all application profiles, use the following command:

$ sudo ufw app list

Get more information about the app profile using the following command:

$ sudo ufw app info OpenSSH

Allow the application profile as follows:

$ sudo ufw allow OpenSSH

Set ufw logging levels [off|low|medium|high|full] with the help of the following command:

$ sudo ufw logging medium

View firewall reports with the show parameter:

$ sudo ufw show added # list of rules added

$ sudo ufw show raw # show complete firewall

Reset ufw to its default state (all rules will be backed up by UFW):

$ sudo ufw reset

There's more…

UFW also provides various configuration files that can be used:

/etc/default/ufw: This is the main configuration file.

/etc/ufw/sysctl.conf: These are the kernel network variables. Variables in this file override variables in /etc/sysctl.conf.

/var/lib/ufw/user[6].rules or /lib/ufw/user[6].rules are the rules added via the ufw command.

/etc/ufw/before.init are the scripts to be run before the UFW initialization.

/etc/ufw/after.init are the scripts to be run after the UFW initialization.

See also

Check logging section of the UFW community page for an explanation of UFW logs at https://help.ubuntu.com/community/UFW

Check out the UFW manual pages with the following command:

$ man ufw

We have installed the XMPP server, Ejabberd. In this recipe, we will learn how to add new user accounts to the Ejabberd server. We will also learn how to configure the XMPP client and connect to our server.

Getting ready

Make sure that you have installed the Ejabberd server and it is running properly.

Additionally, you will need XMPP client software. You can choose from multiple free and open source clients such as pidgin, PSI, Adium, Gajim, and many more. I will be using PSI as it provides various low-level administrative features.

How to do it…

Ejabberd supports multiple methods for registering a new user account. These include adding a new user from the command line, creating a new user from the admin panel, and allowing clients to register with the server using in-band registration. Here, we will create a new user from a command line admin tool. Later in this recipe, I will briefly explain another two methods.

Follow these steps to create a user account and connect it with a XMPP client:

Use the following command to register a new user using the ejabberdctl command:

$ # ejabberdctl register username host password

$ sudo ejabberdctl register user1 ubuntu password

You can get a list of registered users with the registered_users option to ejabberdctl:

$ # ejabberdctl registered_users host

$ sudo ejabberdctl registered_users ubuntu

Now you can create a connection to the server with the XMPP client and your new account. Download and install the XMPP client tool, PSI.

Open PSI, click the General tab, and then select Account Setup. This will open the XMPP Accounts window, which looks something like this:

Click the Add button in the XMPP Accounts window. This will open another window named Add Accounts:

Now, in the Add Account window, enter the name for this connection, or you can choose to keep the name as Default. Click the Add button to open one more window.

In the newly opened window, enter the account details that we created with the ejabberdctl command:

On the Account tab, enter the full XMPP address (JID) and password for your account.

Click on the Connection tab, then click to check the Manually Specify Server Host/Port: checkbox, and then enter the server IP or FQDN and change the port to match your configuration:

Next, click the Save button to complete the account setup and then click Close to close the account setup window. Your account will be listed in the main window of Psi, as follows:

Now you are ready to connect to your XMPP server. Select the listed account and change the drop-down box at the bottom to Online. This will start the connection process and set the user status as Online.

The PSI client will show a prompt regarding self-signed certificates if you are using the default certificate provided by Ejabberd. Click Trust this certificate to proceed.

It will take a few seconds to complete the connection process. Once connected, your PSI status will change to Online:

Now click General menu to add XMPP contacts or to join a group chat or to send a message to existing contact. To change your Instant Messaging account status, click on the Status menu and select your desired option.

How it works…

The preceding example demonstrates the account creation and client setup process for connecting with the XMPP server. We have used an administrative command to create an XMPP account and then configured client software to use the existing account.

You can also create a new account from the Ejabberd web console. The web console lists all the configured hostnames under the Virtual Hosts section, and each host lists options for user and access management, and other administration tools. Both these options need the server administrator to create an account.

Additionally, XMPP supports an extension that enables a user to self-register with the server. This is called in-band registration (xep-0077), where a user can send his registration request with his desired username, password, and other details, such as email, and the server creates a new user account. This is useful with public XMPP servers where administrators cannot handle all registration requests. The Ejabberd server supports in-band registration with the mod_register plugin, which is enabled by default. From the client side, you can use any XMPP client that supports in-band registration. If you have noticed, PSI also supports in-band registration and provides an option to register a new account in the Add Account process:

There's more…

When it is an XMPP administration task, PSI is a handy tool. It provides a debug console where you can monitor all XML data transfers between the client and server, as well as send arbitrary XML stanzas to the server. You can access the XML console from right-clicking the menu of your PSI account. Once opened, check Enable checkbox to enable traffic monitoring. The XML Console looks similar to the following screenshot:

XML Console also allows the filtering of traffic based on packet type. Button Dump Ringbuf can be used to dump any traffic before opening the XML Console.

Another option is service discovery from the right-click menu. You need to log in as an administrator to see all the options under service discovery. From here, you can monitor user accounts and various services that are available on the server. The Service Discovery window looks something like this:

See also

A list of XMPP client tools at https://xmpp.org/xmpp-software/clients/

We have created users and groups. In this recipe, you will work with default file permissions for users and groups, as well as see how to modify those permissions.

Getting ready

Create two users, user1 and user2. Create new group editor and add user1 and user2 as members.

How to do it…

Follow these steps to manage file permissions, follow these steps:

To change groups for files and directories:

Log in with user1.

Create a new directory documents under home:

user1@ubuntu:~$ mkdir documents

Create a text file under documents:

user1@ubuntu:~$ echo "hello world"> documents/file.txt

Now log in with user2:

user1@ubuntu:~$ su user2

Try to edit the same text file. It should say Permission denied:

user2@ubuntu:/home/user1$ echo "hello again"> documents/file.txt

log in as user1 and change the group of documents to editor:

user1@ubuntu:~$ chgrp -R editor documents

Switch to user2 and try editing the same file. Now it should work:

To set permissions with chmod, follow these steps:

Create simple shell script with the following command:

$ echo 'echo "Hello World!!"'> hello.sh

Execute a shell script with the following command:

$ ./hello.sh

Set executable permission to hello.sh with the following command:

$ chmod u+x hello.sh

Check new permission with the following command:

$ ls -l

Execute hello.sh again:

To protect shared files with sticky bit, follow these steps:

Log in as user1 and set sticky bit for directory documents:

user1@ubuntu:~$ chmod +t documents

Log in as user2 and create a new file.

Try to delete any file under documents. It should fail:

How it works…

When you create a new file or directory in Ubuntu, the default permissions for files are read and write access to owner and owner's private group, along with read, write, and execute access for directories. You can check the default setting with umask -S.

In our example, we have user1 and user2. Both of them are members of the editor group. When user1 creates a file, the default permissions are limited to user1 and its private group (user1) named after the user account. This is the reason user2 sees Permission denied on editing file. By changing the group of documents to editor we allow all members of editor to read and write to files in documents.

With the chmod command, we can set permissions at a more granular level. In our example of hello.sh, we have set the executable permission for hello.sh. Similarly, we can set read permission as follows:

$chmod +r filename

To set write permission, use the following command:

$chmod +w filename

You can set more selective permissions with additional parameters before mode expression as follows:

$chmod ugo+x filename

Here, u sets the permission for user, g for group, and o for all others.

To remove permissions, replace + with -. For example, $chmod o-w filename. Alternatively, you can use the Octal format to specify permissions:

$chmod 777 filename

This gives read, write, and execute permission to user group and others, whereas the command $chmod 600 filename gives set, read, and write permissions for owner and no permission to groups and others. In Octal format [777], the first bit is used for the user or owner of the file, the second bit is for group, and the third bit is for everyone else. Check out the following table for more information:

Finally, when you share files within a group of users, there are chances that someone deletes the file that is required by other users. Sticky bit can protect these file from deletion. When sticky bit is set, only the owner or a user with root privileges can delete a file.

You can set sticky bit with the command chmod as $chmod +t directoryName. Sticky bit is shown in long listing (ls -l) with symbol t or T. Additionally, sticky bit works only with directories and is ignored on ordinary files.

There's more…

Many times when working as a root user, all files and directories created are owned by root. A non-root user can't write to these directories or files. You can use the command chown to change the ownership of such files and assign them to respective users.

To change ownership of a file, use the following command:

$chown newuser filename

To change the owner as well as the group of file, use the following command:

$chown newuser:newgroup filename

You can skip changing owner and change only the group with the following command:

$chown :newgroup filename

Note that the chown command can only be used by users with root privileges.

In last few recipes, we learned about LXD, an operating system container service. Now, we will look at a hot new technology called Docker. Docker is an application container designed to package and run a single service. It enables developers to enclose an app with all dependencies in an isolated container environment. Docker helps developers create a reproducible environment with a simple configuration file called a Dockerfile. It also provides portability by sharing the Dockerfile, and developers can be sure that their setup will work the same on any system with the Docker runtime.

Docker is very similar to LXC. Its development started as a wrapper around the LXC API to help DevOps take advantage of containerization. It added some restrictions to allow only a single process to be running in a container, unlike a whole operating system in LXC. In subsequent versions, Docker changed its focus from LXC and started working on a new standard library for application containers, known as libcontainer.

It still uses the same base technologies, such as Linux namespaces and control groups, and shares the same kernel with the host operating system. Similarly, Docker makes use of operating system images to run containers. Docker images are a collection of multiple layers, with each layer adding something new to the base layer. This something new can include a service, such as a web server, application code, or even a new set of configurations. Each layer is independent of the layers above it and can be reused to create a new image.

Being an application container, Docker encourages the use of a microservice-based distributed architecture. Think of deploying a simple WordPress blog. With Docker, you will need to create at least two different containers, one for the MySQL server and the other for the WordPress code with PHP and the web server. You can separate PHP and web servers in their own containers. While this looks like extra effort, it makes your application much more flexible. It enables you to scale each component separately and improves application availability by separating failure points.

While both LXC and Docker use containerization technologies, their use cases are different. LXC enables you to run an entire lightweight virtual machine in a container, eliminating the inefficiencies of virtualization. Docker enables you to quickly create and share a self-dependent package with your application, which can be deployed on any system running Docker.

In this recipe, we will cover the installation of Docker on Ubuntu Server. The recipes after that will focus on various features provided by Docker.

Getting ready

You will need access to the root account or an account with sudo privileges.

How to do it…

Recently, Docker released version 1.11 of the Docker engine. We will follow the installation steps provided on the Docker site to install the latest available version:

First, add a new gpg key:

$ sudo apt-key adv --keyserver hkp://p80.pool.sks- keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

Next, add a new repository to the local installation sources. This repository is maintained by Docker and contains Docker packages for 1.7.1 and higher versions:

$ echo "deb https://apt.dockerproject.org/repo ubuntu-xenial main" | \

sudo tee /etc/apt/sources.list.d/docker.list

Next, update the apt package list and install Docker with the following commands:

$ sudo apt-get update

$ sudo apt-get install docker-engine

Once the installation completes, you can check the status of the Docker service, as follows:

$ sudo service docket status

Check the installed Docker version with docker version:

$ sudo docker version

Client:

Version: 1.11.1

API version: 1.23

...

Server:

Version: 1.11.1

API version: 1.23

...

Download a test container to test the installation. This container will simply print a welcome message and then exit:

$ sudo docker run hello-world

At this point, you need to use sudo with every Docker command. To enable a non-sudo user to use Docker, or to simply avoid the repeated use of sudo, add the respective usernames to the docker group:

$ sudo gpasswd -a ubuntu docker

Now, update group membership, and you can use Docker without the sudo command:

$ newgrp docker

How it works…

This recipe installs Docker from the official Docker repository. This way, we can be sure to get the latest version. The Ubuntu 16.04 repository also contains the package for Docker with version 1.10. If you prefer to install from the Ubuntu repository, it's an even easier task with a single command, as follows:

$ sudo apt-get install docker.io

As of writing this, Docker 1.11 is the latest stable release and the first release to have been built on Open Container Initiative standards. This version is built on runc and containerd.

There's more…

Docker provides a quick installation script, which can be used to install Docker with a single command. This scripts reads the basic details of your operating system, such as the distribution and version, and then executes all the required steps to install Docker. You can use the bootstrap script as follows:

$ sudo curl -sSL https://get.docker.com | sudo sh

Note that with this command, the script will be executed with sudo privileges. Make sure you cross-check the script's contents before executing it. You can download the script without executing it, as follows:

$ curl -sSL https://get.docker.com -o docker_install.sh

See also

The Docker installation guide: http://docs.docker.com/installation/ubuntulinux/

Operating system containers versus application containers: https://blog.risingstack.com/operating-system-containers-vs-application-containers/

What Docker adds to lxc-tools: http://stackoverflow.com/questions/17989306/what-does-docker-add-to-lxc-tools-the-userspace-lxc-tools

A curated list of Docker resources: https://github.com/veggiemonk/awesome-docker

Cloud computing has become the most important terminology in the computing sphere. It has reduced the effort and cost required to set up and operate the overall computing infrastructure. It has helped various businesses quickly start their business operations without wasting time planning their IT infrastructure, and has enabled really small teams to scale their businesses with on-demand computing power.

The term cloud is commonly used to refer to a large network of servers connected to the Internet. These servers offer a wide range of services and are available for the general public on a pay-per-use basis. Most cloud resources are available in the form of Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). A SaaS is a software system hosted in the cloud. These systems are generally maintained by large organizations; a well-known example that we commonly use is Gmail and the Google Docs service. The end user can access these application through their browsers. He or she can just sign up for the service, pay the required fees, if any, and start using it without any local setup. All data is stored in the cloud and is accessible from any location.

PaaS provide a base platform to develop and run applications in the cloud. The service provider does the hard work of building and maintaining the infrastructure and provides easy-to-use APIs that enable developers to quickly develop and deploy an application. Heroku and the Google App Engine are well-known examples of PaaS services.

Similarly, IaaS provides access to computing infrastructure. This is the base layer of cloud computing and provides physical or virtual access to computing, storage, and network services. The service builds and maintains actual infrastructure, including hardware assembly, virtualization, backups, and scaling. Examples include Amazon AWS and the Google Compute Engine. Heroku is a platform service built on top of the AWS infrastructure.

These cloud services are built on top of virtualization. Virtualization is a software system that enables us to break a large physical server into multiple small virtual servers that can be used independently. One can run multiple isolated operating systems and applications on a single large hardware server. Cloud computing is a set of tools that allows the general public to utilize these virtual resources at a small cost.

Ubuntu offers a wide range of virtualization and cloud computing tools. It supports hypervisors, such as KVM, XEN, and QEMU; a free and open source cloud computing platform, OpenStack; the service orchestration tool Juju and machine provisioning tool MAAS. In this article, we will take a brief look at virtualization with KVM. We will install and set up our own cloud with OpenStack and deploy our applications with Juju.

MySQL performance optimizations can be divided into two parts. One is query optimization and the other is MySQL server configuration. To get optimum results, you have to work on both of these parts. Without proper configuration, queries will not provide consistent performance; on the other hand, without proper queries and a database structure, queries may take much longer to produce results.

In this recipe, we will learn how to evaluate query performance, set indexes, and identify the optimum database structure for our data.

Getting ready

You will need access to an admin account on the MySQL server.

You will need a large dataset to test queries. Various tools are available to generate test data. I will be using test data available at https://github.com/datacharmer/test_db .

How to do it…

Follow these steps to optimize MySQL performance:

The first and most basic thing is to identify key columns and add indexes to them:

mysql> alter table salaries add index (salary);

Enable the slow query log to identify long-running queries. Enter the following commands from the MySQL console:

mysql> set global log_slow_queries = 1;

mysql> set global slow_query_log_file = ‘/var/log/mysql/slow.log’;

Once you identify the slow and repeated query, execute that query on the database and record query timings. The following is a sample query:

mysql> select count(*) from salaries where salary between 30000 and 65000 and from_date > ‘1986-01-01’;

Next, use explain to view the query execution plan:

mysql> explain select count(*) from salaries where salary between 30000 and 65000 and from_date > ‘1986-01-01’;

Add required indexes, if any, and recheck the query execution plan. Your new index should be listed under possible_keys and key columns of explain output:

mysql> alter table `salaries` add index ( `from_date` ) ;

If you found that MySQL is not using a proper index or using another index than expected then you can explicitly specify the index to be used or ignored:

mysql> select * from salaries use index (salaries) where salary between 30000 and 65000 and from_date > ‘1986-01- 01’;

mysql> select * from salaries where salary between 30000 and 65000 and from_date > ‘1986-01-01’ ignore index (from_date);

Now execute the query again and check query timings for any improvements.

Analyze your data and modify the table structure. The following query will show the minimum and maximum length of data in each column. Add a small amount of buffer space to the reported maximum length and reduce additional space allocation if any:

mysql> select * from `employees` procedure analyse();

The following is the partial output for the analyse() procedure:

Check the database engines you are using. The two major engines available in MySQL are MyISAM and InnoDB:

mysql> show create table employees;

How it works…

MySQL uses SQL to accept commands for data processing. The query contains the operation, such as select, insert, and update; the target that is a table name; and conditions to match the data. The following is an example query:

select * from employee where id = 1001;

In the preceding query, select * is the operation asking MySQL to select all data for a row. The target is the employee table, and id = 1001 is a condition part.

Once a query is received, MySQL generates query execution plan for it. This step contains various steps such as parsing, preprocessing, and optimization. In parsing and pre-processing, the query is checked for any syntactical errors and the proper order of SQL grammar. The given query can be executed in multiple ways. Query optimizer selects the best possible path for query execution. Finally, the query is executed and the execution plan is stored in the query cache for later use.

The query execution plan can be retrieved from MySQL with the help of the explain query and explain extended. Explain executes the query until the generation of the query execution plan and then returns the execution plan as a result. The execution plan contains table names used in this query, key fields used to search data, the number of rows needed to be scanned, and temporary tables and file sorting used, if any. The query execution plan shows possible keys that can be used for query execution and then shows the actual key column used. Key is a column with an index on it, which can be a primary index, unique index, or non-unique index. You can check the MySQL documentation for more details on query execution plans and explain output.

If a specific column in a table is being used repeatedly, you should consider adding a proper index to that column. Indexes group similar data together, which reduces the look up time and total number of rows to be scanned. Also keep in mind that indexes use large amounts of memory, so be selective while adding indexes.

Secondly, if you have a proper index set on a required column and the query optimization plan does not recognize or use the index, you can force MySQL to use a specific index with the USE INDEX index_name statement. To ignore a specific index, use the statement IGNORE INDEX index_name.

You may get a small improvement with table maintenance commands. Optimize table is useful when a large part of the table is modified or deleted. It reorganizes table index data on physical storage and improves I/O performance. Flush table is used to reload the internal cache. Check table and Analyze table check for table errors and data distribution respectively. The improvements with these commands may not be significant for smaller tables. Reducing the extra space allocated to each column is also a good idea for reducing total physical storage used. Reduced storage will optimize I/O performance as well as cache utilization.

You should also check the storage engines used by specific tables. The two major storage engines used in MySQL are MyISAM and InnoDB. InnnoDB provides full transactional support and uses row-level locking, whereas MyISAM does not have transaction support and uses table-level locking. MyISAM is a good choice for faster reads where you have a large amount of data with limited writes on the table. MySQL does support the addition of external storage engines in the form of plugins. One popular open source storage engine is XtraDB by Percona systems.

There’s more…

If your tables are really large, you should consider partitioning them. Partitioning tables distributes related data across multiple files on disk. Partitioning on frequently used keys can give you a quick boost. MySQL supports various different types of partitioning such as hash partitions, range partitions, list partitions, key partitions, and also sub-partitions.

You can specify hash partitioning with table creation as follows:

create table employees (

id int not null,

fname varchar(30),

lname varchar(30),

store_id int

) partition by hash(store_id) partitions 4;

Alternatively, you can also partition an existing table with the following query:

mysql> alter table employees partition by hash(store_id) partitions 4;

Sharding MySQL

You can also shard your database. Sharding is a form of horizontal partitioning where you store part of the table data across multiple instances of a table. The table instance can exist on the same server under separate databases or across different servers. Each table instance contains parts of the total data, thus improving queries that need to access limited data. Sharding enables you to scale a database horizontally across multiple servers.

The best implementation strategy for sharding is to try to avoid it for as long as possible. Sharding requires additional maintenance efforts on the operations side and the use of proxy software to hide sharding from an application, or to make your application itself sharding aware. Sharding also adds limitations on queries that require access to the entire table. You will need to create cross-server joins or process data in the application layer.

See also

The MySQL optimization guide at https://dev.mysql.com/doc/refman/5.6/en/optimization.html

MySQL query execution plan information at https://dev.mysql.com/doc/refman/5.6/en/execution-plan-information.html

InnoDB storage engine at https://dev.mysql.com/doc/refman/5.6/en/innodb-storage-engine.html

Other storage engines available in MySQL at https://dev.mysql.com/doc/refman/5.6/en/storage-engines.html

Table maintenance statements at http://dev.mysql.com/doc/refman/5.6/en/table-maintenance-sql.html

MySQL test database at https://github.com/datacharmer/test_db

So, we have installed and configured our own centralized auth server with LDAP. Now is the time to use LDAP to authenticate client logins. In this recipe, we will set up a separate Ubuntu server to use our LDAP server for authenticating users.

Getting ready

You will need a new Ubuntu server to be set as an LDAP client. Also, sudo privileges are needed for the initial setup.

Make sure you have followed the previous recipes and have set up your LDAP server.

How to do it…

We will need to install the LDAP client-side package on the client system. This package will install all the required tools to authenticate with the remote LDAP server:

$ sudo apt-get update

$ sudo apt-get install ldap-auth-client nscd

The installation process will ask you some questions regarding your LDAP server and its authentication details. Answer those questions as follows:

LDAP server URI: ldap://you-LDAP-server-IP: Make sure you change the protocol line from ldapi:/// to ldap://

Distinguished name of search base: Match this to the domain set on the LDAP server in the format dc=example,dc=com

LDAP version to use: 3

Make local root database admin: Yes

Does LDAP database require login: No

LDAP account for root: cn=admin,dc=example,dc=com

LDAP root account password: The password for the LDAP admin account

Next, we need to change the authentication configuration to check with the LDAP server. First, run the following command to set the name service switch file /etc/nsswitch.conf:

$ sudo auth-client-config -t nss -p lac_ldap

This will change /etc/nsswitch.conf as follows:

Next, add the following line to /etc/pam.d/common-session. This will create a local home directory for LDAP users. Edit the common-session file and add the following line at the end of the file:

session required pam_mkhomedir.so umask=0022 skel=/etc/skel

Now restart the nscd service with the following command:

$ sudo /etc/init.d/nscd restart

Now you should be able to log in with the user account created on your LDAP server. I have set up an Organizational Unit (OU) named users and created an admin user under it:

Next, change the login to the newly created LDAP user account with the su username command. You will need to enter a password that is configured on LDAP server. As this is a first-time login for this new user, our PAM settings have created a new home directory for him:

This new user is a member of the admin group on the LDAP server, so he will get sudo privileges on the local server as well.

You can always use a default login prompt to log in with LDAP users, as well as local user accounts that already exist on the server.

How it works…

Here we have configured the Ubuntu server to authenticate with our centralized LDAP system. This is not limited to the Ubuntu server and you can configure the Ubuntu desktop in a similar way as well. Using a centralized authentication makes it easy to administer hundreds of user accounts from a single place. A user can still log in as a local user if he has any local credentials.

Using centralized authentication enables you to log in from any system. You will get the same access rights and permissions from any terminal. Additionally, if the LDAP configuration supports roaming profiles then all your data will be replicated to any new system you log in from. You may have noticed the home directory for the LDAP user account is located in the /home/users directory and not in /home. This separates your account from any local users.

Finally, the groups and roles configured on the LDAP server also apply on the system you are logging in from. So, if the user is assigned admin rights on the LDAP server, he will get admin rights, including sudo privileges, on the system he is logged in from. This is because Ubuntu contains a default group named admin with sudo privileges. When a user logs in with his LDAP account, the groups and roles assigned to his LDAP account are matched with local groups and roles. You can either disable such groups from any remote systems, or set the proper access rights on the LDAP server itself.

See also

The Ubuntu community page for LDAP client authentication at https://help.ubuntu.com/community/LDAPClientAuthentication

In this article, we will cover the following recipes:

• Installing the VNC Server
• Installing Hackpad, a collaborative document editor
• Installing Mattermost – a self-hosted slack alternative
• Installing OwnCloud, self-hosted cloud storage