Skip to main content

Resources

Debian environments

Debian includes a wide variety of software as possible. As of this writing, the Debian stable distribution contains over 48,000 pre-compiled in the latest stable version. According to some counts, this is more than any other Linux distribution. To be fair, many of these are niche applications that do not have a wide user base. But the number of packages is only a part of the story.

The support of many different environments is also a distinguishing characteristic of Debian distributions, and probably one of the most startling. In fact, Debian is unique in the number of different processors supported. At the time of writing, they include both 32-bit and 64-bit Intel and AMD chips, ARM (EABI or little endian version), Intel Itanium, MIPS (both big and little endian), PowerPC (yes, this means it will run on ffiM servers!), System/390 (the old ffiM architecture), and SPARe. In addition, the Alpha architecture was supported up until Debian 6.0, and there are unofficial ports to other ARM architectures as well as Amtel's RISC chip (A VR32), HP's PA-RISe chip (up until Debian 6.0), the Motorola 68000, ffiM system Z, and Hitachi SuperH processors. There is also support for FreeBSD as the primary operating system instead of Linux on Intel 32-bit and 64-bit architectures, and there are other unofficial or experimental non-Linux-based Debian distributions for the GNU Hurd operating system.

This commitment results in a distribution that is extremely flexible, which can be used in a great many environments. Because of this, the Debian developers have chosen not to design a default installation package suitable for most users. A default Debian installs (with no optional software selected) includes only the basics. The administrator is expected to select as options, or install later, the appropriate software. This is not difficult as the base system includes everything necessary to easily install additional software.

This contrasts with Ubuntu Linux, which is based on Debian. A basic Ubuntu installation is designed to work out of the box for the majority of users. Thus, it includes more software, making it an appropriate distribution for a new Linux user without extensive knowledge of what may be available, or a preference for exploring what is there, as opposed to wading through packages offered for optional installation. However, this may also result in an installation with unnecessary components. Of course, they may be easily removed, but it is another example of choosing the proper distribution to reduce the administrator's workload.

This is one reason Debian is one of the major players in commercial servers, as only the software and services necessary are installed, which generally leads to better performance and simpler system management. This also means that Debian will run acceptably on older, poorer performing equipment. Note that, in spite of this, it is also most certainly possible to install a wide variety of software, both during and after installation, which will allow a Debian system to fulfill even the most insatiable developers.

Summary of Basic Debian Package Configuration

Although Debian stays close to the upstream software developer's configuration methods, the Debian developers frequently modify configuration file layout to simplify local configuration and upgrades, and often provide additional utilities to ease the administrator's job. Third-party utilities are sometimes available as well, but the administrator should understand that the output of these utilities may not always follow the Debian configuration layout.

Now, with basic software configuration out of the way, it's time to cover more system wide administration issues.

/etc/<package name>

Configuration files are placed in / etc by most software developers, and this is standard practice for Debian packages. If the file is fairly simple, and the only file required, it is often placed directly in / etc with a .conf extension, such as / etc/ inetd. conf for the openbsd-inetd package. For packages that require a more complex configuration, usually via multiple configuration files, those files are placed in a subdirectory of /etc which is generally named after the package. A good example of this is the apache2 package, which has its configuration files in / etc/ apache2 and its subdirectories.

Multiple files in a subdirectory are used for a number of reasons. In some cases, it is because the original software uses different files for different purposes. In other cases, it is done to organize configuration items into a more manageable and understandable group. Occasionally, Debian developers have modified the package to organize the configuration to simplify or streamline administration.

A good example of this is Apache. Normal Apache configuration is either by a single file or multiple files that are included by the master configuration file. Traditionally, SSL (Secure Sockets Layer) certificates and keys, and Virtual Host configuration files are contained in a subdirectory, but all major options are configured in one or a few files in the main directory. Debian has actually split the configuration into many files in various subdirectories. This allows Debian to supply separate packages that provide certain features via individual configuration files that are added to the proper directory, rather than requiring modification of the primary configuration files. Also, there are subdirectories for modules and virtual hosts prefixed by available-or enabled-. This allows a simple script to activate or deactivate modules and virtual hosts easily and quickly.

As it's not yet available in the stable release, Debian developers are taking this concept further in future releases, by providing a config-available and config-enabled directory, as well as allowing features to be activated and deactivated without editing config files or installing or removing feature packages. This will probably be available in Debian 8, code-named Jessie.

Webmin and Debian

Webmin understands the configuration file layout of many distributions. The main problem is when a particular module does not handle certain types of configuration in the way the Debian developers prefer, which can make package upgrades somewhat difficult.

This can be handled in a couple of ways. Most modules provide a means to edit configuration files directly, so if you have read the Debian documentation you can modify the configuration appropriately to use Debian specific configuration techniques. Or, you may choose to allow Webmin to modify files as it sees fit, and handle any conflicts manually when you upgrade the software involved. Finally, you can avoid those modules involved with specific software that are more likely to cause problems.

Webmin modules are constantly changing, and some actually recognize the Debian file layouts well, so it is not possible to give a comprehensive list of modules to avoid at this time.

Best practice when using Webmin is to read the documentation and check the configuration files for specific software prior to using Webmin. Then, after configuring with Webmin, check the files again to determine whether changes may be required to work within the particular package's Debian configuration framework. Based upon this, you can decide whether to continue to configure using Webmin or switch back to manual configuration of that particular software.

How to Use Webmin in Debian Operating System

Webmin runs in the background, and provides an H1TP or H1TPS server on localhost port 10,000. You can use any web browser to connect to http:// loealhost: 10000/ to access Webmin. Upon first installation, only the root user or those in a group allowed to use sudo to access the root account, may log in but Webmin users can be managed separately or in conjunction with local users.

Webmin provides extensive and easy to understand menus and icons for various configuration tasks. Webmin is also highly modular and extensible, and an extensive list of standard modules is included with the base package. Itis not possible to cover Webmin as fully here as it deserves, but a short list of some of its capabilities includes:

  • Configuration of Webmin itself (the server, users, modules, and security)
  • Local system user and password management
  • Filesystem management
  • Bootup and service management
  • CRON job management
  • Software updates
  • Basic filesystem backups
  • Authentication and security configuration
  • APAo-IE, DNS, SSH, and FTP (if you're using ProFTP) configuration
  • User mail management
  • Qmail or sendmail configuration
  • Network and Firewall configuration and management
  • Bandwidth monitoring
  • Printer management

There are even modules that apply to clusters. Also, Webmin can search and allow access to other Webmin servers on the local network or you can define remote servers manually. This allows a central Webmin server, installed on a particular system, to be the gateway to all of the other servers in your environment, essentially providing a single point of access to manage all Webmin enabled servers.

Debian Selection lists

Both aptitude and dselect provide a basic, interactive interface as well. You can navigate through a list of available packages, classified according to sections, or you can search for packages using a number of criteria. The interface is based on the simple curses library, and can seem cumbersome at times, although it is an improvement over the command line utilities mentioned previously. One advantage of these interfaces (as well as the command line utilities) is that they can be used in a terminal environment and do not require a graphic desktop environment be installed. They are frequently used on high-performance servers where a graphic desktop environment is not installed for security or performance reasons.

On the other hand, Synaptic provides a full GUI interface for browsing, searching, and selecting packages, as well as configuring repositories, selecting installation options, and providing information on available and installed packages. It requires a graphic desktop environment, such as GNOME or KDE, in order to operate.

Synaptic can be run remotely, over a Secure Shell connection, from " a system that does have a graphic window manager installed. Some administrators install Synaptic on servers without graphical desktops and use it in this manner to avoid security or performance issues of graphic environments on the server itself.

Directory encryption

An alternative to encrypting full partitions or disks is to encrypt portions of a filesystem, usually a directory and everything below it in the hierarchy, by using special features of Linux so that the encryption and decryption are handled automatically by the kernel or special software in the background. Thus, there are no implications for booting (as long as the boot directory isn't encrypted) and no installation issues, as it is configured after installation. Directory encryption is appropriate for servers containing sensitive information that resides in certain parts of the directory hierarchy.

Debian provides several packages for this type of encryption. The two most common are encls and cryptsetup.

Default user group of Debian

Traditionally, there are two ways to set up the default group assigned to a new user. The first is to have everyone assigned to a single users group. This will allow all users on the system access files and directories with group read (and execute) permissions, and write to files with group write permissions.

The other way is to give each user his own groups, usually with the same name as his user ID. This is the default in Debian, and this scheme is often referred to as user private groups. Ifuser A wants others to be able to access certain files or directories via the group permissions, someone with system privileges that allow group modification must add the appropriate user IDs to user A's group.

At one point, a single users group was common practice. Best practice at present, though, is to use user private groups.

This is similar to the most common way to handle developer's access to each other's files. In this case, though, a new group, such as web-dev or whatever you feel is an appropriate group name, is created and the developers are assigned to it as necessary. A similar technique is used for some administration tasks as well. For example, an ftpadmin group that is allowed to modify the FTP server directories and files in ways that the normal (and especially the anonymous) ftp user cannot. Careful creation and assignment of groups can facilitate proper user access to files while preventing undesired access or modification.

User and group maintenance is handled by the standard command-line functions, such as useradd, usermod, userdel, groupadd, groupmod, and groupdel, all documented in the passwd package. Of course, there are GUI utilities available for user and group maintenance as well.

Reiser File System

Reiser File System (ReiserFS) was intended to supplant ext3 as the filesystem of choice for Linux, offering improved performance. At one point, ReiserFS version 3 was the default format choice for SuSE Linux. Version 4 was released, but development waned when the company went out of business, and SuSE eventually decided to go back to ext3 as its default.

ReiserFS offered some advantages over fonnats existing at the time, but it has fallen behind in some performance areas. It does support dynamic resizing, while other filesystems must be offline in order to be resized, or use a logical volume manager to provide virtual resizing support.

Partition resizing of Debian

Partition resizing may be required ifyou need more space. To get a quick overview of your disk space usage, use the df command. There is also a disk usage (du) command that summarizes file and directory space. Refer to the manual pages for details.

Ifyou must resize a disk partition, the procedures are fairly straightforward but there are some considerations.

There is always a risk in modifying your partition layout. Make sure you have current backups prior to modifying the layout.

If the partition is being expanded, there must be room to expand it. Ifyou are using Logical Volume Manager (LVM), this can be as simple as adding a physical volume to the volume group ifthere is not enough space left in it. If you are not using L VM, or ifyou have multiple partitions configured on the same logical volume, then there must be free space available on the physical disk or logical volume following the end of the partition to be expanded. Ifthere isn't, it must be created somehow. Ifthe partition is to be shrunk, space is not an issue, although you may want to move or expand other partitions to use the space freed up by shrinking. Also, it is sometimes necessary to move partitions around. This is not necessary ifyou use L VM and assign one partition per logical volume.

Moving or shrinking a partition may only be done with unmounted (offline) partitions, since itrequires moving files and risks corruption ifany files are in use. Some filesystems, EXT3 and EXT4 in particular, may be expanded while the partition is still mounted, however, since this doesn't require moving any files, just modifying the filesystem metadata.

While each filesystem architecture has appropriate tools for resizing and there are simple tools for managing the partition table, the primary tool for managing partitions is parted. There are other, mostly commercial tools, but parted does pretty much everything that is necessary, is available as an easy to use graphical utility (gparted), and it is free. There is even a live CD version you can boot to manage your partitions regardless of what operating system is installed. Many Windows administrators use it quite successfully. Ithandles both enlarging and shrinking partitions as well as moving them, and will handle many different filesystem architectures, and ensures that the partition table matches the filesystem sizes.

Unfortunately, neither parted nor gparted will not allow you to expand a mounted partition. Ifyou must expand a partition while it is in use, you must delete and recreate the partition with the same, identical beginning position, force a reread of the partition table, and then use the appropriate filesystem resizing commands. This is extremely dangerous and should never be used for resizing the root partition.

The procedures for modifying partition layout are straightforward. .ct Note that ifyou aren't booted from a gparted live CD or similar utility disk, you cannot unmount partitions required to run the necessary utilities and operating system services.

To manually modify your partition layout:

  1. Ensure there is adequate space for the expanded partition, or to hold the partition being moved. If necessary, shrink or move other partitions or expand the logical volume.
  2. You must ensure the partition table matches the resized or moved partition exactly. Failing to do this can result in corrupted or lost data. gparted will handle this for you automatically.
  3. To shrink a partition, it must be offline (unmounted). Resize the filesystem first, then modify the partition table to match the new size.
  4. To expand a partition, modify the partition table first, then expand the filesystem. Ifyou are doing this with a mounted partition, you must use the procedure outlined previously.
  5. To move a partition, you must first create a new partition of the same size as the original. Then you must unmount the current partition, copy the data to the new partition exactly, and delete the old partition.

In all cases, the partition must be clean prior to modifying or moving it, and should be checked afterwards prior to mounting it. The gparted utility or similar will handle the steps above automatically, including the necessary filesystem checks, data movement, and partition table modifications.

Note that either gparted or the manual procedures may be used on a Jive system, provided the partitions being modified can be unmounted, or if a mounted partition must be modified, itis only being expanded and gparted is not being used.

Unless you must expand a partition while it remains mounted and in use, the recommended procedure is to boot from a gparted live CD. lhis greatly simplifies the whole process, allows modification of all partitions, not just those that aren't required for operating system services and necessary utilities, and reduces the risk of corruption.

Repositories of Debian

Another result of Debian's licensing policies is the existence of three distinct software repositories:

  • main: These are packages whose license conforms to the DFSG
  • contrib: These packages have licenses that also conform to the DFSG, but that depend on other packages or libraries that do not
  • non-free: These are packages whose license does not conform to the DFSG but that are allowed to be distributed with Debian

Users are free to choose whether to allow software from the contrib and non-free classes to be installed. If it is installed, the users are responsible for knowing and following the appropriate licenses.

Other, non-official repositories also exist, which host software that, for one reason or another, isn't included in any of the official Debian repositories.

Other examples

The Apache example should give you a feel for how Debian splits configuration files into smaller, more easily managed files, and provides means to ease various administration tasks, such as activating and deactivation modules. Other software will have different layouts, as well as, different configuration features and utilities, all described in the appropriate documentation.

Exim, for example, uses a single, monolithic configuration file. In order to split files out, Debian provides a utility update-exim4. conf that combines all of the separate configuration files from the / etc/exim4 / conf . d hierarchy into the single file required by the exim4 software. There are several minor variations in how this may be done (all explained in the documentation), so that an administrator can choose his preferred method.

Debian sendmail, another email package, uses another method. All of its configuration files are in the /etc/mail hierarchy, and GNU make is used after updating the configuration to create all of the necessary database and config files. In fact, a special utility, sendmailconfig, will not only run make for you, it will also handle incorporating /etc/default/sendrnail changes, and reload the sendmail configuration after processing is complete.

All of this shows how the Debian developers provide various methods of making configuration easier. Simpler packages may have few, one, or no configuration files. Those with larger or more complex configuration won't all be handled identically, although the technique of splitting files into smaller, easier to modify pieces and providing utilities to ease config administration will be common to most (and, per Debian policy, described in the documentation for each package).

/etc/default

The configuration files in this directory are intended to control the standard initialization scripts found in /etc/init.d (known as init scripts), or the way various system libraries behave. They contain environment variable definitions that are used by system libraries or in the init scripts, to control execution options, software module loading, and anything else that can be controlled dynamically.

In the case of definitions used by the init scripts, the variables usually include a definition of command line options to be used. In some cases, setting one of the variables modifies the behavior of the script. For example, setting RESOLVCONF to anything but no in /etc/default/bind9 will cause the init script to use the resolvconf utility to add an appropriate resolver line for the localloopback interface during initialization (and remove it during shutdown).

For those scripts that apply to system libraries, they generally define default behavior for commands or software that the libraries affect. A good example of this is /etc/ default/nss, which controls how the Name Service Switch (NSS) functions in the GNU C library. In particular, it controls what information the library will consider authoritative for NSS calls, as well as the behavior of certain function calls.

In special cases, the default definition file may not be a single file, but " a subdirectory of /etc/default. TItis is done to split information into groups of related variables, both for ease of update, and to allow local modifications that won't be overwritten when configuration files are replaced.

Knowing and managing CentOS background services

Linux system services are one of the most fundamental concepts of every Linux server. They are programs which run continuously in your system, waiting for external events to process something or do it all the time. Normally, when working with your server, a system user will not notice the existence of such a running service because it is running as a background process and is therefore not visible. There are many services running all the time on any Linux server. These can be a web server, database, FTP, SSH or printing, DHCP, or LDAP server to name a few. In this, we will show you how to manage and work with them.

To Start With: What Do You Need?

To complete this process, you will require a working installation of the CentOS 7 operating system with root privileges, a console-based text editor of your choice, and a connection to the Internet to facilitate the download of additional packages. Some commands shown here use less navigation in their output. 

The Process

systemctl is a program that we will use to manage all our background service tasks in a CentOS 7 system. Here, we will show you how to use it, taking the Apache web server serves as an example in order to get familiar with it.

  1. First, we log in as root and install the Apache web server package:
    yum install httpd
  2. Next we will check Apache’s service status:
    systemctl status httpd.service
  3. Start the web server service in the background and print out its status again:
    systemctl start httpd.service
    systemctl status httpd.service
  4. Next, let’s print out a list of all services currently running in the background of your system; in this list, you should identify the httpd service you just started:
    systemctl -t service -a --state running
  5. Now, let’s make a backup of the Apache configuration file:
    cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.BAK
  6. Now, we will make some changes to the main Apache configuration file using sed:
    sed -i 's/Options Indexes FollowSymLinks/Options -Indexes
    +FollowSymLinks/g' /etc/httpd/conf/httpd.conf
  7. Now, type the following command to stop and start the service and apply our changes:
    systemctl stop httpd.service
    systemctl start httpd.service
    systemctl status httpd.service
  8. Next, let’s enable the httpd service to start automatically at boot time:
    systemctl enable httpd.service
  9. The last command will show how to restart a service:
    systemctl restart httpd.service 

How it works…

As we have seen, the systemctl utility can be used to take full control of your system’s services. The systemctl is the control program for systemd, which is the system and service manager in CentOS 7 Linux. The systemctl command can be used for a variety of other tasks as well, but here we concentrate on managing services.

So, what have we learned from this experience?

We started this process by logging in as root and installed the Apache web server package as we want to use it for showing how to manage services in general using the systemctl program. Apache or the httpd.service, as it is called by systemd, is just an example we will use; other important services that might be running in a basic server environment could be sshd.service, mariadb.service, crond.service, and so on. Afterward, we checked httpd’s current status with the systemctl status command parameter. The output showed us two fields: Loaded and Active. The Loaded field tells us if it is currently loaded and if it will automatically be started at boot time; the Active field denotes whether the service is currently running or not. Next, we showed how to start a service using systemctl. The command’s exact starting syntax for services is the systemctl start .service.

Note

By starting a service, the program gets detached from the terminal by forking off a new process that gets moved into the background where it runs as a non-interactive background process. This is sometimes called daemon.

Next, after we started the Apache webserver daemon, we then used systemctl’s status parameter again to show how the status changes if we run it. The output shows us that it is currently loaded but disabled on reboot. We also see that it is running, along with the latest logging output from this service and other detailed information about the process. To get an overview of all status information for all services on the system, use systemctl -type service --all.A systemctl service must not be running all the time. Its state can also be stopped, degraded, maintained, and so on. Next, we used the following command to get a list of all currently running services on your system:

systemctl -t service -a --state running

As you can see here, we used the -t flag in order to filter only for type service units. As you may guess, systemctl can not only deal with service units but also with a lot of other unit types. systemd units are resources systemd can manage using configuration files, and which encapsulate information about services, listening sockets, saved system state snapshots, mounting devices, and other objects that are relevant to the system. To get a list of all possible unit types, type systemctl -t help. These configuration unit files reside in special folders in the system, and the type they belong to can be read from the extension; all the service unit files have the file extension, .service (for example, device unit files have the extension, .device). There are two places where the system stores them. All the systemd unit files installed by the basic system during installation are in /usr/lib/systemd/system, all other services that come from installing packages such as Apache or for your own configurations should go to /etc/systemd/system. We can find our Apache service configuration file exactly at /usr/lib/systemd/system/httpd.service. Next, we showed the user how to stop a service, which is the opposite of starting it, using the syntax, systemctl stop . Finally, as a last step, we used systemctl’s restart parameter, which just handles the stopping and starting of service in one step with less typing. This is often useful if a service hangs and is unresponsive, and you quickly need to reset it to get it working. Before showing how to stop and restart a service, we did another important thing. While the Apache service was running, we changed its main service configuration file with the sed command, adding an -Indexes option that disables the directory web site file listings, and which is a common measure to increase the security of your web server. Since the Apache web server was already running and loading its configuration into memory during service startup, any changes to this file will never be recognized by the running service.

Note
Normally, to apply any configuration file change, running services need a full-service restart, because configuration files will normally only be loaded during startup initialization.

Now, imagine that your web server is reachable from the Internet and at the moment there are a lot of people accessing your web pages or applications in parallel. If you restart the Apache normally, the web server will be inaccessible for a while (as long as it takes to restart the server) as the process will actually end and afterward start all over again. All the current users would get HTML 404 error pages if they were to request something at that moment. Also, all the current session information would have gone; imagine you have an online webshop where people use shopping carts or logging in. All this information would also be gone. To avoid the disruption of important services such as the Apache web server, some of these services have a reload option (but not every service has this feature!) that we can apply instead of the restart parameter. This option just reloads and applies the service’s configuration file, while the service itself stays online and does not get interrupted during execution. For Apache, you can use the following command-line: systemctl reload httpd.service. To get a list of all the services that have the reload functionality, use the following lines:
grep -l "ExecReload" /usr/lib/systemd/system/*.service
/etc/systemd/system/*.service

So, having completed this recipe, we can say that we now know how to work with the basic systemctl parameters to manage services. It can be a very powerful program and can be used for much more than only starting and stopping services. Also, in this recipe, we have used different names that all mean the same: system service, background process, or daemon.

There's more…

There is another important unit type called target. Targets are also unit files and there are quite a number of them already available in your system. To show them, use the following:
ls -a /usr/lib/systemd/system/*.target /etc/systemd/system/*.target

Simply said, targets are collections of unit files such as services or other targets. They can be used to create runlevel-like environments, which you may know from earlier CentOS versions. Runlevels define which services should be loaded at which system state. For example, there is a graphical state or a rescue mode state, and so on. To see how the common runlevels correspond to our targets, run the following command, which shows us all the symbolic links between them:
ls -al /lib/systemd/system | grep runlevel

Targets can be dependent on other targets; to get a nice overview of target dependencies, we can run the following command to show all dependencies from the multi-user target to all the other targets (green means active and red means inactive):
systemctl list-dependencies multi-user.target

You can show the current target that we are in at the moment with:
systemctl get-default

You can also switch to another target:
systemctl set-default multi-user.target

 

 

Creating CentOS USB installation media on Windows or OS X

Here in this process, we will learn how to create a USB installation media on Windows or OS X. Nowadays, more and more server systems, desktop PCs, and laptops get shipped without an optical drive. Installing a new operating system, such as CentOS Linux using USB devices gets essential for them as no other installation option is available, as there is no other way to boot the installation media. Also, installing CentOS using USB media can be considerably faster than using the CD/DVD approach.

To Start With: What Do You Need?

Before we begin, it is assumed that you have followed the previous process in which you were shown how to download a minimal CentOS image and confirm the checksum of the relevant image files. It is also assumed that all the downloads (including the downloaded ISO file) are stored on Windows in your C:\Users\\Downloads folder or if using an OS X system, in the /Users//Downloads folder. Next, you will need a free USB device which can be discovered by your operating system, with enough total space, and which is empty or with data on it that can be discarded. The total space of the USB device needed for preparing as an installation media for CentOS 7 for the minimal version must be roughly 700 megabytes. If you are working on a Windows computer, you will need a working Internet connection to download additional software. On OS X, you need an administrator user account.

The Process

To initiate this process, start up your Windows or OS X operating system, then connect a free USB device with enough capacity, and wait until it gets discovered by File Manager under Windows or Finder under OS X.

  1. On a Windows-based system, we need to download an additional software called dd. Visit http://www.chrysocome.net/dd in your favorite browser. Now download the latest dd-XX.zip file you can find there, with XX being the latest stable version number. For example, dd-0.5.zip.
  2. On Windows, navigate to your Downloads folder using File Manager. Here you will find the dd-05.zip file. Right-click on it and click on Extract All, and extract the dd.exe file without creating any subdirectory.
  3. On Windows, open the command prompt (typically found at Start | All Programs | Accessories | Command Prompt) and type the following commands:
    cd downloads dd.exe --list
  4. On OS X, open the program Finder | Applications | Utilities | Terminal, and then type the following commands:
    cd ~/Downloads diskutil list
  5. On Windows, to spot the name of the right USB device you want to use as installation media, look into the output of the command under the removable media section. Below that, you should find a line starting with Mounting on and then a drive letter, for example, \.\e:. This cryptic written drive letter is the most important part we need in the next step, so please write it down.
  6. On OS X, the device path can be found in the output of the former command and has the format of /dev/disk, where number is a unique identifier of the disk. The disks are numbered, starting with zero (0). Disk 0 is likely to be the OS X recovery disk, and disk 1 is likely to be your main OS X installation. To identify your USB device, try to compare the NAME, TYPE, and SIZE columns to the specifications of your USB stick. If you have identified the device name, write it down, for example, /dev/disk3.
  7. On Windows, type the following command, assuming your USB device selected as an installation media has the Windows device name \\.\e: (change this as required and be careful what you type – this can create tremendous data loss). Also, substitute XXXX with the correct iso file version number in the next command
    dd.exe if=CentOS-7-x86_64-Minimal-XXXX.iso of=\\.\e: bs=1M
  8. On OS X, you need two commands which will ask for the administrator password (replace XXXX and disk3 with the correct version number and the correct USB device path):
    sudo diskutil unmountDisk /dev/disk3
    sudo dd if=./CentOS-7-x86_64-Minimal-XXXX.iso of=/dev/disk3 bs=1m
  9. After the dd program finishes, there will be some output statistics on how long it took and how much data has been transferred during the copy process. On OS X, ignore any warning messages about the disk not being readable.
  10. Congratulations! You now have created your first CentOS 7 USB installation media. You now can safely remove the USB drive in Windows or OS X, and physically unplug the device and use it as a boot device for installing CentOS 7 on your target machine.

How it works...

So what have we learned from this experience?

The purpose of this process was to introduce you to the concept of creating an exact copy of a CentOS installation ISO file on a USB device, using the dd command-line program. The dd program is a Unix based tool which can be used to copy bits from a source to a destination file. This means that the source gets read bit by bit and written to a destination without considering the content or file allocation; it just involves reading and writing pure raw data. It expects two file name based arguments: input file (if) and output file (of). We will use the CentOS image file as our input filename to clone it exactly 1:1 to the USB device, which is accessible through its device file as our output file parameter. The bs parameter defines the block size, which is the amount of data to be copied at once. Be careful, it is an absolute expert tool and overwrites any existing data on your target while copying data on it without further confirmation or any safety checks. So at least double-check the device drive letters of your target USB device and never confuse them! For example, if you have a second hard disk installed at D: and your USB device at E: (on OS X, at /dev/disk2 and /dev/disk3 respectively) and you confuse the drive letter E: with

D: (or /dev/disk3 with /dev/disk2), your second hard disk would be erased with little to no chances of recovering any lost data. So handle with care! If you’re in doubt of the correct output file device, never start the dd program!

In conclusion, it is fair to say that there are other far more convenient solutions available for creating a USB installation media for CentOS 7 than the dd command, such as the Fedora Live USB Creator. But this process was not only to create a ready-to-use CentOS USB installer but also to get you used to the dd command. It’s a common Linux command that every CentOS system administrator should know how to use. It can be used for a broad variety of daily tasks. For example, for securely erasing hard disks, benchmarking network speed, or creating random binary files.

 

Creating your own images from Dockerfiles and uploading to Docker Hub on CentOS

Besides images and containers, Docker has a third very important term called a Dockerfile. A Dockerfile is like a process on how to create an environment for a specific application, which means that it contains the blueprint and exact description of how to build a specific image file. For example, if we would like to containerize a webserver-based application, we would define all the dependencies for it, such as the base Linux system that provides the system dependencies such as Ubuntu, Debian, CentOS, and so on (this does not mean we virtualize the complete operating system but just use the system dependencies), as well as all applications, dynamic libraries, and services such as PHP, Apache, and MySQL in the Dockerfile and also all special configuration options or environment variables. There are two ways to build your own custom images. One, you could download an existing base image as we did in the previous Wordpress process and then attach to the container using BASH, install your additional software, make the changes to your configuration files, and then commit the container as a new image to the registry. Alternatively, here in this process, we will teach you how to build your own Docker image from a new Dockerfile for an Express.js web application server and upload it to your own Docker Hub account.

To Start With: What Do You Need?

To complete this process, you will require a working installation of the CentOS 7 operating system with root privileges, a console-based text editor of your choice, and a connection to the Internet in order to communicate with the Docker Hub. It is expected that Docker is already installed and is running. Also, for uploading your new image to the Docker Hub, you need to create a new Docker Hub user account there. Just go to https://hub.docker.com/ and register there for free. In our example, we will use a fictitious new Docker Hub user ID called johndoe.

The Process

  1. To begin, log in as root and create a new directory structure using your Docker Hub user ID (substitute the johndoe directory name appropriately with your own ID), and open an empty Dockerfile where you put in your image’s building blueprint:
    mkdir -p ~/johndoe/centos7-expressjs
    cd $_; vi Dockerfile
  2. Put in the following content into that file:
    FROM centos:centos7
    RUN yum install -y epel-release;yum install -y npm;
    RUN npm install express --save
    COPY . ./src
    EXPOSE 8080
    CMD ["node", "/src/index.js"]
  3. Save and close the file. Now create your first Express.js web application, which we will deploy on the new container. Open the following file in the current directory:
    vi index.js
  4. Now put in the following JavaScript content:
    var express = require('express'), app = express();
    app.get('/', function (req, res) {res.send('Hello CentOS 7
    cookbook!\n');});
    app.listen(8080);
  5. Now to build an image from this Dockerfile, stay in the current directory and use the following command (don’t forget the dot at the end of this line and replace johndoe with your own Docker Hub ID):
    docker build -t johndoe/centos7-expressjs .
  6. After successfully building the image, let’s run it as a container:
    docker run -p 8081:8080 -d johndoe/centos7-expressjs
  7. Finally, test if we can make an HTTP request to our new Express.js web application server running in our new container:
    curl -i localhost:8081
  8. If the Docker image is successfully running on the Express.js server, the following HTTP response should occur (truncated to the last line):
    Hello CentOS 7 cookbook!

Uploading your image to the Docker Hub

  1. After creating a new Docker Hub account ID called johndoe, we will start to login to the site using the following command—stay in the directory where you put your Dockerfile from the last step–for example ~/johndoe/centos7-expressjs (provide the username, the password, and the registration e-mail when asked):
    docker login
  2. Now, to push your new image created in this process to the Docker Hub (again replace johndoe with your own user ID), use:
    docker push johndoe/centos7-expressjs
  3. After uploading, you will be able to find your image on the Docker Hub web page search. Alternatively, you can use the command line:
    docker search expressjs

How Does It Work?

Here in this short process, we showed you how to create your first Dockerfile which will create a CentOS 7 container to serve Express.js applications, which is a modern alternative to LAMP stacks where you program JavaScript on the client-and server-side.

So what did we learn from this experience?

As you can see, a Dockerfile is an elegant way to describe all the instructions on how to create an image. The commands are straight-forward to understand and you use special keywords to instruct Docker what to do in order to produce an image out of it. The FROM command tells Docker which base image we should use. Fortunately, someone has already created a base image from the CentOS 7 system dependencies (this will be downloaded from Docker Hub). Next, we used the RUN command, which just executes commands as on a BASH command-line. We use this command to install dependencies on our system in order to run Express.js applications (it’s based on the Node.js rpm package which we access by installing the EPEL repository first). The COPY command copies files from our host machine to a specific location on the container. We need this to copy our index.js file which will create all our Express.js web server code in a later step on to the container. EXPOSE, as the name implies, exposes an internal container port to the outside host system. Since by default Express.js is listening on 8080, we need to do this here. While all these commands shown up to this point will only be executed once when creating the image, the next command CMD will be run every time we start the container. The command node /src/index.js will be executed and instructs the system to start the Express.js web server with the index.js file (which we already provided in this directory by copying it from the host machine). We don’t want to go into any details about the JavaScript part of the program—it just handles HTTP GET requests and returns the Hello World string. In the second part of this process, we showed you how to push our new created image to the Docker Hub. In order to do so, login with your Docker user account. Then we can push our image to the repository.

As this is a very simple Dockerfile, there is much more to learn about this subject. To see a list of all the commands available in the Dockerfile, use man Dockerfile. Also, you should visit the Docker Hub and browse the Dockerfiles (under the section Source Repository hosted on GitHub) of some interesting projects to learn how to create some highly sophisticated image files with just a handful of commands on your own.

 

Installing a PostgreSQL server and managing a database in CentOS

In this process, we will not only learn how to install the PostgreSQL DBMS on our server, but we will also discover how to add a new user and create our first database. PostgreSQL is considered to be the most advanced open source database system in the world. It is known for being a solid, reliable, and well-engineered system that is fully capable of supporting high-transaction and mission-critical applications. PostgreSQL is a descendant of the Ingres database. It is community-driven and maintained by a large collection of contributors from all over the world. It may not be as flexible or as pervasive as MariaDB, but because PostgreSQL is a very secure database system that excels in data integrity, it is the purpose of this process to show you how to begin exploring this forgotten friend.

To Start With: What Do You Need?

To complete this process, you will require a working installation of the CentOS 7 operating system with root privileges, a console-based text editor of your choice, and a connection to the Internet in order to facilitate the download of additional packages. It is expected that your server will be using a static IP address.

The Process

PostgreSQL (also known as Postgres) is an object-relational database management system. It supports a large part of the SQL standard and it can be extended by the server administrator in many ways. However, in order to begin, we must start by installing the necessary packages:

  1. Start by logging in your server as root and type:
    yum install postgresql postgresql-server
  2. Having installed the database system, we must now enable the database server at boot by typing:
    systemctl enable postgresql
  3. When you have finished, initialize the database system as follows:
    postgresql-setup initdb
  4. Now complete this process by starting the database server:
    systemctl start postgresql
  5. Now set a new initial password for our postgres administrator of your choice. As the default postgres user is currently using peer authentication, we need to execute any Postgres-related command with user postgres:
    su -postgres -c "psql --command '\password postgres'"
  6. To get rid of the requirement, that the postgres user has to be logged in on a system user basis before he can execute Postgres-related commands such as psql, and to allow login with database user accounts in general, we need to change the authentication method for localhost from peer to md5 in the Postgres client authentication configuration file. You can do this manually or use the sed tool as shown next, after you have made a backup of the file first:
    cp /var/lib/pgsql/data/pg_hba.conf /var/lib/pgsql/data/pg_hba.conf.BAK
    sed -i 's/^\(local.*\)peer$/\1md5/g' /var/lib/pgsql/data/pg_hba.conf
  7. Next, we have to restart the postgresql service in order to apply our changes:
    systemctl restart postgresql
  8.  Now you will be able to login to your Postgres server with user postgres without the need to log in the postgres Linux system user first:
    psql -U postgres
  9. To exit the shell (postgres=#), type the following command (followed by the Return key):
    \q
  10. We will now issue a shell command to create a new database user, by substituting with a relevant user name to fit your own needs (type in a new password for the user when prompted, repeat it, and afterwards enter the password for the administrator user postgres to apply these settings):
    createuser -U postgres -P
  11. Now, also on the shell create your first database and assign it to our new user by replacing the and values with something more appropriate to your needs (enter the password for the postgres user):
    createdb -U postgres -O
  12. Finally, test if you can access the Postgres server with your new user by printing all the database names:
    psql -U -l

How Does It Work?

PostgreSQL is an Object-Relational Database Management System and it is available to all CentOS servers. Postgres may not be as common as MariaDB, but its architecture and a large array of features to make it an attractive solution for many companies concerned with data integrity.

So what did we learn from this experience?

We began this process by installing the necessary server and client rpm packages using yum. Having done this, we then proceeded to make the Postgres system available at boot before initializing the database system using the postgresql-setup initdb command. We completed this process by starting the database service. In the next stage, we were then required to set the password for the Postgres administrator user to harden the system. By default, the postgresql package creates a new Linux system user called postgres (which is also used as an administrative Postgres user account to access our Postgres DBMS), and by using su -postgres -c we were able to execute the psql commands as the postgres user, which is mandatory upon installation (this is called peer authentication).

Having set the admin password, to have more like a MariaDB shell-type of login procedure where every database user (including the administrator postgres user) can log in using the database psql client’s user -U parameter, we changed this peer authentication to md5 database password-based authentication for the localhost in the pg_hba.conf file (see the next process). After restarting the service, we then used Postgres’s createuser and createdb command line tools to create a new Postgres user and connect it to a new database (we needed to provide the postgres user with the -U parameter because only he has the privileges for it). Finally, we showed you how to make a test connection to the database with your new user using the -l flag (which lists all the available databases). Also, you can use the -d parameter to connect to a specific database using the syntax: psql -d -U .

There's more…

Instead of using the createuser or createdb Postgres command-line tools, as we have been showing you in this process, to create your databases and users, you can also do the same using the Postgres shell. In fact, those command-line tools are actually just wrappers around the Postgres shell commands, and there is no effective difference between the two. psql is the primary command-line client tool for entering SQL queries or other commands on a Postgres server, similar to the MariaDB shell shown to you in another process in this chapter division. Here, we will launch psql with a template called template1, the boilerplate (or default template) that is used to start building databases. After login (psql -U postgres template1), and typing in the administrator password you should be presented with the interactive Postgres prompt (template1=#). Now to create a new user in the psql shell, type:
CREATE USER WITH PASSWORD '
';

To create a database, type:
CREATE DATABASE ;

The option to grant all privileges on the recently created database to the new user is:
GRANT ALL ON DATABASE to ;

To exit the interactive shell, use: \q followed by pressing the Return key.

Having completed this process you could say that you not only know how to install PostgreSQL, but this process has served to highlight some simple architectural differences between this database system and MariaDB.

 

Using WebDAV for file sharing in CentOS

The Web-based Distributed Authoring and Versioning (WebDAV) open standard can be used for sharing files over the network. It is a popular protocol to conveniently access remote data as an online hard disk. There are a lot of online storage and e-mail providers who offer online space through WebDAV accounts. Most graphical Linux or Windows systems can access WebDAV servers in their file managers out-of-the-box. For other operating systems, there are also free options available. Another big advantage is that WebDAV is running over normal HTTP or HTTPS ports, so you can be sure that it will work in almost any environment, even behind restricted firewalls.

Here, we will show you how to install and configure WebDAV as an alternative for the FTP protocol for your file sharing needs. We will use HTTPS as our communication protocol for secure connections.

To Start With: What Do You Need?

To complete this process, you will require a working installation of the CentOS 7 operating system with root privileges and a console-based text editor of your choice. You will need a working Apache web server with SSL encryption enabled and reachable in your network; see Providing Mail Services for how to install the HTTP daemon, and especially the process of Setting up HTTPS with SSL. Also, some experience working with the Apache config file format is advantageous.

The Process

  1. Create a location for sharing your data and for a WebDAV lock file:
    mkdir -p /srv/webdav /etc/httpd/var/davlock
  2. Since WebDAV is running as an Apache module over HTTPS, we have to set proper permissions to the standard httpd user:
    chown apache:apache /srv/webdav /etc/httpd/var/davlock chmod 770 /srv/webdav
  3. Now, create and open the following Apache WebDAV configuration file:
    vi /etc/httpd/conf.d/webdav.conf
  4. Put in the following content:
    DavLockDB "/etc/httpd/var/davlock"
    Alias /webdav /srv/webdav

          DAV On
          SSLRequireSSL
          Options None
          AuthType Basic
          AuthName webdav
          AuthUserFile /etc/httpd/conf/dav_passwords
          Require valid-user
  5. Save and close the file. Now, to add a new WebDAV user named john (enter a new password for the user as prompted):
    htpasswd -c /etc/httpd/conf/dav_passwords john
  6.  Finally, restart the Apache2 web server:
    systemctl restart httpd
  7. To test if we can connect to our WebDAV server, you can use a graphical user interface (most Linux file managers support WebDAV browsing) from any client in your network, or we can mount the drive using the command line.
  8. Log in on any client machine as root in the same network as our WebDAV server (on CentOS, you need the davfs2 filesystem driver package to be installed from the EPEL repository, and the usage of file locks must be disabled as the current version is not capable of working with file locks), enter the password for our DAV user account named john, and confirm the self-signed certificate when asked:
    yum install davfs2
    echo "use_locks 0" >> /etc/davfs2/davfs2.conf
    mkdir /mnt/webdav
    mount -t davfs https:///webdav /mnt/webdav
  9. Now, to see if we can write to the new network storage type:
    touch /mnt/webdav/testfile.txt
  10. If you’ve got connection problems, check the firewall settings on your WebDAV server for the services http and https, as well as on your client.

How Does It Work?

Here in this process, we showed you how easy it is to set up a WebDAV server for easy file sharing.

So, what did we learn from this experience?

We started our journey by creating two directories: one, where all the shared files of our WebDAV server will live, and one for creating a lock file database for the WebDAV server process. The latter is needed so that users can block access to documents to avoid collisions with others if files are currently modified by them. As WebDAV runs as a native Apache module (mod_dav) that is already enabled by default in CentOS 7, all we need to do is create a new Apache virtual host configuration file, where we can set up all our WebDAV settings. First, we have to link our WebDAV host to the full path of the lock database that is used to track user locks. Next, we defined an alias for our WebDAV sharing folder, which we then configured using a Location directive. This will be activated if someone is using specific HTTP methods on the /webdav path URL. Within this area, we specified that this URL will be a DAV-enabled share, enabled SSL encryption for it, and specified basic user-based password authentication. The user account’s passwords will be stored in a user account database called /etc/httpd/conf/dav_passwords. To create valid accounts in this database file, we then used the Apache2 htpasswd utility on the command line. Finally, we restarted the service to apply our changes.

For testing, we used the davfs filesystem driver, which you need to install on CentOS 7 using the davfs2 package from the EPEL repository. There are many other options available, such as the cadaver WebDAV command-line client (also from the EPEL repository); alternatively, you can access it directly using integrated WebDAV support in a graphical user interface such as GNOME, KDE, or Xfce.

 

Using a CentOS third-party repository

In this process, we will investigate the desire to take full advantage of the packages that are available to CentOS by installing both the EPEL and Remi repositories. CentOS is an enterprise-based operating system that prides itself on stability, and during the lifetime of your server, it is possible that not every piece of software you need can be found in the default repositories. It is also possible that you may require updated packages of current software, and for these reasons, many server administrators choose to install both the EPEL and Remi repositories. These are not the only repositories available, but because they represent one of the most popular combinations, it is the purpose of this process to show you how both the EPEL and Remi repositories can be added to your system.

To Start With: What Do You Need?

To complete this process, you will require a working installation of the CentOS 7 operating system with root privileges, a console-based text editor of your choice, and a connection to the Internet in order to facilitate the download of additional packages.

The Process

Before we start, it is assumed that you have followed the previous process that showed you how to install and activate YUM priorities.

  1. To begin, log in as root and install the EPEL release repository using YUM:
    yum install epel-release
  2. Next, from your home directory, type the following commands to download the remi release rpm package:
    curl -O http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
    Note
    Please note that, while you are reading this, this URL may have changed; if so, please do some Internet research to find out if there is a new URL available.
  3. The preceding file should now be located in your home folder. To proceed, type the following command:
    rpm -Uvh remi-release-7.rpm
  4. After the installation is done, open the Remi repository file with your favorite text editor:
    vi /etc/yum.repos.d/remi.repo
  5. Change enabled=0 to enabled=1 and add the line priority=10 to the end of the [remi] section.
  6. Now, open the EPEL repository file with your favorite text editor:
    vi /etc/yum.repos.d/epel.repo
  7. Again, change enabled=0 to enabled=1 if not set automatically and add the line priority=10 in the [epel] section.
  8. To finish, update YUM as shown here:
    yum update
  9. If updates are available, choose Y to proceed. Having completed the update process, you will now be able to download and install packages from both the Remi and EPEL repositories as an addition to those that are used by default. 

How it works...

In order to use and enjoy the benefits of a third-party repository, you are required to install and enable it first using the YUM and RPM package manager.

So, what did we learn from this experience?

Having started the process, the task of installing both the Remi and EPEL repositories is a remarkably smooth process. While the installation of the EPEL repository using YUM is very safe to changes, the preceding URL for Remi is maintained at the discretion of the repository owners, so you should always ensure that they are the most current. However, having obtained the necessary repository setup file, it was then a matter of applying an RPM-based command in order to install all necessary repository files on your system. Having done this, we were then required to open the relevant configuration files of each of the installed repositories and enable them (by changing enabled=0 to enabled=1) and setting a priority value (priority=10). While the former value will merely switch the repository on, the latter one will be used by YUM to correctly identify which repositories were the most appropriate when we called the update command. As it was discussed in the previous process regarding YUM priorities, the simple rule of thumb is based on remembering the phrase “the lower the number, the higher the priority.” This, in itself (depending on your reasons), may not be a bad thing to do, but for the purpose of this process, it is shown that the default CentOS repositories should take priority over all others. Of course, you may disagree with this, and yes, there is nothing stopping you from applying the same priority rule to a third-party supplier, but I do caution you before diving in, and this is particularly the case if this is for a mission-critical production server. Remember, if all the priority values are the same, then YUM will attempt to download the latest version by default.

The reason for setting both Remi and EPEL to a higher value than the existing CentOSbased repositories is based on the need to consider security updates. Unless you have determined otherwise, it is always advised that the base files should come from CentOS first. This includes, but it is not limited to, Kernel updates, SELinux, and related packages. Third-party repositories should be used for additional packages that cannot be obtained from the original sources, or for access to particular updates that may not be available to the base release of CentOS. This may include packages such as Apache, MariaDB, or PHP. As a final footnote, you will have noticed that both Remi and EPEL repositories shared the same priority value. This is by design as these repositories are often viewed as partners. However, if you decide to begin mixing repositories, or use this process as a gateway to installing other repositories not mentioned here, then you should always do your research and evaluate every third-party on a case-by-case basis. The Remi and EPEL repositories are very popular, so if you do intend to add more third-party resources, read around the subject, choose your repositories carefully, and stay loyal.

There's more…

There are many other interesting repositories available for CentOS 7, such as ELRepo, which focuses on hardware-related packages such as filesystem drivers, graphics drivers, network drivers, sound drivers, and webcam or video drivers. Go to http://elrepo.org to learn how to install and access it.