Memory is another important component of system performance. All files and data that are currently being used are kept in the system main memory for faster access. The CPU performance also depends on the availability of enough memory. Swap, on the other hand, is an extension to main memory. Swap is part of persistent storage, such as hard drives or solid state drives. It is utilized only when the system is low on main memory.
In this article, we will learn how to monitor system memory and swap utilization.
Getting ready
You may need sudo privileges for some commands.
How to do it…
In the last recipe, we used commands top and vmstat to monitor CPU utilization. These commands also provided details of memory usage. Let's start with the top command again:
Run the top command and check for the Mem and Swap rows:
The memory line displays the size of total available memory, size of used memory, free memory, and the memory used for buffers and the file system cache. Similarly, swap row should display the allocated size of the swap if you have enabled the swapping. Along with these two lines, top shows per process memory utilization as well. The columns VIRT, RES, SHR, and %MEM all show different memory allocation for each process:
Similar to the top command, you can query memory statistics for a specific PID or program by using the pidstat command. By default, pidstat displays only CPU statistics for a given process. Use flag -r to query memory utilization and page faults:
$ pidstat -C mysql -r
Next, we will go through the vmstat command. This is an abbreviation of virtual memory statistics. Enter the command vmstat in your console and you should see output similar to the following screenshot:
Using vmstat without any option returns a single line report of memory, swap, io, and CPU utilization. Under the memory column, it shows the amount of swap, free memory, and the memory used for cache and buffers. It also display a separate swap column with Swap In (si) and Swap Out (so) details.
To get detailed statistics of memory and event counters, use flag -s. This should display a table, as follows:
$ vmstat -s
Another handy command is free, which displays the amount of used and available memory in the system. Use it as follows, with the -h flag to get human-friendly units:
$ free -h
Finally, command sar can give you periodic reports of memory utilization. Simply enable sar to collect all reports and then extract memory reports from it or set a specific command to log only memory and swap details.
Finally, use sar to monitor current memory and swap utilizations. The following command will query the current memory (-r) and swap (-S) utilization:
$ sar -rS 1 5
For more details on using sar, check Monitoring the CPU recipe or read the manual pages using the man sar command. The command sar is available in the package sysstat; you will need to install it separately if not already installed.
All these tools show process-level memory statistics. If you are interested in memory allocation inside a particular process, then the command pmap can help you. It reports the memory mapping of a process, including details of any shared libraries in use and any program extensions with their respective memory consumptions. Use pmap along with the PID you want to monitor as follows:
$ sudo pmap -x 1322
How it works…
System memory is the primary storage for processes in execution. It is the fastest available storage medium, but is volatile and limited in storage space. The limited storage is generally extended with the help of slower, disk-based Swap files. Processes that are not being actively executed are swapped to disk so that active processes get more space in the faster main memory. Similar to other operating systems, Ubuntu provides various tools to monitor system-wide memory utilization as well as memory uses by process. Commonly used tools include top, vmstat, and free.
We have used the top command to monitor CPU uses and know that top provides a summarized view of system resource utilization. Along with a CPU summary, top also provides the memory statistics. This includes overall memory utilization plus per process usage. The summary section in the top output displays the total available and used memory. It also contains a separate row for swap. By default, all Ubuntu systems enable the swap partition with nearly the same size as main memory. Some cloud service providers disable the cache for performance reasons.
The details section of top shows per process memory usage separated into multiple columns:
Column VIRT shows the virtual memory assigned to a task or process; this includes memory assigned for program code, data, and shared libraries, plus memory that is assigned but not used.
Column RES shows the non-swapped physical memory used by processes. Whereas column SHR shows the amount of shared memory, this is the memory that can be shared with other processes through shared libraries.
The column %MEM shows the percentage of main memory assigned to a specific process. This is a percentage of RES memory available to task out of total available memory.
By default, all memory values are shown in the lowest units, KB. This can be changed using the key combination, Shift + E for summary rows and E for process columns.
Similar to top, the command ps lists running processes but without refreshing the list. Without any options, ps shows the list of processes owned by the current user. Use it as follows to get a list of all running processes:
$ ps aux
The command vmstat gives you overall detail regarding memory and swap utilization. The memory column shows the amount of available memory. Next to the memory column, the swap column indicates the amount of memory read from disk (si) or written to disk (so) per second. Any activity in the si and so columns indicates active swap utilization. In that case, you should either increase the physical memory of the system or reduce the number of processes running. Large numbers under the swap column may also indicate higher CPU utilization, where the CPU waits for IO operations (wa) to complete. As seen before, you can specify the delay and interval options to repeatedly query vmstat reports.
One more command, named free, shows the current state of system memory. This shows overall memory utilization in the first row and swap utilization in the second row. You may get confused by looking at the lower values in the free column and assume higher memory uses. Part of free memory is being used by Linux to improve file system performance by caching frequently used files. The memory used for file caching is reflected in the buff/cache column and is available to other programs when required. Check the last column, named available, for the actual free memory.
The second row of free output displays the swap utilization. You may see swap being used under the used column. This is the amount of swap allocated but not effectively used. To check if your system is effectively swapping, use the command vmstat 1 and monitor si/so columns for any swap activity.
System swapping behavior also depends on the value of the kernel parameter named vm.swappiness. Its value can range between 0 to 100, where 0 configures the kernel to avoid swapping as much as possible and 100 sets it to swap aggressively. You can read the current swappiness value using the following command:
$ sudo sysctl vm.swappiness
vm.swappiness = 60
To modify the swappiness value for the current session, use the sysctl command with a new value, as follows. It is a good idea to use lower values and avoid swapping as much as possible:
$ sudo sysctl vm.swappiness=10
vm.swappiness = 10
To permanently set swappiness, you need to edit the /etc/sysctl.conf file and add or uncomment vm.swappiness=10 to it. Once the file is updated, use the following command to read and set a new value from the configuration file:
$ sudo sysctl -p
Check the swapon and swapoff commands if you need to enable swapping or disable it.
There's more…
Most of these statistics are read from the /proc partition. The two main files listing details of memory and swap are /proc/meminfo and /proc/swaps.
The command lshw (list hardware) can give you the details of actual hardware. This includes the physical memory configuration, the firmware version, CPU details, such as clock speed, the cache, and various other hardware information. Use lshw as follows:
$ sudo lshw
See also
Check the swapon and swapoff commands to enable or disable swap files:
$ man swapon
$ man swapoff
