At Orange Sputnik, we prefer to keep the development environment the same as live servers to have hassles free while the deployment of projects for our clients and do 🙅 not fight windmills. So when C# applications’ natural environment is Windows, then it would be better to stick with Windows for C# Developer as a development environment. Isn’t it? And in case having NodeJS or PHP based projects, we will use the Linux environment for NodeJS Developers the same as for PHP Developers.
Mobile Applications Developers stay in different environments.
When iOS Developer should always stick iOS and macOS as a development setup, Android Mobile Applications Developer is preferred to stay with Linux based OS like Ubuntu, the Hybrid Mobile Application Developer must have access to iOS/macOS and Android OS based hardware at the same time.
The most popular Linux distro is Ubuntu and so let’s talk about Ubuntu Desktop distro. In this article, I would like to get together all stuff related to Ubuntu Desktop optimization what applies for almost any Ubuntu releases. What do you think about speedup in 4x – 5x time for boot or guest OS related? 🤪 Frankly sound crazy, but it is possible! 🤯 And especially it’s a very nice sentiment when you are working on a virtual machine like VPS or running the guest operating system under VirtualBox, VMware, Parallels, or other similar software.
Let’s stick to some intro input first.
Nowadays, in most cases, you have a laptop with SSD.
Nowadays, in most cases, you have UEFI instead of BOIS.
Nowadays, in most cases, your SSD is NVME.
In most cases, you don’t really care about “default setup” and “default installed applications”, but should be. 🤷
I better stick to Midnight Commander (MC) and its editor (mcedit) rather nano or vim. In case when it is not possible to use in environment MC then for sure I’ll use nano or vim. Whenever it is possible to stick with MC – I’ll stick with MC. Ok? 👌
Install MC with the command:
sudo apt install mc
This comes after operating system installation. First, let’s go through our checklist related to installation and pre-installation steps.
BIOS and UEFI first
NVME and SSD are recognized correctly usually by the BIOS and UEFI. In case if it is not properly recognized then you need to enable AHCI instead of IDE for your SATA Configuration.
SSD Alignment – Outdated
It was a true story about that partitions on SSD should be properly aligned first years ago. Not today! In all current versions of operating systems, including Ubuntu, all tools automatically align filesystems and partitions to the optimal 4096 bytes page size. It means you don’t need to care about SSD alignment.
SSD Over-provisioning
Today SSD over-provisioning is outdated. For those who don’t know SSD over-provisioning (OP) is a technology used in solid-state drives (SSD) to reserve free space for specific controller activity. Using OP allows you to extend SSD lifespan, increase the stability of its work, but reduces the free disk space available to the user. Normal over-provisioning for home computers is 7-10%, for corporate use (read: office use) and use in servers – 28% and higher is recommended.
However, in the new generations of SSD’s manufacturers deliver some built-in unallocated space for over-provisioning, and this space is not accessible to the user.
ext4 partition… by default
The best file system for partitions on SSD is the usual default EXT4. So you can just go with default selection while Ubuntu installation.
As you might know, EXT4 is a journaling file system, but it doesn’t do journaling much, so don’t worry about that. Moreover, journaling is an important feature for system crash recovery.
noatime after installation
Putting “noatime” in /etc/fstab, you disable access timestamp writing. This timestamp operating system puts on a file when it’s read by the operating system. As less writes is good for SSD, then “noatime” is also good for SSD. The only cases when you really need this feature I would recommend to not use noatime, otherwise – enable noatime. Example? You are developing some specific project when you need to have the latest file access timestamp.
Enable noatime in /etc/fstab this way:
UUID=xxxxxxx / ext4 noatime,errors=remount-ro 0 1
TRIM support
First, check for TRIM support with your SDD. Run next command in terminal:
lsblk -D
Then you should see something like this output:
NAME DISC-ALN DISC-GRAN DISC-MAX DISC-ZERO loop0 0 4K 4G 0 ... loop11 0 4K 4G 0 sda 0 0B 0B 0 └─sda1 0 0B 0B 0 ├─ubuntu--vg-root 0 0B 0B 0 └─ubuntu--vg-swap_1 0 0B 0B 0 sr0 0 0B 0B 0
TRIM/discard is available if the DISC-MAX column is not 0B.
Do you need to enable TRIM? Another command to check if your drive supports TRIM is next:
sudo hdparm -I /dev/sda
For search lines about TRIM only add “| grep -i trim” to the end of the command. In the output of hdparm command you see:
/dev/sda:
ATA device, with non-removable media
Model Number: VBOX HARDDISK
Serial Number: VB5060b879-b252d4d1
Firmware Revision: 1.0
Standards:
Used: ATA/ATAPI-6 published, ANSI INCITS 361-2002
Supported: 6 5 4
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors: 16514064
LBA user addressable sectors: 67108864
LBA48 user addressable sectors: 67108864
Logical/Physical Sector size: 512 bytes
device size with M = 10241024: 32768 MBytes device size with M = 10001000: 34359 MBytes (34 GB)
cache/buffer size = 256 KBytes (type=DualPortCache)
Capabilities:
LBA, IORDY(cannot be disabled)
Queue depth: 32
Standby timer values: spec'd by Vendor, no device specific minimum
R/W multiple sector transfer: Max = 128 Current = 128
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
Cycle time: no flow control=120ns IORDY flow control=120ns
Commands/features:
Enabled Supported:
* Power Management feature set
* Write cache
* Look-ahead
* 48-bit Address feature set
* Mandatory FLUSH_CACHE
* FLUSH_CACHE_EXT
* Gen2 signaling speed (3.0Gb/s)
* Native Command Queueing (NCQ)
Checksum: correct
This output sample shows that drive does not have TRIM support… but it’s a guest OS under VirtualBox, basically. 😉 A couple of years ago you had to create a specific script to enable TRIM. Starting from Ubuntu 18.04 developers of Ubuntu already took care of automatic checking.
TRIM (fstrim/discard) is enabled to run weekly by default for all mounted filesystems on devices that support the discard operation.
You can check TRIM support enabled with this command:
systemctl cat fstrim.service
This is to view the configuration of fstrim.service. You should see something like this:
/lib/systemd/system/fstrim.service [Unit] Description=Discard unused blocks ConditionVirtualization=!container [Service] Type=oneshot ExecStart=/sbin/fstrim -av
Note here for the line: ExecStart=/sbin/fstrim -av
Explanation of parameters:
-a – stands for trim all mounted filesystems on devices that support the discard operation.
-v – goes for verbose execution. Output the number of bytes passed from the filesystem down the block stack to the device for potential discard.
Another command is for viewing the status of fstrim.timer:
systemctl status fstrim.timer
You can see then it’s configured to run on a weekly basis:
● fstrim.timer - Discard unused blocks once a week
Loaded: loaded (/lib/systemd/system/fstrim.timer; enabled; vendor preset: enabled)
Active: active (waiting) since Tue 2020-05-12 12:18:25 EEST; 13min ago
Trigger: Mon 2020-05-18 00:00:00 EEST; 5 days left
Docs: man:fstrim
After this what you need is to disable disk queues under virtual guest OS if you’re running VPS, VirtualBox instance or similar.
sudo mcedit /etc/default/grub
Find a line:
GRUB_CMDLINE_LINUX=""
And replace it with:
GRUB_CMDLINE_LINUX="elevator=noop"
Then remember to update the grub:
sudo update-grub
Cool! 😎
Why do so? Modern operating systems are modular, and they often have different modules to work with memory, network I/O, CPU scheduling, disk I/O. Now we are talking about the Completely Fair Queuing (CFQ) scheduler.
Completely Fair Queuing (CFQ) is an I/O scheduler for the Linux kernel which was written in 2003 by Jens Axboe.
Source: Wikipedia.
CFQ places synchronous requests submitted by processes into a number of per-process queues and then allocates time slices for each of the queues to access the disk. The length of the time slice and the number of requests a queue is allowed to submit depends on the I/O priority of the given process. Asynchronous requests for all processes are batched together in fewer queues, one per priority. While CFQ does not do explicit anticipatory I/O scheduling, it achieves the same effect of having good aggregate throughput for the system as a whole, by allowing a process queue to idle at the end of synchronous I/O thereby “anticipating” further close I/O from that process. It can be considered a natural extension of granting I/O time slices to a process.
In the physical world, with operating systems on bare metal, like dedicated servers, standalone computers, laptops, we will choose an algorithm that is best suited according to our workload.
In case you’re running guest OS (means Virtual Machine) there is a hypervisor between our OS and the disks. Such hypervisors have their own disk queues. It works as listed:
- Disk I/O occurs on your Virtual Machine
- Guest OS uses some CPU time to sort disk I/O into an order according to the active disk algorithm.
- Guest OS makes requests to the underlying virtual hardware with a new order.
- The hypervisor of the main OS takes requests and re-sorts them based on its own algorithms.
- The hypervisor makes the requests to the real hardware according to its own order.
So in simple words, we’re removing useless sorting which is never being used by specifying “elevator=noop” for GRUB under guest OS on Virtual Machine. When we enable the NOOP scheduler, the guest OS will do as less as possible to the disk I/O before passing it to the main OS.
Now let’s optimize our Ubuntu Desktop.
What I always do is installing of apt-fast, htop and preload.
apt-fast is a shell-script wrapper for apt-get, apt or aptitude that can drastically improve apt download while installing software. It uses aria2 for downloads which do this by downloading packages in parallel, with multiple connections per package.
htop is an interactive system-monitor process-viewer and process-manager. This is an alternative to the Unix program top. It shows a frequently updated list of the processes running on a computer. I like it since it uses color and gives visual information about processor, swap, and memory status and can display the processes as a tree-view. Next to it can sort processes by any available column and filter the list. Comfy! 😏
preload – is a daemon that collects statistics about usage of programs using Markov chains; files of more frequently used programs are, during a computer’s spare time, loaded into memory. This results in faster startup times as less data needs to be fetched from disk. And the most interesting it does not need any configuration. It works perfect out-of-the-box.
Simply run the commands:
sudo apt install htop sudo apt install preload
Next is to install apt-fast. Since it is not yet in Ubuntu distro as default available package you need to add its repository first before the installation.
sudo add-apt-repository ppa:apt-fast/stable sudo apt update sudo apt install apt-fast
If you get an error “add-apt-repository command not found” run this first:
sudo apt install software-properties-common
Back to /etc/fstab
Let’s apply some low writings to the disk, as we have SSD, parameters for the root file system.
UUID=XXXXXX / ext4 noatime,nodelalloc,barrier=0,i_version,commit=30,inode_readahead_blks=64,errors=remount-ro 0 1
sysctl tuning
As for Desktop environment it would be great having these lines in /etc/sysctl.conf:
mcedit /etc/sysctl.conf
vm.swappiness=1 vm.vfs_cache_pressure=50 vm.dirty_writeback_centisecs=1500
vm.swappiness stands for how eager OS should consume swap.
This control is used to define how aggressive the kernel will swap memory pages. Higher values will increase aggressiveness, lower values decrease the amount of swap. A value of 0 instructs the kernel not to initiate swap until the amount of free and file-backed pages is less than the high water mark in a zone.
Source: Kernel.org.
vm.vfs_cache_pressure is a percentage value that controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.
At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a “fair” rate with respect to pagecache and swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.
Source: Kernel.org.
Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are.
vm.dirty_writeback_centisecs is about how fast write cache flush to the disk.
The kernel flusher threads will periodically wake up and write ‘old’ data out to disk. This tunable expresses the interval between those wakeups, in 100’ths of a second.
Source: Kernel.org.
Setting this to zero disables periodic writeback altogether.
To take effect you need to reboot, the other way is to force sysctl to reload config:
sudo sysctl -p
Disable unused Systemd services
Before disabling these services think twice before. Maybe you need them. Sure, you can unmask and re-enable it, but better disable what you really don’t need. Just keep it in your mind. 😉
systemctl disable mpd.socket systemctl disable snapd systemctl disable avahi-daemon systemctl disable apparmor systemctl disable ModemManager systemctl mask ModemManager systemctl disable NetworkManager-wait-online systemctl mask NetworkManager-wait-online systemctl disable nmbd systemctl disable smbd systemctl disable lm-sensors systemctl mask lm-sensors systemctl mask apparmor systemctl mask bluetooth systemctl mask speech-dispatcher systemctl mask pppd-dns systemctl mask apport
plymouth-quit-wait.service
Some computers and especially Virtual Machines with Desktop enabled can start very slow. This can be affected by plymouth-quit-wait.service.
See the sample of the output of a system boot analysis:
sudo systemd-analyze blame
You get a listing of services started and time spent. Something like this:
15.405s plymouth-quit-wait.service
9.843s dev-sdb4.device
9.049s NetworkManager-wait-online.service
6.305s bolt.service
5.889s snapd.service
4.243s udisks2.service
4.092s grub-common.service
3.806s networking.service
3.780s ModemManager.service
3.325s dev-loop10.device
3.295s apparmor.service
3.190s dev-loop13.device
2.162s accounts-daemon.service
As for me, I don’t like such delays.
And what is this plymouth is?
Plymouth is the application that provides the graphical “splash” screen when booting and shutting down an Ubuntu system. I don’t really see a big requirement for it in my setup.
In my case, Ubuntu 18.04 was starting about 30 seconds as a guest OS under VirtualBox. I decided to disable it:
sudo systemctl disable plymouth-quit-wait.service
Next to it, I disabled plymouth during the boot process. This can be done by editing file /etc/default/grub.
The line was looking this way:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
Now it looks this way:
GRUB_CMDLINE_LINUX_DEFAULT="noplymouth video=SVIDEO-1:d"
Basically, the side effect of this is you won’t have a splash screen while loading… and shutting down…
Nah! 🤔 As long as you get 6-7 seconds for a startup instead of 30, I guess it is outstanding! 🤯
If you go the same way, remember to update the GRUB:
sudo update-grub
tlp – another nice tool to reduce overheating and power consumption
Overheating is a common problem in computers these days. But it’s not for sure… In scope of the latest Intel CPU, for example, 9th or 10th generation this can be not really true. They produce so low levels of heat that service engineers sometimes says it could work without an active cooling system.
But anyway, an overheated computer runs quite slow. You can use tlp to reduce overheating and thus get a better system performance in Linux.
tlp is a feature-rich command-line utility for Linux, saving laptop battery power. The default setup of tlp is already optimized for battery life and implement Powertop’s recommendations out of the box. So, just grab, and use.
Install tlp with these commands:
sudo add-apt-repository ppa:linrunner/tlp sudo apt update sudo apt install tlp tlp-rdw sudo tlp start
Ubuntu Desktop lightweight environment
By default, Ubuntu comes with GNOME. It is a great desktop environment, the same as huge. The problem arises when you need to work under a Virtual Machine. Running under guest OS GNOME can be dramatically slow.
In that case, I would recommend installing an alternative lightweight desktop environment and switch to it on the login screen.
You can install Xfce or LXDE, as you wish.
sudo apt install xfce4 sudo apt install lxde
zram and zswap
If you running on low memory like 8G of total memory with some old laptop and use huge memory-consuming apps, then you definitely “fall into swap”.
This becomes a disaster if you have SSD since the swap is frankly a drive-killer in such a case.
Especially for such cases, I would recommend installing zram and/or zswap.
zram, formerly called compcache, is a Linux kernel module for creating a compressed block device in RAM, i.e. a RAM disk, but with on-the-fly “disk” compression. The block device created with zram can then be used for swap or as a general-purpose RAM disk. The two most common uses for zram are for the storage of temporary files (/tmp) and as a swap “disk”. Initially, zram had only the latter function, hence the original name “compcache” (“compressed cache”).
Source: Wikipedia.
zswap is a Linux kernel feature that provides a compressed write-back cache for swapped pages, as a form of virtual memory compression. Instead of moving memory pages to a swap device when they are to be swapped out, zswap performs their compression and then stores them into a memory pool dynamically allocated in the system RAM. Later, writeback to the actual swap device is deferred or even completely avoided, resulting in a significantly reduced I/O for Linux systems that require swapping; the tradeoff is the need for additional CPU cycles to perform the compression.
Source: Wikipedia.
You need to decide what is best in your case. Let me just put the cons and pros of both.
zram pros:
- Eliminates the need for the physical swap device. This became popular when netbooks first showed up. It allows users to avoid swap shortening the lifespan of SSDs in these memory-constrained systems.
- A zram block device can be used for other applications other than swap, anything you might use a block device for conceivably.
- Frankly, zram not need to be configured.
zram cons:
- Once a page is stored in zram it will remain there until paged in or invalidated. The first pages to be paged out will be the oldest pages (LRU list). These are “cold pages” that are rarely accessed. As the system continues to swap it will move on to pages that are “warm pages” (means more frequently accessed). These may not be able to be stored due to the swap slots consumed by the “cold pages”.
What zram can’t is to evict pages out to a physical disk.
As I understand the ideal, you would want is to age data out of the in-kernel compressed swap space out to disk so you can use kernel memory for caching “warm swap pages” or free it for more productive use.
zswap pros:
- Integration with swap code (using Frontswap API) allows zswap to store only pages that compress well and handle memory allocation failures, in those cases it sends pages to the backing swap device.
- It pushes out the oldest pages in the cache to backing swap device to leave more memory for newer pages, this solves the LRU inversion problem that a lack of pages eviction would present.
- No need to install package.
zswap cons:
- Needs a physical swap device or permanent swap file.
- Can work with “default” settings, but frankly, requires configuration
You can install zram with the following command:
sudo apt install zram-config
Then you must reboot. If you want to check if zram is working here is the command:
swapon -s
If you see zram in devices, then it works:
Filename Type Size Used Priority /dev/dm-1 partition 999420 0 -2 /dev/zram0 partition 1536792 0 5 /dev/zram1 partition 1536792 0 5 /dev/zram2 partition 1536792 0 5 /dev/zram3 partition 1536792 0 5
If you already changed the value of vm.swappiness to reduce the threshold for the swap usage, then it would be better to change it to something like 40. So then when 60% of the RAM is already used, zram start working.
To enable zswap you need to edit /etc/default/grub:
sudo mcedit /etc/default/grub
Find the line with GRUB_CMDLINE_LINUX_DEFAULT and add text into it:
GRUB_CMDLINE_LINUX_DEFAULT="zswap.enabled=1 noplymouth video=SVIDEO-1:d"
If you change /etc/default/grub always remember to update the GRUB:
sudo update-grub
Finally, you need to reboot to take zswap into effect.
After reboot, you can check if zswap is working:
dmesg | grep zswap
You will see something like this output:
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-99-generic root=/dev/mapper/ubuntu--vg-root ro elevator=noop zswap.enabled=1 noplymouth video=SVIDEO-1:d [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-99-generic root=/dev/mapper/ubuntu--vg-root ro elevator=noop zswap.enabled=1 noplymouth video=SVIDEO-1:d [ 1.280904] zswap: loaded using pool lzo/zbud
In case if you need to limit pool size zswap simply put extra text in /etc/default/grub:
GRUB_CMDLINE_LINUX_DEFAULT="zswap.enabled=1 zswap.max_pool_percent=XXX noplymouth video=SVIDEO-1:d"
Where XXX is a number of percents of memory size for szwap.
Take care of tmpfs
tmpfs is a temporary file storage paradigm implemented in Unix-like operating systems. As for the user, it appears as a mounted file system, but all files are stored in volatile memory instead of persistent storage (HDD, SSD, SD-card, etc.).
It means that all stored in tmpfs will be lost on reboot.
The memory used by tmpfs grows and shrinks to keep the files it contains.
After all applications installed edit your /etc/fstab:
sudo mcedit /etc/fstab
And add these lines to the end of this file:
tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0 tmpfs /var/tmp tmpfs defaults,noatime,mode=1777 0 0
Now, all files under these locations will become temporary and will be stored in RAM.
While you’re not using a production server, and you don’t look at system logs you can also put /var/log under tmpfs.
tmpfs /var/log tmpfs defaults,noatime,mode=0755 0 0
This is intended to minimize writes from your OS and applications installed, which is good when you have SSD.
Office applications
This is the last topic in this article. And this is not really about optimization. We had very often situations of the exchange of documents between developers who work in different environments and with different office applications. Documents were saved and re-opened by another person, and it was looking broken. 😳 This how the compatibility of office applications working… Not really! 🤪 For example, when you open an Excel file in LibreOffice you can notice that formulas are gone… values are in place… but where formulas have gone? 🤨 When you save it, you won’t have formulas in Excel back. 😒
Distributed default LibreOffice is not relevant anymore in case of working with different from Open Document Format. Same with OpenOffice. Based on our experience MS Office file formats are more popular globally. Thus in our Ubuntu Desktop setup, for those who work with different file formats, and especially with MS Office files we install ONLYOFFICE.
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys CB2DE8E5 sudo echo "deb https://download.onlyoffice.com/repo/debian squeeze main" > /etc/apt/sources.list.d/onlyoffice.list sudo apt update sudo apt install onlyoffice-desktopeditors
So guys, this is a default Ubuntu Desktop environment we use in Orange Sputnik.
👍 Like? Share! 🥰 Cheers! 🥂 With love… ❤
