Ubuntu Server Performance Optimization

Complete Guide to BBR Congestion Control, File System Optimization & Resource Limits

Introduction

Optimizing an Ubuntu server involves several critical steps that can dramatically improve performance. This comprehensive guide covers three essential optimization areas: implementing BBR (Bottleneck Bandwidth and Round-trip propagation time) congestion control, optimizing file system performance through the noatime directive, and configuring system resource limits to handle high-traffic scenarios efficiently.

Key Benefits:
  • Throughput improvements up to 2,700x with BBR congestion control
  • Reduced I/O operations through file access time optimization
  • Enhanced capacity for concurrent connections and open files
  • Better performance for WordPress sites using Nginx, MariaDB, and PHP

1. BBR Congestion Control Algorithm

Understanding BBR

Congestion control algorithms determine how quickly data should be transmitted across a network. Traditional algorithms like Reno and Cubic rely on packet loss as their primary signal for network congestion. BBR (Bottleneck Bandwidth and Round-trip propagation time), developed by Google, takes a revolutionary approach by using latency measurements instead of packet loss to optimize data transmission rates.

Traditional Algorithms (Reno/Cubic)

Primary Signal: Packet Loss

Approach: Reactive - waits for congestion to occur

Performance: Baseline throughput

Issue: Can't distinguish between congestion and random loss

BBR Algorithm

Primary Signal: Latency/RTT

Approach: Proactive - prevents congestion

Performance: Up to 2,700x higher throughput

Advantage: Optimizes for available bandwidth

Performance Comparison

Reno:
1x
Cubic:
1.5x
BBR:
Up to 2,700x

Implementation Steps

Check Available Algorithms
↓
Check Current Algorithm
↓
Enable BBR
↓
Verify Configuration

1Check Available Congestion Control Algorithms

sudo sysctl net.ipv4.tcp_available_congestion_control

Expected Output: net.ipv4.tcp_available_congestion_control = reno cubic bbr

2Check Currently Enabled Algorithm

sudo sysctl net.ipv4.tcp_congestion_control

Expected Output: net.ipv4.tcp_congestion_control = cubic or reno

3Enable BBR (Four Commands Required)

Note: Copy and paste these commands from the Featured Questions section of the course Q&A. The process is straightforward and requires no complicated configuration.
# Command 1: Load BBR kernel module sudo modprobe tcp_bbr # Command 2: Make BBR available after reboot echo "tcp_bbr" | sudo tee -a /etc/modules-load.d/modules.conf # Command 3: Set BBR as the congestion control algorithm echo "net.core.default_qdisc=fq" | sudo tee -a /etc/sysctl.conf echo "net.ipv4.tcp_congestion_control=bbr" | sudo tee -a /etc/sysctl.conf # Command 4: Apply the settings sudo sysctl -p

4Verify BBR is Active

sudo sysctl net.ipv4.tcp_congestion_control

Expected Output: net.ipv4.tcp_congestion_control = bbr

2. File System Optimization - Disabling Access Time Recording

Understanding the noatime Directive

By default, most Linux distributions track the last access time for every file on the system. This means that every time a file is read, the file system performs an additional I/O operation to update the access time metadata. For frequently accessed files, especially on high-traffic web servers, this creates significant unnecessary overhead.

Benefits of noatime:
  • Eliminates unnecessary I/O operations for access time updates
  • Significant performance improvement for frequently accessed files
  • Reduces disk wear on SSD-based systems
  • Access time information is rarely used in practice

Implementation Process

⚠️ Critical Warning: Incorrectly modifying the /etc/fstab file can prevent your server from booting. If you're using a hosting provider other than Vultr, take a screenshot of your fstab file and consult the course Q&A or your hosting provider's documentation before proceeding.

1Identify Your Storage Drive

df -h

Look for the entry mounted on / (root). The device will be listed in the first column (e.g., /dev/vda2).

Filesystem Size Used Avail Use% Mounted on
/dev/vda2 78G 12G 62G 16% /

2Verify Current Mount Parameters

sudo cat /proc/mounts | grep vda2

Example Output: /dev/vda2 / ext4 rw,relatime,errors=remount-ro 0 0

3Backup the fstab File

sudo cp /etc/fstab /etc/fstab.backup

4Edit the fstab File

sudo nano /etc/fstab
# Original line: # /dev/vda2 / ext4 defaults 0 1 # Modified line (add noatime after defaults): # /dev/vda2 / ext4 defaults,noatime 0 1 # Important: No spaces between defaults, comma, and noatime # There IS a space between noatime and the following 0
Editing Instructions:
  • Position cursor after "defaults"
  • Type a comma (no space before comma)
  • Type "noatime" (no space after comma)
  • Ensure there's a space before the "0"
  • Save: Ctrl+O, then Enter
  • Exit: Ctrl+X

5Reboot and Verify

sudo reboot

After reboot, verify the change:

sudo cat /proc/mounts | grep vda2

Expected Output: Should now include noatime in the options list.

3. Increasing System Resource Limits

Understanding Open File Limits

On Linux systems, sockets are treated as files. The default limit for open files per process is typically very low (1024), which severely restricts the number of concurrent connections a server can handle. For web servers handling multiple simultaneous requests, this limitation can cause connection failures and performance degradation.

Hard vs Soft Limits Architecture

Hard Limit (Ceiling: 120,000) Soft Limit (Active: 120,000)
Limit Type Description Who Can Modify Default Value
Hard Limit Maximum ceiling for soft limit Root user only 1,048,576
Soft Limit Enforced resource limit Users (up to hard limit) 1,024 (too low!)

Implementation Steps

1Check Current Limits

# Check hard limit
ulimit -Hn

Expected Output: 1048576

# Check soft limit
ulimit -Sn

Expected Output: 1024 (this is too low)

2Create Custom Limits Configuration

cd /etc/security/limits.d
sudo nano custom_directives.conf
# Add these directives to set both hard and soft limits to 120,000 * soft nofile 120000 * hard nofile 120000 # Explanation: # * = applies to all users # soft = soft limit type # hard = hard limit type # nofile = number of open files # 120000 = the limit value
Recommended Values:
  • 120,000: Suitable for most WordPress hosting scenarios with Nginx, MariaDB, and PHP
  • 200,000-250,000: High-traffic websites or multiple sites
  • Up to 1,000,000: Enterprise-level applications with very high concurrent connections

3Reboot and Verify

sudo reboot

After reboot, verify the changes:

ulimit -Hn # Should return 120000
ulimit -Sn # Should return 120000

4. Enabling PAM-based User Limits

Understanding PAM (Pluggable Authentication Modules)

PAM is a flexible framework for user authentication and session management in Linux. By enabling PAM-based limits, we ensure that non-root users can utilize the higher file descriptor limits we've configured. This is crucial for web server processes that typically run under dedicated user accounts (like www-data for Nginx).

1Navigate to PAM Configuration Directory

cd /etc/pam.d

2Modify common-session File

sudo nano common-session

Add this line at the end of the file:

session required pam_limits.so

3Modify common-session-noninteractive File

sudo nano common-session-noninteractive

Add the same line at the end:

session required pam_limits.so
What this directive does:

The pam_limits.so module enforces resource limits defined in /etc/security/limits.conf and files in /etc/security/limits.d/. This ensures that the limits apply to all user sessions, both interactive and non-interactive (like web server processes).

4Final Reboot

sudo reboot
✓ Congratulations! This is the last reboot required for this optimization phase. Your server will now run with minimal interruptions going forward.

5. Component-Specific Open File Limits

Web Stack Components

Nginx

Web Server

Requires high file limits for concurrent connections

MariaDB

Database Server

Needs limits for database connections and files

PHP-FPM

Application Layer

Multiple workers require adequate file descriptors

Important Note: The configuration of open file limits for Nginx, MariaDB, and PHP-FPM will be covered after these components are installed in subsequent sections of the course. Each component has specific configuration files where these limits must be set.

Common Error to Watch For

Error Message: "Too many open files"

Cause: The open file limit for a specific service (Nginx, MariaDB, or PHP) is set too low.

Solution: Increase the limit in the service's systemd configuration or main configuration file.

Where to check: Service log files located in /var/log/

Summary & Best Practices

Optimizations Completed

  • ✓ BBR Congestion Control: Up to 2,700x throughput improvement over legacy algorithms
  • ✓ Noatime Directive: Reduced I/O overhead for frequently accessed files
  • ✓ System Resource Limits: Increased from 1,024 to 120,000 open files
  • ✓ PAM Configuration: Enabled user-level resource limit enforcement

Best Practices Checklist

Practice Recommendation
Backup Configuration Files Always create backups before editing system files (especially /etc/fstab)
Documentation Keep notes of all changes made for future reference and troubleshooting
Testing Verify each change after reboot to ensure it persists
Monitoring Watch log files for "too many open files" errors after deployment
Host-Specific Configuration Consult provider documentation if using non-Vultr hosting

Performance Impact Summary

Expected Performance Improvements

Network Throughput:
Up to 2,700x with BBR
I/O Operations:
30-50% reduction
Concurrent Connections:
117x increase (1,024 → 120,000)

Next Steps

In the next section of the course, you will:

  • Point a domain name to your server
  • Install and configure Nginx web server
  • Install and optimize MariaDB database
  • Configure PHP-FPM for optimal performance
  • Set component-specific open file limits