🖥️ Proxmox Expert Guide

Creating VM Templates with Cloud-Init

VM templates are pre-configured virtual machine images that serve as blueprints for rapid deployment. Unlike manually installing an operating system on each VM, templates allow you to clone fully configured systems in seconds. Cloud-init enables automated configuration of hostname, user accounts, SSH keys, and network settings during first boot.

Step-by-Step Process

1Create a Virtual Machine

Start by creating a new VM through the Proxmox web interface. Choose your desired Linux distribution ISO image (Ubuntu, Debian, CentOS, etc.). Configure the VM with appropriate CPU, memory, and disk size for your template needs.

2Remove ISO Attachment

After the VM is created but before converting to a template, detach the installation ISO from the VM's CD/DVD drive. This prevents the template from trying to boot from the ISO.

3Add Cloud-Init Drive

Navigate to the VM's hardware tab and add a Cloud-Init drive. This creates a special disk that stores cloud-init configuration data.

4Configure Cloud-Init Settings

Set up the default credentials and access methods for VMs cloned from this template:

• User: Define the default username for cloned VMs
• Password: Set a secure password (can be changed on individual clones)
• SSH Public Key: Paste your SSH public key for passwordless authentication

5Convert to Template

Right-click on the VM and select "Convert to Template". The VM icon will change to indicate it's now a template and can no longer be started directly.

6Clone the Template

To create a new VM from the template, right-click the template and select "Clone". Choose "Full Clone" for a completely independent copy, or "Linked Clone" for faster deployment with shared base disk (requires specific storage types like LVM-Thin or ZFS).

Linux Containers (LXC)

Linux Containers (LXC) are lightweight virtualization solutions that share the host kernel while providing isolated userspace environments. Unlike full virtual machines that emulate hardware, containers have significantly lower overhead, faster startup times, and better resource efficiency. They're ideal for running multiple isolated services on the same host.

Understanding LXC Containers

Creating a Container Template

Step 1: Download Container Template

Container templates are pre-built minimal root filesystems for various Linux distributions.

Step 2: Create Container

1Click "Create CT"

• Node: Select the Proxmox host where the container will run
• CT ID: Unique identifier (e.g., 100, 101, 102...)
• Hostname: DNS hostname for the container (e.g., webserver01)
• Password: Root password for the container
• SSH Public Key: Optional but recommended for secure access

2Template Selection

3Root Disk Configuration

4CPU Allocation

5Memory Configuration

6Network Settings

Container Options

First Login and Hardening

After starting the container, you can log in through the Proxmox console using the root account and the password you set during creation.

Converting Containers to Templates

Creating reusable container templates ensures consistency across deployments and significantly speeds up provisioning. The key to a good template is removing all system-specific identifiers so each clone starts with a unique identity.

Why Remove SSH Keys and Machine IDs?

Preparation Steps

Step 1: Update and Clean the System

Step 2: Delete SSH Host Keys

SSH host keys uniquely identify a machine. Removing them ensures each clone generates its own unique keys on first boot.

Step 3: Clear Machine ID

The machine ID is used by systemd, DHCP clients, and various system services to uniquely identify the host. Truncating (not deleting) this file allows systemd to regenerate a new unique ID on the first boot of each clone.

Step 4: Power Off and Convert

In Proxmox web interface: Right-click container → Convert to Template

Cloning from Template

Right-click the template and select Clone → Full Clone. Give the clone a new CT ID and hostname.

Post-Clone Configuration

After the first boot of your clone, you must regenerate SSH host keys since there's no Cloud-Init for LXC containers.

Regenerate SSH Host Keys

Additional Post-Clone Checklist

Proxmox User Management

Proxmox supports multiple authentication realms, allowing you to choose where user credentials are stored and validated. Understanding these realms is crucial for implementing proper access control and security policies.

Authentication Realms Explained

PAM vs PVE Realm

Example: Creating a PVE User

Role-Based Access Control (RBAC)

Proxmox uses RBAC to assign permissions. A user's effective permissions are determined by their assigned roles on specific resources (VMs, storage, nodes, etc.).

Built-in Roles

Example: Granting VM Access to a User

Backups vs Snapshots

Understanding the difference between backups and snapshots is critical for implementing an effective disaster recovery strategy. While both capture the state of a VM or container, they serve different purposes and have different characteristics.

Comparison Overview

When to Use Snapshots

When to Use Backups

Working with Snapshots

Creating a Snapshot

Restoring a Snapshot

Deleting Snapshots

Working with Backups

Backup Storage Configuration

Before creating backups, ensure you have a storage location configured for backup files:

Creating a Manual Backup

Backup Modes Explained

Scheduling Automatic Backups

Manual backups are unreliable because humans forget. Automation is essential for proper data protection.

Backup Retention Policy

Instead of simply setting "Max Backups", modern Proxmox versions support granular retention:

The 3-2-1 Backup Rule

Backup Storage Options

Option 1: Proxmox Backup Server (PBS) ⭐ Recommended

Option 2: NAS (Network Attached Storage)

NAS devices like TrueNAS, Synology, or QNAP are popular for home labs and small businesses.

Option 3: USB Drive (Development/Testing Only)

Option 4: Cloud Storage (Offsite Copy)

Use tools like rclone to sync your backup directory to cloud providers (S3, Backblaze B2, Google Drive, etc.) for offsite protection.

Restoring a Backup

Recommended Setup Summary

Integrating QNAP NAS for Backups

QNAP NAS devices provide excellent network storage for Proxmox backups. They support both NFS and SMB/CIFS protocols, with NFS being the preferred choice for Linux-based systems due to better performance and native compatibility.

Why QNAP NAS for Proxmox Backups?

• Dedicated storage: Separate hardware from your Proxmox host
• RAID protection: Built-in redundancy protects against disk failures
• Snapshots: QNAP's own snapshot feature adds another recovery layer
• Easy management: Web-based interface for configuration
• Expansion: Can easily add more drives as backup needs grow

Part 1: Configure NFS Share on QNAP

Step 1: Enable NFS Service

Step 2: Create Dedicated Backup Folder

Step 3: Configure NFS Permissions

Part 2: Add QNAP Share to Proxmox

Using NFS (Recommended)

Using SMB/CIFS (Alternative)

If you prefer SMB/CIFS or have mixed Windows/Linux environments:

Part 3: Configure Backup Job

Verification & Testing

Test Manual Backup

Verify on QNAP

Troubleshooting Common Issues

Issue: "Permission Denied" when backing up

Issue: Export path not appearing in dropdown

Best Practices for QNAP + Proxmox

Proxmox Firewall Configuration

Proxmox includes a built-in firewall that operates at multiple levels: datacenter-wide, per-node, and per-VM/container. This layered approach provides granular control over network access while maintaining security.

Firewall Architecture Levels

Emergency Recovery Access

If you lock yourself out by misconfiguring the firewall, use IPMI or KVM console access:

Initial Firewall Configuration

Step 1: Configure Rules BEFORE Enabling Firewall

Allow Web UI Access (Port 8006)

Allow SSH Access (Port 22)

Allow ICMP (Ping)

Step 2: Set Default Policy

Step 3: Enable Firewall

Only after confirming all necessary rules are in place:

Per-VM/Container Firewall Rules

You can add specific rules for individual VMs or containers. For example, to allow HTTP/HTTPS to a web server VM:

Common Firewall Macros

Firewall Best Practices

Firewall Logging

Enable logging to track firewall activity and troubleshoot connectivity issues:

Proxmox CLI Commands

While the Proxmox web interface is powerful, command-line tools provide faster access for experienced administrators and enable automation through scripts. The primary tools are qm (QEMU Manager for VMs) and pct (Proxmox Container Toolkit for LXC containers).

VM Management with qm

Basic VM Operations

Lists all virtual machines with their VMID, name, status, memory, and CPU allocation.

Starts VM with ID 101. The VM boots through its normal boot process.

Sends an ACPI shutdown signal to VM 101. The guest OS receives the signal and performs a graceful shutdown, similar to pressing the power button on a physical machine.

Gracefully reboots VM 101 by sending an ACPI reboot signal.

Performs a hard reset of VM 101. This is equivalent to pressing the reset button on a physical computer. Use only when a VM is unresponsive, as it can cause data corruption.

Immediately stops VM 101 without shutting down the guest OS. Equivalent to pulling the power plug. Use only in emergencies.

VM Configuration

Displays all configuration parameters for VM 101, including CPU, memory, disks, network interfaces, and boot options.

Enables automatic start of VM 101 when the Proxmox host boots. Use --onboot 0 to disable.

Changes VM 101's RAM allocation to 2048 MB (2 GB). The VM must be shut down for this change to take effect.

Allocates 4 CPU cores to VM 101.

Changes the display name of VM 101 to "webserver01".

Advanced VM Operations

Creates a full clone of VM 101 as a new VM with ID 102.

Live-migrates VM 101 to another node called "pve2" in the cluster.

Container Management with pct

Basic Container Operations

Lists all containers with their CT ID, status, name, and resource allocation.

Starts container 104.

Gracefully shuts down container 104.

Restarts container 104.

Forcefully stops container 104.

Container Shell Access

Opens a root shell inside container 104. This is extremely useful for quick access without SSH. Similar to docker exec -it. Type exit to leave the container shell.

Container Configuration

Displays all configuration options for container 104.

Enables auto-start on host boot for container 104.

Changes container 104's memory limit to 1024 MB.

Allocates 2 CPU cores to container 104.

Changes the hostname inside container 104.

Quick Reference Table

Automation Examples

Bulk Start All VMs

Bulk Shutdown All Containers

List All Running VMs

Network Management & Segmentation

Proper network segmentation is crucial for security. By separating management traffic from VM traffic, you minimize the risk that a compromised VM can attack your Proxmox host or other infrastructure.

Why Separate Management and VM Networks?

Default Network Setup

By default, Proxmox creates a single Linux bridge (vmbr0) connected to your primary network interface (e.g., enp5s0f0). Both management and VMs share this bridge.

Creating a Separate VM Network

Hardware Requirements

To properly separate networks, you need:

• Two physical network interfaces on your Proxmox host (or VLANs on a single interface)
• A managed switch that supports VLANs (for VLAN-based separation)

Step 1: Create Second Linux Bridge

Network Configuration Example

Step 2: Configure VMs to Use New Bridge

For new VMs:

For existing VMs:

VLAN-Based Segmentation

If you only have one physical network interface but a managed switch, you can use VLANs:

Firewall Rules for Network Segmentation

After segmenting networks, add firewall rules to enforce isolation:

Best Practices

Shared Storage with TrueNAS

TrueNAS (formerly FreeNAS) is a powerful open-source network-attached storage (NAS) operating system based on FreeBSD. It provides enterprise-grade storage features like ZFS, snapshots, replication, and high availability. Integrating TrueNAS with Proxmox enables centralized storage for VMs, containers, backups, and ISO images.

Why Use TrueNAS with Proxmox?

Creating a Dataset on TrueNAS

Step 1: Create Storage Pool

If you haven't already created a ZFS pool on TrueNAS:

Step 2: Create Dataset for Proxmox

Step 3: Configure Permissions

Sharing Storage via NFS

Step 1: Enable NFS Service

Step 2: Create NFS Share

Adding TrueNAS Storage to Proxmox

For VM/Container Storage

For Backups (Separate Dataset Recommended)

Create a second dataset specifically for backups:

Storage Types and Use Cases

Using Shared Storage

Creating a VM on TrueNAS Storage

Migrating Existing VM to TrueNAS

TrueNAS Snapshots for Additional Protection

TrueNAS ZFS snapshots are independent of Proxmox snapshots and provide an additional recovery layer:

Schedule Automatic Snapshots

Performance Considerations

Proxmox Clustering (Without High Availability)

A Proxmox cluster allows you to manage multiple Proxmox nodes from a single web interface, share resources, and migrate VMs between hosts. This section covers basic clustering without high availability features—HA requires at least 3 nodes and is covered in the next section.

What is a Proxmox Cluster?

Prerequisites

• At least 2 Proxmox nodes (can be physical servers or VMs)
• Each node must have unique hostname and IP address
• All nodes must be on the same network or have reliable network connectivity
• Time synchronization (NTP) across all nodes
• For migration: shared storage (NFS, Ceph, iSCSI, etc.) or sufficient bandwidth for offline migration

Network Preparation

Before creating a cluster, configure a dedicated network bridge on each Proxmox node for cluster communication. This is optional but recommended for separating cluster traffic from VM traffic.

Create Cluster Network Bridge (on each node)

Creating the Cluster

Step 1: Create Cluster on Primary Node

Step 2: Get Join Information

Step 3: Join Additional Nodes to Cluster

Step 4: Verify Cluster

VM/Container Migration

Once clustered, you can migrate VMs and containers between nodes. There are two types of migration:

Offline Migration (VM powered off)

Suitable when you don't need zero downtime. Works with both local and shared storage.

Online/Live Migration (VM stays running)

Zero-downtime migration. Requires shared storage or very fast network.

Migration Performance Considerations

Troubleshooting Cluster Issues

Node Not Appearing After Join

Cannot Join Cluster

Cluster Management Commands (CLI)

Shows cluster status, quorum information, and node list.

Lists all nodes in the cluster.

Temporarily allows cluster to operate with only 1 node (emergency use only).

High Availability Clustering

High Availability (HA) is a critical feature that automatically restarts VMs on surviving nodes when a Proxmox host fails. This minimizes downtime and ensures business continuity. However, HA is fundamentally different from basic clustering and has specific requirements.

Understanding High Availability

What HA Protects vs. What It Doesn't

The Quorum Problem: Why 3 Nodes Minimum?

Understanding Split-Brain

Imagine you have 2 nodes and the network link between them breaks. Both nodes are still running, but they can't communicate. Each node faces an impossible question:

How Quorum Solves This

Quorum requires a majority vote to make cluster decisions. The math is simple:

Workaround for 2-Node Clusters

Proxmox supports a QDevice (Quorum Device)—a lightweight third voter that doesn't run VMs. This can be:

• A Raspberry Pi
• A small VM on a separate physical host
• A low-power miniPC

The QDevice breaks ties in split-brain scenarios. However, 3 full nodes is still the recommended approach for production environments.

Setting Up High Availability

Prerequisites

• At least 3 Proxmox nodes in a working cluster
• Shared storage accessible from all nodes (NFS, Ceph, iSCSI, etc.)
• Reliable network connectivity between all nodes
• VMs you want to protect must be on shared storage

Step 1: Move VM to Shared Storage

HA only works with VMs on shared storage. If your VM is on local storage, move it first:

Step 2: Configure HA Resource

Step 3: Start the VM

HA Configuration Parameters Explained

Testing High Availability

After configuring HA, you should test it to ensure it works as expected.

Safe Test Method

What to Expect During Failover

1. 0-10 seconds: Cluster detects node is unresponsive
2. 10-30 seconds: Quorum confirms node is dead, begins fencing
3. 30-60 seconds: HA manager selects a new node and starts VM boot process
4. 60-120 seconds: VM boots on new node and becomes accessible

Live Migration vs. HA Fail