Automate VM creation and configuration using Infrastructure as Code principles
- Overview
- Architecture
- Prerequisites
- Project Structure
- Configuration Guide
- Usage Instructions
- Advanced Features
- Troubleshooting
This project provides a comprehensive automation solution for creating and configuring virtual machines in Proxmox using Terraform and Cloud-init. The system streamlines infrastructure deployment through Infrastructure as Code (IaC) principles, ensuring consistent and reproducible VM configurations.
- ✅ Automated VM provisioning with Terraform
- ✅ Template-based deployment for consistency
- ✅ Cloud-init integration for OS configuration
- ✅ QEMU Guest Agent setup and verification
- ✅ Network configuration automation
- ✅ Multi-disk support for data-intensive applications
graph TB
A[Terraform Configuration] --> B[Proxmox API]
B --> C[VM Template]
C --> D[Cloud-init Process]
D --> E[VM Instance]
F[Shell Scripts] --> G[Template Creation]
G --> C
H[user-data.yml] --> D
🔧 Component Breakdown
| Component | Purpose | Technology |
|---|---|---|
| Terraform | Infrastructure as Code orchestration | HashiCorp Terraform |
| Proxmox | Virtualization platform | Proxmox VE |
| Cloud-init | VM initialization and configuration | Cloud-init |
| Shell Scripts | Template creation and testing | Bash |
| QEMU Guest Agent | VM-host communication | QEMU |
🔍 System Requirements
- Proxmox VE installed on physical or virtual server
- API access configured with appropriate tokens
- Storage backend (local-lvm or shared storage)
- Network bridge configured (vmbr0)
- Terraform >= 1.0
- Cloud-init package
- QEMU Guest Agent
- SSH access to Proxmox host
- Proxmox API URL and credentials
- Sufficient privileges for VM creation
- Network connectivity to target infrastructure
├── 📄 main.tf # Primary Terraform configuration
├── 📄 variables.tf # Variable definitions
├── 📄 user-data.yml # Cloud-init configuration
├── 📄 create-template.sh # Template creation script
└── 📄 README.md # Project documentation
📋 File Descriptions
| File | Description | Purpose |
|---|---|---|
main.tf |
Main Terraform configuration | Defines VM resources and infrastructure |
variables.tf |
Variable definitions | Customizable parameters for deployment |
user-data.yml |
Cloud-init configuration | OS initialization and package installation |
create-template.sh |
Shell automation script | Template creation and validation |
Provider Configuration
Configure your Proxmox connection in variables.tf:
variable "pm_api_url" {
description = "Proxmox API URL"
type = string
default = "https://your-proxmox-server:8006/api2/json"
}
variable "pm_api_token_id" {
description = "Proxmox API Token ID"
type = string
default = "your-token-id"
}
variable "pm_api_token_secret" {
description = "Proxmox API Token Secret"
type = string
sensitive = true
default = "your-token-secret"
}Main Terraform Configuration
Complete main.tf Configuration:
terraform {
required_providers {
proxmox = {
source = "telmate/proxmox"
version = "~> 2.9"
}
}
}
provider "proxmox" {
pm_api_url = var.pm_api_url
pm_api_token_id = var.pm_api_token_id
pm_api_token_secret = var.pm_api_token_secret
pm_tls_insecure = var.pm_tls_insecure
}
resource "proxmox_vm_qemu" "cloudinit-example" {
vmid = 0
name = var.vm_name
target_node = "pve"
agent = 1
memory = var.vm_memory
boot = "order=scsi0"
clone = var.vm_clone
scsihw = "virtio-scsi-single"
vm_state = "running"
automatic_reboot = true
cicustom = "vendor=local:snippets/user-data.yml"
ciupgrade = true
ipconfig0 = "ip=dhcp,ip6=dhcp"
skip_ipv6 = true
ciuser = var.vm_ciuser
cipassword = var.vm_cipassword
cpu {
cores = var.vm_cpu
sockets = 1
type = "host"
}
serial {
id = 0
}
disks {
scsi {
scsi0 {
disk {
storage = "local-lvm"
size = var.vm_root_disk_size
}
}
scsi1 {
disk {
storage = "local-lvm"
size = var.vm_data_disk_size
}
}
}
ide {
ide1 {
cloudinit {
storage = "local-lvm"
}
}
}
}
network {
id = 0
bridge = "vmbr0"
model = "virtio"
}
}Cloud-init Configuration
Complete user-data.yml Configuration:
#cloud-config
hostname: user
user: user
password: user
chpasswd: { expire: False }
ssh_pwauth: True
package_update: true
package_upgrade: true
packages:
- qemu-guest-agent
- ansible
- curl
- wget
- htop
- vim
write_files:
- path: /etc/systemd/system/qemu-guest-agent.service
content: |
[Unit]
Description=QEMU Guest Agent
BindsTo=dev-virtio\x2dports-org.qemu.guest_agent.0.device
After=dev-virtio\x2dports-org.qemu.guest_agent.0.device
[Service]
ExecStart=/usr/sbin/qemu-ga \
--method=virtio-serial \
--path=/dev/virtio-ports/org.qemu.guest_agent.0 \
--blacklist=guest-file-open,guest-file-close,guest-file-read,guest-file-write,guest-file-seek,guest-file-flush
Restart=always
RestartSec=0
[Install]
WantedBy=multi-user.target
runcmd:
- systemctl daemon-reload
- systemctl enable qemu-guest-agent.service
- systemctl start qemu-guest-agent.service
- echo "Cloud-init configuration completed" >> /var/log/cloud-init-custom.logAutomated Template Creation
Run the template creation script:
chmod +x create-template.sh
./create-template.shScript Functions:
- ✅ Downloads base OS image
- ✅ Configures Cloud-init integration
- ✅ Installs QEMU Guest Agent
- ✅ Validates template functionality
- ✅ Prepares for Terraform deployment
Infrastructure Deployment
# Initialize Terraform
terraform init
# Validate configuration
terraform validate
# Plan deployment
terraform plan
# Apply configuration
terraform applyDeployment Process:
- 🔍 Planning Phase - Reviews infrastructure changes
- 🚀 Apply Phase - Creates VM resources
- ✅ Validation Phase - Confirms successful deployment
Post-Deployment Verification
Automated Checks:
# Check VM status
terraform show
# Validate QEMU Guest Agent
proxmox-vm-test.sh
# Network connectivity test
ping -c 4 <vm-ip-address>Manual Verification:
- SSH connectivity
- Service status verification
- Resource allocation confirmation
- Network configuration validation
🔧 Customization Options
| Variable | Description | Default | Type |
|---|---|---|---|
vm_name |
Virtual machine name | "terraform-vm" |
string |
vm_memory |
RAM allocation (MB) | 2048 |
number |
vm_cpu |
CPU core count | 2 |
number |
vm_root_disk_size |
Root disk size | "20G" |
string |
vm_data_disk_size |
Data disk size | "50G" |
string |
resource "proxmox_vm_qemu" "cluster_nodes" {
count = var.vm_count
name = "${var.vm_name}-${count.index + 1}"
# ... rest of configuration
}🔒 Security Enhancements
# Add to user-data.yml
ssh_authorized_keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAA... user@hostname
# Disable password authentication
ssh_pwauth: False# Add to runcmd section
runcmd:
- ufw enable
- ufw default deny incoming
- ufw default allow outgoing
- ufw allow ssh🚨 Common Issues and Solutions
Symptoms:
- VM shows as "QEMU Guest Agent not running"
- Terraform cannot detect VM IP address
Solution:
# Check service status
systemctl status qemu-guest-agent
# Restart service
systemctl restart qemu-guest-agent
# Verify configuration
cat /etc/systemd/system/qemu-guest-agent.serviceSymptoms:
- Default hostname not changed
- Packages not installed
- User not created
Solution:
# Check cloud-init logs
sudo cloud-init status --long
sudo journalctl -u cloud-init
# Re-run cloud-init
sudo cloud-init clean
sudo cloud-init initSymptoms:
- API connection timeouts
- Authentication errors
- Resource creation failures
Solution:
# Verify API connectivity
curl -k https://your-proxmox:8006/api2/json/version
# Check token permissions
# Ensure token has appropriate VM privileges
# Increase timeout values in provider configuration
provider "proxmox" {
pm_timeout = 600
# ... other configuration
}📊 Monitoring and Logging
| Component | Log Location | Purpose |
|---|---|---|
| Cloud-init | /var/log/cloud-init.log |
Initialization logs |
| QEMU Guest Agent | /var/log/qemu-ga.log |
Agent communication |
| Terraform | terraform.log |
Infrastructure operations |
| System | /var/log/syslog |
General system events |
# Real-time log monitoring
tail -f /var/log/cloud-init.log
# Service status check
systemctl status qemu-guest-agent
# Resource usage
htop
df -hThis automation framework provides a robust foundation for managing Proxmox virtual machines through Infrastructure as Code principles. The combination of Terraform, Cloud-init, and shell scripting creates a powerful ecosystem for:
- 🚀 Rapid deployment of consistent VM configurations
- 🔄 Reproducible infrastructure across environments
- 📊 Scalable operations for multiple VM management
- 🛡️ Reliable automation with comprehensive error handling
- Customize the configuration for your specific requirements
- Extend the automation with additional services
- Integrate with CI/CD pipelines for automated deployments
- Scale the solution for production workloads