Python Cloudy

Infrastructure automation toolkit for cloud and server management

Overview

Python Cloudy is a comprehensive infrastructure automation toolkit that simplifies server configuration, database management, web server setup, and cloud deployments. Built on top of Fabric, it provides over 120 organized commands for system administration and DevOps workflows.

Key Features

🚀 High-level deployment recipes for complete server setups
🗄️ Database automation (PostgreSQL, MySQL, Redis, Memcached)
🌐 Web server management (Apache, Nginx, Supervisor)
☁️ Cloud integration (AWS EC2)
🔒 Security & firewall configuration
🔧 System administration utilities

Quick Start

Automated Setup (Recommended)

git clone https://github.com/un33k/python-cloudy
cd python-cloudy
./bootstrap.sh

Manual Setup

python3 -m venv .venv
source .venv/bin/activate
pip install -e .

Verify Installation

source .venv/bin/activate
fab -l

Important: Sudo Password Configuration

⚠️ CRITICAL: Due to underlying issues with Fabric, Python Cloudy requires explicit sudo password configuration since interactive password prompts are not supported in automated deployments.

For any operations requiring sudo privileges (when not running as root), you must export the sudo password as an environment variable:

export INVOKE_SUDO_PASSWORD=your_sudo_password

This applies to all non-root operations including:

System administration tasks
Package installations
Service management
Configuration file updates

Output Control & Debugging

Python Cloudy features a smart output system that provides clean, professional command execution while maintaining full debugging capabilities when needed.

⚡ Quick Start: Use CLOUDY_VERBOSE=1 before any command to see full output:

# Example: PostgreSQL installation with verbose output
CLOUDY_VERBOSE=1 fab -H [email protected] recipe.psql-install --cfg-paths="./.cloudy.psql"

# Example: System update with verbose output  
CLOUDY_VERBOSE=1 fab -H admin@server:22022 sys.update

Output Control Modes

Default Mode (Smart Output)

By default, Python Cloudy intelligently categorizes commands:

Shows: Status commands (ufw status, systemctl status, df, ps, etc.)
Hides: Noisy installation commands (apt install, wget, make, pip install, pg_createcluster, etc.)
Indicators: Success (✅) or failure (❌) messages for hidden commands

# Clean output - hides installation noise, shows status information
fab -H admin@server:22022 db.pg.install
fab -H admin@server:22022 fw.status  # Always shows output

Verbose Mode

Shows all command output for debugging and troubleshooting. Note: Fabric does not have a --verbose flag, so use the environment variable:

# Show all command output using environment variable
export CLOUDY_VERBOSE=1
fab -H admin@server:22022 db.pg.install
fab -H admin@server:22022 sys.update

# Or inline for single command
CLOUDY_VERBOSE=1 fab -H admin@server:22022 db.pg.install

# Clear verbose mode
unset CLOUDY_VERBOSE

Debug Mode

Enables Fabric's built-in debug mode plus all command output:

# Enable debug mode with full output
fab -H admin@server:22022 --debug db.pg.install
fab -H admin@server:22022 -d fw.secure-server

Echo Mode

Echo commands before execution (Fabric built-in):

# Echo commands before running
fab -H admin@server:22022 --echo sys.hostname --hostname=myserver
fab -H admin@server:22022 -e web.nginx.install

Recipe Success Messages

All deployment recipes provide comprehensive success summaries:

fab -H root@server recipe.gen-install --cfg-file="./.cloudy.generic"

Example Output:

🎉 ✅ GENERIC SERVER SETUP COMPLETED SUCCESSFULLY!
📋 Configuration Summary:
   ├── Hostname: myserver.example.com
   ├── Timezone: America/New_York
   ├── Admin User: admin (groups: admin,www-data)
   ├── SSH Port: 22022
   ├── Root Login: Disabled
   ├── SSH Keys: Configured
   └── Firewall: UFW enabled and configured

🚀 Generic server foundation is ready for specialized deployments!
   └── SSH Access: admin@server:22022 (key-based authentication)

Environment Variables

For programmatic control, you can use environment variables:

# Enable verbose output via environment variable (RECOMMENDED)
export CLOUDY_VERBOSE=1
fab -H server sys.update

# Or use inline for single commands
CLOUDY_VERBOSE=1 fab -H server sys.update

# Clear verbose mode
unset CLOUDY_VERBOSE

Note: CLOUDY_VERBOSE=1 is the recommended way to enable verbose output since Fabric does not have a built-in --verbose flag.

Best Practices

Development: Use CLOUDY_VERBOSE=1 or --debug when troubleshooting issues
Production: Use default mode for clean output and success confirmations
Automation: Set CLOUDY_VERBOSE=1 in CI/CD environments for full logs
Learning: Use --echo to see exact commands being executed

Quick Reference

# Default mode (clean output)
fab -H server sys.update

# Verbose mode (show all output)
CLOUDY_VERBOSE=1 fab -H server sys.update

# Debug mode (Fabric debug + all output)
fab -H server --debug sys.update

# Echo mode (show commands before running)
fab -H server --echo sys.update

Usage Examples

Secure Server Deployment Workflow

⚠️ IMPORTANT: Python Cloudy implements a secure two-phase deployment:

Phase 1: Initial Setup (as root)

# Setup secure server - creates admin user, disables root login, configures firewall
source .venv/bin/activate
fab -H [email protected] recipe.gen-install --cfg-file="./.cloudy.generic"

After this step:

✅ Root login is disabled for security
✅ Admin user created with SSH key authentication
✅ SSH port changed (default: 22022)
✅ UFW firewall configured

Phase 2: Ongoing Management (as admin user)

⚠️ CRITICAL: Due to underlying Fabric issues, for any sudo operations, you must export the password as an environment variable since interactive password prompts are not supported:

# Set sudo password for automation (REQUIRED for sudo operations)
export INVOKE_SUDO_PASSWORD=your_admin_password

# Install additional services (Nginx, PostgreSQL, etc.)
fab -H [email protected]:22022 web.nginx.install
fab -H [email protected]:22022 db.pg.install
fab -H [email protected]:22022 fw.allow-http
fab -H [email protected]:22022 fw.allow-https

# Use environment variable for verbose output or --debug flag
CLOUDY_VERBOSE=1 fab -H [email protected]:22022 db.pg.status
fab -H [email protected]:22022 --debug fw.status

Other High-Level Recipes

# Redis cache server setup
fab -H [email protected] recipe.redis-install --cfg-file="./.cloudy.redis"

# Django web server setup  
fab -H [email protected] recipe.web-install --cfg-file="./.cloudy.web"

# PostgreSQL + PostGIS database setup
fab -H [email protected] recipe.psql-install --cfg-file="./.cloudy.db"

# Use environment variable to see detailed installation progress
CLOUDY_VERBOSE=1 fab -H [email protected] recipe.redis-install --cfg-file="./.cloudy.redis"

Database Management

# PostgreSQL operations
fab -H [email protected] db.pg.create-user --username=myuser --password=secure123
fab -H [email protected] db.pg.create-db --database=myapp --owner=myuser
fab -H [email protected] db.pg.grant-privs --database=myapp --username=myuser

# MySQL operations  
fab -H [email protected] db.my.install
fab -H [email protected] db.my.create-db --database=wordpress
fab -H [email protected] db.my.create-user --username=wpuser --password=pass123

System Administration

# For non-root users, export sudo password first
export INVOKE_SUDO_PASSWORD=your_sudo_password

# System setup and updates
fab -H [email protected] sys.init
fab -H [email protected] sys.update
fab -H [email protected] sys.hostname --hostname=myserver.example.com

# User management
fab -H [email protected] sys.add-user --username=deploy
fab -H [email protected] sys.add-sudoer --username=deploy

# SSH configuration
fab -H [email protected] sys.ssh-port --port=2222
fab -H [email protected] sys.ssh-disable-root

# Use --echo to see exact commands being executed
fab -H [email protected] --echo sys.hostname --hostname=myserver.example.com

# System status checks (always show output)
fab -H admin@server:22022 sys.services  # Shows service status
fab -H admin@server:22022 sys.memory-usage  # Shows memory info

Firewall & Security

# Firewall setup
fab -H [email protected] fw.install
fab -H [email protected] fw.secure-server --ssh-port=2222
fab -H [email protected] fw.allow-http
fab -H [email protected] fw.allow-https
fab -H [email protected] fw.allow-postgresql

Web Server Setup

# Nginx setup
fab -H [email protected] web.nginx.install
fab -H [email protected] web.nginx.setup-domain --domain=example.com --proto=https

# Apache setup
fab -H [email protected] web.apache.install
fab -H [email protected] web.apache.configure-domain --domain=mysite.com

# Site management
fab -H [email protected] web.site.create-site-dir --domain=example.com
fab -H [email protected] web.site.create-venv --domain=example.com

Cloud Management (AWS)

# EC2 instance management
fab aws.list-nodes
fab aws.create-node --name=web-server --image=ami-12345 --size=t3.micro
fab aws.get-node --name=web-server
fab aws.destroy-node --name=web-server

Service Management

# Docker setup
fab -H [email protected] services.docker.install
fab -H [email protected] services.docker.add-user --username=deploy

# Redis configuration
fab -H [email protected] services.cache.install
fab -H [email protected] services.cache.configure
fab -H [email protected] services.cache.password --password=redis123

# VPN setup
fab -H [email protected] services.vpn.docker-install
fab -H [email protected] services.vpn.create-client --name=user1

Command Structure

Python Cloudy organizes commands into intuitive hierarchical namespaces:

recipe.* - High-level deployment recipes (7 commands)
sys.* - System administration (31 commands)
db.* - Database management (31 commands)
- db.pg.* - PostgreSQL (17 commands)
- db.my.* - MySQL (7 commands)
- db.pgb.* - PgBouncer (3 commands)
- db.pgp.* - PgPool (2 commands)
- db.gis.* - PostGIS (4 commands)
web.* - Web servers (13 commands)
fw.* - Firewall (9 commands)
services.* - Service management (17 commands)
aws.* - Cloud management (16 commands)

Global Flags (Available for any command)

--debug, -d - Enable Fabric debug mode + all output
--echo, -e - Echo commands before running (Fabric built-in)
CLOUDY_VERBOSE=1 - Environment variable for verbose output

List All Commands

fab -l                    # Show all available commands
fab -l | grep "recipe\."  # Show only recipe commands
fab -l | grep "db\.pg\."  # Show only PostgreSQL commands

# Get help for Python Cloudy features
fab help                  # Show comprehensive help with examples

Configuration

Python Cloudy uses hierarchical configuration files with INI format:

Configuration Precedence (lowest to highest)

cloudy/cfg/defaults.cfg - Built-in defaults
~/.cloudy - User home directory config
./.cloudy - Current working directory config
--cfg-file - Explicitly passed files

Example Configuration

[COMMON]
git-user-full-name = John Doe
git-user-email = [email protected]
timezone = America/New_York
admin-user = admin
hostname = my-server
python-version = 3.11

[WEBSERVER]
webserver = gunicorn
webserver-port = 8080
domain-name = example.com

[DBSERVER]
pg-version = 17
db-host = localhost
db-port = 5432

Using Multiple Configs

fab -H [email protected] recipe.gen-install --cfg-file="./.cloudy.base,./.cloudy.production"

Development

Running Tests

./test.sh              # Run minimal test suite
python tests/test_runner.py  # Run tests directly

Code Quality

./lint.sh              # Run linting (Black, isort, flake8, mypy)

Environment Setup

source .venv/bin/activate    # Always activate before development

License

Released under the MIT license.

Versioning

X.Y.Z Versioning

MAJOR version: Incompatible API changes
MINOR version: Backwards-compatible functionality
PATCH version: Backwards-compatible bug fixes

Python Compatibility

Python Cloudy is compatible with Python versions 3.8 and above. Please ensure that your environment meets this requirement before installing.

Development

To contribute to Python Cloudy, please follow these steps:

Fork the repository on GitHub.
Create a new branch for your feature or bug fix.
Make your changes and commit them with clear, descriptive messages.
Push your branch to your fork on GitHub.
Submit a pull request to the main repository, detailing your changes and the problem they solve.

Please ensure that your code adheres to the existing style and conventions used in the project.

Name		Name	Last commit message	Last commit date
Latest commit History 276 Commits
.github/workflows		.github/workflows
.vscode		.vscode
cloudy		cloudy
tests		tests
.cspell.json		.cspell.json
.flake8		.flake8
.gitignore		.gitignore
.python-version		.python-version
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
LICENSE		LICENSE
README.md		README.md
__init__.py		__init__.py
bootstrap.sh		bootstrap.sh
fabfile.py		fabfile.py
lint.sh		lint.sh
pyproject.toml		pyproject.toml
requirements.txt		requirements.txt
test.sh		test.sh

License

un33k/python-cloudy

Folders and files

Latest commit

History

Repository files navigation

Python Cloudy

Overview

Key Features

Quick Start

Automated Setup (Recommended)

Manual Setup

Verify Installation

Important: Sudo Password Configuration

Output Control & Debugging

Output Control Modes

Default Mode (Smart Output)

Verbose Mode

Debug Mode

Echo Mode

Recipe Success Messages

Environment Variables

Best Practices

Quick Reference

Usage Examples

Secure Server Deployment Workflow

Phase 1: Initial Setup (as root)

Phase 2: Ongoing Management (as admin user)

Other High-Level Recipes

Database Management

System Administration

Firewall & Security

Web Server Setup

Cloud Management (AWS)

Service Management

Command Structure

Global Flags (Available for any command)

List All Commands

Configuration

Configuration Precedence (lowest to highest)

Example Configuration

Using Multiple Configs

Development

Running Tests

Code Quality

Environment Setup

License

Versioning

Sponsors

Python Compatibility

Development

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors 4

Uh oh!

Languages

Packages