Infrastructure repo

This repository contains deployment tooling and templates for running services on your servers.

This README is written for a non-Python user and explains how to get started, run deployments, and where to find service templates.

Prerequisites

• Python 3.10+: required to run the tooling. Install using your system package manager (e.g. apt, dnf, brew).
• A working SSH user with access to your target hosts.
• pyinfra CLI is used by the deployment scripts. It will be installed from requirements.txt below.

Quick setup (Linux)

1. Open a terminal in the repository root.
2. Create and activate a Python virtual environment and install dependencies:

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

3. Verify pyinfra is available:

pyinfra --help

If pyinfra is not found after installing, ensure your virtual environment is activated.

Where things live

• Repository root: deployment scripts and inventory.
• Templates: templates/ — each service has a subfolder containing unit, container and configuration templates.
• Services code: services/ — Python functions that translate templates into pyinfra operations.
• Inventory: inventory.py — list of hosts the deploy scripts use.
• Deploy scripts: deploy/deploy_all.py and deploy/update_all.py
• Encrypted secrets file: secrets.enc.yaml — this repository keeps secrets encrypted; handle with your usual secret-management workflow.

Inventory basics

Edit inventory.py to add your hosts. Each host is represented by a tuple: ("name", {"ssh_port": 22, "ssh_hostname": "1.2.3.4"}).

Example (add a host named myserver):

my_hosts.append((
    "myserver",
    {"ssh_port": 22, "ssh_hostname": "203.0.113.10"}
))

Running deployments

Deployment scripts are written for the pyinfra runner. Use pyinfra with the repo inventory.py to run the scripts.

Examples:

• Run the full deploy (applies all configured services for the host):

pyinfra @inventory.py deploy/deploy_all.py

• Run the update flow:

pyinfra @inventory.py deploy/update_all.py

Notes:

• Running these commands requires SSH access to the target hosts and proper keys configured on your machine.
• These scripts expect to be run from the repository root.

Understanding services and templates

• Services are implemented in services/. The list of available setup functions is in services/init.py.
• Each service has a corresponding directory under templates/. For example, templates/pdfding/ contains the container and systemd unit files used to deploy the pdfding service.
• To add or change behavior:
  • Edit existing templates under templates/<service>/ or add new files.
  • Update the corresponding services/*.py function to reference new templates or behaviour.

Example: check which services will run for a host

1. Open services/__init__.py to see the __all__ list. That list controls which service setup functions are invoked by deploy/deploy_all.py.

Troubleshooting

• If a dependency fails to install, ensure your Python version and pip are up-to-date and the virtual environment is active.
• If pyinfra cannot SSH to a host, check ssh keys and that ssh_hostname and ssh_port in inventory.py are correct.
• If you see template rendering errors, check the templates in templates/<service>/ for missing variables.

Security and secrets

This repository contains secrets.enc.yaml — it is encrypted. Do not commit unencrypted secrets. Use sops to decrypt the file. You need a SOPS_AGE_KEY variable defined in .env to run any of the scripts.

Using SOPS

This project uses Mozilla SOPS (with age keys) to encrypt secrets.enc.yaml.

1. Install SOPS (example):

# Debian/Ubuntu
sudo apt install sops

# Fedora
sudo dnf install sops

# macOS (Homebrew)
brew install sops

2. Provide the SOPS_AGE_KEY in a local .env file (do not commit .env). The key is available in samuel.kdbx under the entry named "Secret Operations" — copy the age private key text from there and put it into .env like:

SOPS_AGE_KEY=AGE-SECRET-KEY-1...   # replace with the real key from samuel.kdbx

3. Source .env and decrypt the secrets for local use:

source .env
sops -d secrets.enc.yaml > secrets.yaml

4. After editing secrets.yaml, re-encrypt before committing:

sops -e secrets.yaml > secrets.enc.yaml

5. Or you could use sops edit secrets.enc.yaml and it will allow you to edit and view the file from your $EDITOR

EDITOR=nvim sops edit secrets.enc.yaml

Notes:

• Never commit secrets.yaml or your .env file containing SOPS_AGE_KEY.
• If sops fails to decrypt, verify SOPS_AGE_KEY is correctly set and that you copied the private key value from samuel.kdbx under "Secret Operations".