diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ac22520..73e8598 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,99 +1,38 @@ # Architecture +Technical details for the hantim infrastructure. See README.md for usage +instructions, USECASES.md for expected behaviors. + ## Servers -### Hantim server (hantim) +### Hantim (Vultr, Rocky Linux 9) -The production application server. Runs Rocky Linux 9 on Vultr. +Production web server. This repo is cloned to `/opt/hantim`. -- Hosts all app containers and the nginx reverse proxy via Docker Compose -- Provisioned by `bootstrap.sh` + `configure.sh` in this repo -- This repo is cloned to `/opt/hantim` -- The `deploy` user receives SSH commands from CI to trigger deploys +- Runs nginx reverse proxy + all app containers via Docker Compose - Certbot runs on the host (not in Docker) and manages Let's Encrypt certs +- The `deploy` user receives SSH commands from CI to trigger deploys +- Garage storage node for media replication -### Gitea server (git.timothykim.net) +### Argento (home server) -A separate server running the Gitea instance. Provides: - -- Git hosting for all repos +- Gitea instance at `git.timothykim.net` - Docker container registry (`git.timothykim.net/hantim/`) -- Gitea Actions CI/CD (runner registered at site level) -- Organization-level secrets and variables shared across repos +- Gitea Actions CI runner (registered at site level) +- Second Garage storage node (replication partner) +- Garage web UI for administration ### Bitwarden Secrets Manager (cloud) -Secrets are stored in Bitwarden Secrets Manager (free tier, project: `hantim`). -The `bws` CLI fetches secrets using machine account access tokens. - -- `hantim-ci-registry-push` -- Gitea API token for registry access -- `hantim-deploy-ssh-private-key` -- deploy user SSH private key -- `hantim-deploy-ssh-public-key` -- deploy user SSH public key -- `hantim-new-app-script` -- Gitea API token for creating repos -- `hantim-vultr-api-key` -- Vultr API key for DNS management -- `hantim-garage-rpc-secret` -- Garage cluster RPC secret -- `hantim-garage-argento-node-id` -- Argento's Garage node ID + address -- `hantim-garage-admin-token` -- Garage admin API token -- `hantim-garage-media-key-id` -- S3 access key ID for media uploads -- `hantim-garage-media-secret-key` -- S3 secret key for media uploads +All secrets stored in project `hantim`, fetched via `bws` CLI. Machine accounts: -- `hantim-server` -- used by `bootstrap.sh` and `configure.sh` on the server - (access token at `/etc/bws-token`) -- `hantim-ci` -- reserved for future CI use +- `hantim-server` — used by `bootstrap.sh` and `configure.sh` (token at `/etc/bws-token`) +- `hantim-ci` — reserved for future CI use -### Gitea secrets - -Workflow secrets stored in the `hantim` Gitea org. These duplicate some -Bitwarden secrets for direct use in CI workflows without installing `bws`: - -- `DEPLOY_SSH_KEY` -- deploy SSH private key -- `CI_REGISTRY_TOKEN` -- Gitea registry token -- `DEPLOY_HOST` -- server IP (variable, not secret) - -## Repos - -All repos live under the `hantim` organization on Gitea. - -### hantim-server - -This repo. Contains server provisioning, Docker Compose configs, nginx -configs, deploy scripts, and Gitea Actions workflows. - -``` -scripts/bootstrap.sh # First-time server setup (manual, uses Bitwarden) -scripts/configure.sh # Idempotent server config (CI-safe) -scripts/deploy.sh # Receives deploy + cert + provision commands via SSH -tools/new-app.sh # Scaffolds + deploys a new static site (run from dev machine) -tools/new-service.sh # Scaffolds a new Docker Compose service (run from dev machine) -tools/remove-app.sh # Removes a static site (run from dev machine) -docker/nginx/ # nginx reverse proxy - compose.yml # nginx:alpine container - nginx.conf # Main nginx config - conf.d/.conf # Per-app server blocks -docker//compose.yml # Per-app compose files -docker/garage/ # Garage S3-compatible object storage - compose.yml - Dockerfile - garage.toml - .env.keys -.gitea/workflows/deploy-.yml # Per-app deploy workflows -.gitea/workflows/provision.yml # Triggers configure.sh on setup changes -``` - -### static-site-template - -A Gitea template repo used by `new-app.sh` to scaffold new static sites. -Contains a Dockerfile, nginx config, placeholder HTML, and a `build.yml` -workflow that builds/pushes a Docker image and triggers a deploy. - -Uses Gitea template variables (`${REPO_NAME}`) so the image tag and deploy -command are automatically set to the repo name. - -### App repos (e.g., example.com) - -Each app is its own repo created from `static-site-template`. Contains the -app source code and a `build.yml` workflow. +Gitea org-level secrets (`hantim`) duplicate some bws secrets for direct +CI use without installing `bws`: `DEPLOY_SSH_KEY`, `CI_REGISTRY_TOKEN`, +`DEPLOY_HOST` (variable). ## Network topology @@ -103,16 +42,16 @@ app source code and a `build.yml` workflow. [Hantim Server] | nginx (port 80/443) - | | - example_com example_org ... - (shared Docker network) + / | | \ + example_com app_two garage ... + (shared Docker network) ``` - nginx is the only container with host port bindings (80, 443) - All app containers are on the `shared` Docker network -- nginx proxies `https://www.` to the app's container by name +- nginx proxies `https://www.` to the app container by name - The `shared` network is created by the nginx compose file and referenced - as `external: true` by app compose files + as `external: true` by all other compose files ## Deploy flow @@ -120,21 +59,20 @@ app source code and a `build.yml` workflow. ``` Developer pushes to app repo (e.g., example.com) - -> build.yml workflow runs on Gitea runner - -> docker build + push to git.timothykim.net registry + -> build.yml runs on Gitea runner + -> docker build + push to registry -> SSH to deploy@hantim as deploy- -> deploy.sh: git pull, docker compose pull, docker compose up -d - -> (if image not yet in registry, skips gracefully) ``` ### Infrastructure change (push to hantim-server) ``` Developer pushes to hantim-server - -> deploy-.yml triggers (based on changed paths under docker//) + -> deploy-.yml triggers (path match on docker//) -> SSH to deploy@hantim as deploy- -> deploy.sh: git pull, docker compose pull, docker compose up -d - -> (nginx only): auto-issues certs for new domains, then tests config with nginx -t + -> nginx: auto-issues certs for new domains, tests config with nginx -t ``` ### Setup change (push to hantim-server) @@ -143,18 +81,109 @@ Developer pushes to hantim-server Developer pushes change to scripts/configure.sh -> provision.yml triggers -> SSH to deploy@hantim as provision - -> deploy.sh: git pull, configure.sh (firewall, certbot, start services) + -> deploy.sh: git pull, runs configure.sh ``` -## Conventions +## SSH deploy protocol + +The `deploy` user's `authorized_keys` uses `command=` restrictions. The SSH +client sends a command string (e.g., `deploy-example.com`), and `deploy.sh` +parses it to determine the action: + +- `deploy-` — deploy a specific app +- `deploy-nginx` — deploy nginx (with cert handling) +- `cert-` — issue a cert only +- `provision` — run `configure.sh` + +## SSL certificate handling + +Cert names are derived from nginx conf filenames (e.g., `example.com.conf` → +cert `example.com`). All `server_name` values in the conf are included in the +cert as SANs. + +**New domain (nginx not yet running):** +1. `deploy.sh` detects missing cert files +2. Stops nginx temporarily +3. Issues cert via `certbot --standalone` +4. Starts nginx +5. On error, cleans up the temp conf file (trap) + +**New domain (nginx already running):** +1. Issues cert via `certbot --webroot` using the ACME challenge directory +2. Zero downtime + +**Renewal:** certbot timer/cron runs daily, deploy hook reloads nginx. + +## `.env.keys` mechanism + +Services needing secrets declare them in `.env.keys`: + +``` +ENV_VAR=bws-secret-name +``` + +During deploy, `deploy.sh`: +1. Reads each line from `.env.keys` +2. Fetches the secret value from bws: `bws secret get --access-token ...` +3. Writes `ENV_VAR=value` to `.env` +4. Docker Compose reads `.env` on startup + +If any secret is missing, deploy fails with an error (not a warning). + +## Naming conventions - **App name = domain** (e.g., `example.com`) -- **Container name** = domain with dots replaced by underscores (e.g., - `example_com`) -- **Cert name** = config filename without `.conf` (stored at `/etc/letsencrypt/live//`) -- **One domain per conf file** -- `deploy.sh` derives the cert name from the conf - filename and collects all `server_name` values for that cert. Putting unrelated - domains in the same conf file will bundle them into one cert incorrectly. +- **Container name** = domain with dots → underscores (e.g., `example_com`) - **Image name** = `git.timothykim.net/hantim/:latest` -- All HTTP traffic redirects to `https://www.` -- Bare domain HTTPS redirects to `https://www.` +- **Cert name** = nginx conf filename without `.conf` +- **One domain per conf file** — `deploy.sh` derives the cert name from the + filename and collects all `server_name` values for that cert + +## URL routing + +- All HTTP → HTTPS redirect +- Bare domain HTTPS → `https://www.` redirect +- Each domain gets its own conf file (even redirect-only domains) + +## Garage (S3-compatible storage) + +Two-node cluster: hantim + argento, `replication_factor = 2`. + +**Components:** +- Garage v2.2.0 (`dxflrs/garage:v2.2.0`) +- Port 3900: S3 API (proxied via `s3.hantim.net`) +- Port 3901: RPC (inter-node, firewall opened) +- Port 3902: Web endpoint (serves files via `Host: .web.garage`) +- Port 3903: Admin API (proxied via `garage.hantim.net`) + +**Media serving path:** +``` +Client -> nginx (443) -> /media/ location -> garage:3902 + (proxy_set_header Host .web.garage) +``` + +**Config generation:** `garage.toml` uses `envsubst` placeholders (`${RPC_SECRET}`, +`${ARGENTO_NODE_ID}`, `${ADMIN_TOKEN}`). The Dockerfile copies the template and +runs envsubst at container start. Values come from `.env` generated by the +`.env.keys` mechanism. + +**Bucket setup** (done by `new-app.sh` via admin API v2): +1. `POST /v2/CreateBucket` with `globalAlias: ` +2. `POST /v2/AllowBucketKey` to grant media-key read/write access +3. `POST /v2/UpdateBucket` to enable website access + +## App repos + +Every site repo follows the same pattern (created from `static-site-template`): + +``` +Dockerfile # Copies static/ into nginx:alpine, applies nginx.conf +nginx.conf # Per-app HTTP config (port 80, routing, cache headers) +static/ # HTML/CSS/JS content +static/media/ # Local media files (gitignored) +.gitea/workflows/build.yml # Build image, push to registry, SSH deploy +``` + +The template uses Gitea template variables (`${REPO_NAME}`) so the image tag +and deploy command match the repo name automatically. Workflow files must +escape `$$` to prevent template variable consumption. diff --git a/CLAUDE.md b/CLAUDE.md index a125105..0fdf0e5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,93 +1,44 @@ # CLAUDE.md -## Project overview - -Server provisioning and app management for the hantim webserver. This repo -lives at `/opt/hantim` on the production server (Rocky Linux 9 on Vultr) and -manages Docker Compose configs, deploy scripts, nginx configs, and Gitea -Actions workflows. +Quick context for Claude Code. See README.md for full details, ARCHITECTURE.md +for technical deep-dives, USECASES.md for expected behaviors. ## Key files - `scripts/bootstrap.sh` -- First-time server setup (manual, uses Bitwarden) - `scripts/configure.sh` -- Idempotent server config (firewall, certbot, start services; CI-safe) - `scripts/deploy.sh` -- SSH-triggered deploy + cert issuance + provision (deploy-*, cert-*, provision commands) -- `tools/new-app.sh` -- Single-command new static site (DNS + Gitea repo + cert + deploy + verify) +- `tools/new-app.sh` -- Single-command new static site (DNS + Gitea repo + cert + bucket + deploy + verify) - `tools/new-service.sh` -- Scaffold a new Docker Compose service (compose + workflow) - `tools/remove-app.sh` -- Remove a static site (local files + Gitea repo) -- `docker/nginx/` -- Reverse proxy (nginx:alpine + certbot on host) +- `docker/nginx/conf.d/` -- Per-app nginx server blocks - `docker//compose.yml` -- Per-app compose files -- `.gitea/workflows/deploy-.yml` -- Per-app deploy workflows -- `.gitea/workflows/provision.yml` -- Triggers configure.sh on setup changes +- `docker/garage/` -- Garage S3 object storage (Dockerfile, garage.toml, .env.keys) +- `.gitea/workflows/` -- Per-app deploy workflows + provision.yml ## Conventions - **App name = domain** (e.g., `example.com`) -- **Container name** = domain with dots replaced by underscores (e.g., `example_com`) -- **Image** = `git.timothykim.net/hantim/:latest` -- **Cert name** = config filename without `.conf` (at `/etc/letsencrypt/live//`) +- **Container name** = dots → underscores (e.g., `example_com`) - **One domain per conf file** -- deploy.sh derives cert name from filename -- All HTTP redirects to HTTPS; bare domain redirects to `www.` +- All HTTP → HTTPS; bare domain → `www.` +- All shell scripts use `set -euo pipefail` and must be idempotent - The `shared` Docker network is created by nginx compose and used by all apps +- Secrets are in Bitwarden Secrets Manager (see README.md for full list) +- Service secrets use `.env.keys` files (see README.md) -## Secrets +## Deploy flow -Bitwarden Secrets Manager (project: `hantim`, fetched via `bws` CLI): -- `hantim-ci-registry-push` -- Gitea registry token (used by bootstrap.sh) -- `hantim-deploy-ssh-private-key` -- deploy user SSH private key (used by new-app.sh) -- `hantim-deploy-ssh-public-key` -- deploy user SSH public key (used by bootstrap.sh) -- `hantim-new-app-script` -- Gitea API token for creating repos (used by new-app.sh) -- `hantim-vultr-api-key` -- Vultr API key for DNS management (used by new-app.sh) -- `hantim-garage-rpc-secret` -- Garage cluster RPC secret (used by deploy.sh) -- `hantim-garage-argento-node-id` -- Argento's Garage node ID + address (used by deploy.sh) -- `hantim-garage-admin-token` -- Garage admin API token (used by deploy.sh, new-app.sh) -- `hantim-garage-media-key-id` -- S3 access key ID for media uploads (used by new-app.sh) -- `hantim-garage-media-secret-key` -- S3 secret key for media uploads (used by aws CLI) - -Machine accounts: -- `hantim-server` -- access token stored at `/etc/bws-token` on hantim -- `hantim-ci` -- access token stored as Gitea secret (reserved for future use) - -Gitea org-level (`hantim`) secrets/variables: -- `DEPLOY_HOST` -- Variable (server IP) -- `DEPLOY_SSH_KEY` -- Secret (deploy SSH private key) -- `CI_REGISTRY_TOKEN` -- Secret (registry token) - -## Working with this repo - -### Adding a new static site - -Run `./tools/new-app.sh ` from dev machine. Handles everything: -DNS, Gitea repo, SSL cert, commit/push, build trigger, and verification. -See USECASES.md for full details. - -### Adding a new Docker service - -Run `./tools/new-service.sh ` to scaffold the compose file and deploy -workflow, then edit the compose file and commit. - -### Editing scripts - -All shell scripts use `set -euo pipefail`. Keep all steps idempotent. -`configure.sh` must be safe to re-run from CI. - -### Deploy flow - -App repo push -> `build.yml` builds image, pushes to registry, SSHes to server --> `deploy.sh` pulls repo, pulls image, restarts container. - -Infrastructure push (this repo) -> `deploy-.yml` triggers on path match --> same `deploy.sh` flow. nginx deploys test config with `nginx -t` first. - -Setup change push -> `provision.yml` triggers -> `deploy.sh provision` --> `configure.sh` (firewall, certbot, start services). +- App repo push → `build.yml` → build/push image → SSH `deploy-` → deploy.sh +- This repo push → `deploy-.yml` (path match) → SSH `deploy-` → deploy.sh +- Setup change → `provision.yml` → SSH `provision` → configure.sh ## Build and test No build step. No tests. Verify changes by: 1. Reading scripts and checking idempotency 2. Walking through each use case in USECASES.md -3. Cross-checking documentation (README.md, ARCHITECTURE.md, USECASES.md) against scripts +3. Cross-checking docs against scripts ## Commit style diff --git a/README.md b/README.md index 6a4a551..3380026 100644 --- a/README.md +++ b/README.md @@ -4,21 +4,25 @@ Server provisioning and app management for the hantim webserver. This repo lives at `/opt/hantim` on the server and contains Docker Compose configs, deploy scripts, nginx configs, and Gitea Actions workflows for each app. +See ARCHITECTURE.md for technical deep-dives, USECASES.md for expected behaviors. + +## Infrastructure overview + +Two servers and a cloud secrets service: + +- **Hantim** (Vultr, Rocky Linux 9) — production web server running nginx, + app containers, and a Garage storage node via Docker Compose +- **Argento** (home server) — Gitea instance (`git.timothykim.net`) with + Docker registry, CI runner, and a second Garage storage node (with web UI) +- **Bitwarden Secrets Manager** (cloud) — all secrets, fetched via the `bws` CLI + +All app containers share a `shared` Docker network. Only nginx binds host ports +(80/443). Garage provides S3-compatible media storage replicated across both +nodes. + ## Repo structure ``` -docker/ - nginx/ # Reverse proxy (nginx:alpine) + SSL config - nginx.conf # Main nginx config - conf.d/ # Per-app server blocks (e.g. example.com.conf) - compose.yml - / # Per-app static site (e.g. example.com) - compose.yml - garage/ # Garage S3-compatible object storage - compose.yml - Dockerfile - garage.toml - .env.keys scripts/ bootstrap.sh # First-time server setup (manual, uses Bitwarden) configure.sh # Idempotent server config (CI-safe) @@ -27,7 +31,19 @@ tools/ new-app.sh # Add a new static site (run from dev machine) new-service.sh # Add a new Docker service (run from dev machine) remove-app.sh # Remove a static site (run from dev machine) -.gitea/workflows/ # Per-app deploy workflows triggered by path changes +docker/ + nginx/ # Reverse proxy (nginx:alpine) + SSL config + nginx.conf # Main nginx config + conf.d/ # Per-app server blocks (e.g. example.com.conf) + compose.yml + / # Per-app compose files (e.g. example.com) + compose.yml + garage/ # Garage S3-compatible object storage + compose.yml + Dockerfile + garage.toml + .env.keys +.gitea/workflows/ # Per-app deploy workflows + provision.yml ``` ## Provisioning a new server @@ -40,50 +56,33 @@ git clone https://git.timothykim.net/hantim/hantim-server.git /opt/hantim bash /opt/hantim/scripts/bootstrap.sh ``` -`bootstrap.sh` runs once manually. It: +`bootstrap.sh` runs once manually. It installs dependencies, sets up the `bws` +CLI, creates the `deploy` user, and calls `configure.sh`. -1. Installs system dependencies (jq, certbot, unzip, Docker) -2. Installs the `bws` CLI (Bitwarden Secrets Manager) -3. Prompts for a Bitwarden Secrets Manager access token (saved to `/etc/bws-token`) -4. Fetches the Docker registry token and deploy SSH public key via `bws` -5. Logs into the Gitea Docker registry -6. Creates the `deploy` user with restricted SSH access and sudo -7. Runs `configure.sh` (firewall, certbot, start services) +`configure.sh` is idempotent and CI-safe. It opens firewall ports, sets up +certbot renewal, and deploys all services (nginx first, then apps). -`configure.sh` is idempotent and CI-safe. It: - -1. Opens firewall ports (HTTP, HTTPS, Garage RPC) -2. Sets up certbot renewal hooks and timer -3. Deploys all services via `deploy.sh` (nginx first, then all apps) - - Generates `.env` files from bws for services with `.env.keys` - - Issues SSL certs for new domains (nginx only) - - Builds or pulls images, starts containers - -**Prerequisites**: +**Prerequisites:** - DNS for all configured domains must point to the server - Bitwarden Secrets Manager access token for the `hantim-server` machine account -## Adding a new app +Restoring from backup is identical — run the same three commands on a fresh +server. Everything is idempotent. + +## Adding a new static site ```bash ./tools/new-app.sh ``` -Example: `./tools/new-app.sh example.com` - -This single command handles everything: creates DNS records on Vultr, creates -the Gitea repo from `static-site-template`, generates all config files, issues -the SSL cert (zero downtime), commits and pushes, triggers the first build, -and verifies the site is live. +This single command handles everything: DNS records (Vultr), Gitea repo +(from `static-site-template`), nginx/compose/workflow configs, SSL cert, +Garage media bucket, first build, and verification. **Prerequisites:** domain nameservers pointed to Vultr, `bws`/`jq`/`dig` -installed. Uses the deploy SSH key from Bitwarden Secrets Manager -- no -personal server access needed. See USECASES.md for full details. +installed, `BWS_ACCESS_TOKEN` env var or token at `~/.config/hantim/bws-token`. -`new-app.sh` also creates a Garage media bucket for the domain, allowing -media files to be served at `https://www./media/`. - -After the site is live, clone the app repo and customize it: +After the site is live, clone the app repo and customize: ```bash git clone git@git.timothykim.net:hantim/.git @@ -94,58 +93,40 @@ git add . && git commit -m "initial content" && git push ## Deploying changes -**App code change** (push to an app repo like `example.com`): -- The app's `build.yml` builds the Docker image, pushes to the registry, and - SSHes into the server to trigger a deploy +**App code change** — push to an app repo (e.g., `example.com`): +- `build.yml` builds the image, pushes to registry, SSHes to server to deploy -**Compose or nginx config change** (push to this repo): -- Gitea Actions workflows trigger based on which `docker//` paths changed -- `deploy.sh` pulls the latest code and runs `docker compose up -d` -- For nginx deploys, the config is tested before applying to prevent downtime +**Compose/nginx config change** — push to this repo: +- Path-matched workflows trigger and SSH to server +- nginx deploys test config before applying (`nginx -t`) -## Garage cluster initialization - -After provisioning a fresh server, Garage starts and connects to argento -automatically via `bootstrap_peers`. However, the cluster layout must be -assigned manually (node IDs change on fresh installs): - -```bash -# Check both nodes are connected -docker exec garage /garage status - -# Assign roles (use the node IDs from status output) -docker exec garage /garage layout assign -z hantim -c 1G -docker exec garage /garage layout assign -z argento -c 1G -docker exec garage /garage layout apply --version 1 -``` - -This only needs to be done once. If only hantim is reprovisioned, argento -retains the layout and hantim reconnects automatically. +**Setup change** — push changes to `scripts/configure.sh`: +- `provision.yml` triggers and re-runs `configure.sh` ## Media hosting -Media files (images, videos) are stored in Garage and served via nginx at -`/media/` on each domain. This keeps large files out of git repos. +Media files are stored in Garage and served via nginx at `/media/` on each +domain. This keeps large files out of git repos. **Endpoints:** -- `s3.hantim.net` — S3 API for uploads (authenticated with `media-key`) +- `s3.hantim.net` — S3 API for uploads (authenticated) - `garage.hantim.net` — Admin API for bucket management - `https://www./media/` — public file access -**Uploading media:** +**Uploading:** ```bash -# Configure AWS CLI (one-time) -aws configure # key ID, secret key from bws, region: garage, output: json +# One-time AWS CLI setup +aws configure # key ID + secret from bws, region: garage, output: json # Then add to ~/.aws/config under [default]: # endpoint_url = https://s3.hantim.net -# Upload files +# Upload aws s3 cp photo.jpg s3://example.com/photo.jpg aws s3 sync static/media/ s3://example.com/ ``` -**Local development:** Store media in `static/media/` (gitignored) so files +**Local development:** store media in `static/media/` (gitignored) so files are accessible at `/media/` when serving locally. **Creating buckets manually** (for existing domains without buckets): @@ -164,16 +145,26 @@ curl -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/jso "https://garage.hantim.net/v2/UpdateBucket?id=BUCKET_ID" ``` -## Restoring from backup +## Garage cluster initialization -Run the same three commands as provisioning. `bootstrap.sh` is idempotent: -- Existing packages are skipped -- Existing certs are skipped (or re-issued if the server is new) -- All services are started +After provisioning a fresh server, Garage connects to argento automatically. +The cluster layout must be assigned manually once (node IDs change per install): + +```bash +docker exec garage /garage status +docker exec garage /garage layout assign -z hantim -c 1G +docker exec garage /garage layout assign -z argento -c 1G +docker exec garage /garage layout apply --version 1 +``` + +If only hantim is reprovisioned, argento retains the layout and hantim +reconnects automatically. ## Secrets -Bitwarden Secrets Manager (project: `hantim`, fetched via `bws` CLI on the server): +### Bitwarden Secrets Manager + +Project: `hantim`. Fetched via `bws` CLI on the server. | Secret | Purpose | Used by | |---|---|---| @@ -192,12 +183,13 @@ Machine accounts: `hantim-server` (token at `/etc/bws-token`), `hantim-ci` (rese ### Service secrets (`.env.keys`) -Services that need secrets declare them in a `.env.keys` file (format: -`ENV_VAR=bws-secret-name`, one per line). `deploy.sh` reads this file, -fetches each secret from bws, and generates a `.env` file before starting -the service. See `docker/garage/.env.keys` for an example. +Services that need secrets declare them in `.env.keys` (format: +`ENV_VAR=bws-secret-name`, one per line). `deploy.sh` fetches each secret +from bws and generates `.env` before starting the service. -Gitea org-level (`hantim`) secrets/variables: +### Gitea org-level secrets + +Stored in the `hantim` Gitea org for direct use in CI workflows: | Name | Type | Purpose | |---|---|---| diff --git a/USECASES.md b/USECASES.md index 125bfb9..df2bbac 100644 --- a/USECASES.md +++ b/USECASES.md @@ -1,193 +1,99 @@ # Use Cases +Expected behaviors for each operation this repo supports. See README.md for +instructions, ARCHITECTURE.md for technical details. + ## 0. Initial Bitwarden Secrets Manager setup -One-time setup before anything else works. Create the secrets project and -machine accounts in Bitwarden Secrets Manager. +**Trigger:** one-time, before anything else works. -1. Create a Bitwarden organization (free tier) -2. Enable Secrets Manager for the organization -3. Create a project named `hantim` -4. Create all required secrets (see README.md Secrets table for the full list) -5. Create machine accounts: - - `hantim-server` — grant access to the `hantim` project - - `hantim-ci` — reserved for future CI use -6. Generate access tokens for each machine account +**Expected outcome:** +- Bitwarden organization with Secrets Manager enabled +- Project `hantim` with all secrets listed in README.md +- Machine accounts `hantim-server` and `hantim-ci` with project access +- Access tokens generated for each machine account ## 1. Provision a new server -Set up a fresh Rocky Linux 9 server from scratch. +**Trigger:** run `bootstrap.sh` on a fresh Rocky Linux 9 install. -```bash -dnf install -y git -git clone https://git.timothykim.net/hantim/hantim-server.git /opt/hantim -bash /opt/hantim/scripts/bootstrap.sh -``` +**Prerequisites:** DNS pointed to server, bws access token ready. -**What happens:** -- Installs all dependencies (jq, certbot, unzip, Docker) -- Installs `bws` CLI, prompts for Bitwarden Secrets Manager access token -- Fetches registry token and deploy SSH public key via `bws` -- Logs into Docker registry -- Creates deploy user with restricted SSH + sudo -- Runs `configure.sh`: - - Opens firewall ports (HTTP, HTTPS, Garage RPC) - - Sets up certbot renewal - - Deploys all services via `deploy.sh` (generates `.env` files, issues certs, starts containers) - -**Prerequisites:** -- DNS for all configured domains must point to the server -- Bitwarden Secrets Manager access token for the `hantim-server` machine account -- Docker images must exist in the Gitea registry (or apps will start on next push) +**Expected outcome:** +- All dependencies installed (Docker, certbot, bws, etc.) +- `deploy` user exists with restricted SSH and sudo +- Firewall allows HTTP, HTTPS, Garage RPC +- Certbot renewal configured +- All services running (nginx, apps, Garage) +- SSL certs issued for all configured domains +- `.env` files generated from bws for services with `.env.keys` ## 2. Restore from backup -Identical to provisioning a new server. Run the same three commands on a -fresh server. +**Trigger:** same three commands as provisioning on a fresh server. -- `bootstrap.sh` and `configure.sh` are idempotent -- safe to re-run -- SSL certs are re-issued automatically (Let's Encrypt) -- Docker images are pulled from the Gitea registry -- No data migration needed for stateless static sites +**Expected outcome:** identical to provisioning. All scripts are idempotent. +Certs are re-issued, images pulled from registry. No data migration needed +for stateless static sites. ## 3. Add a new static site -From your dev machine (single command): +**Trigger:** `./tools/new-app.sh example.com` from dev machine. -```bash -./tools/new-app.sh example.com -``` +**Prerequisites:** domain nameservers on Vultr, `bws`/`jq`/`dig` installed. -**Prerequisites:** -- Bitwarden Secrets Manager must contain `hantim-new-app-script`, - `hantim-vultr-api-key`, and `hantim-deploy-ssh-private-key` -- Domain nameservers must be pointed to Vultr (configured on directnic.com) -- Dependencies: `bws`, `jq`, `dig` -- `BWS_ACCESS_TOKEN` env var or token saved at `~/.config/hantim/bws-token` - -**What happens (fully automated):** -1. Resolves server IP from `hantim.net` -2. Fetches Gitea API token, Vultr API key, and deploy SSH key from Bitwarden Secrets Manager -3. Creates DNS zone on Vultr (if needed) and A records for bare + www -4. Waits for DNS to propagate -5. Creates Gitea repo `hantim/example.com` from `static-site-template` -6. Creates local files: - - `docker/example.com/compose.yml` (container name: `example_com`) - - `.gitea/workflows/deploy-example.com.yml` - - `docker/nginx/conf.d/example.com.conf` (HTTP->HTTPS, bare->www, proxy) -7. SSHes to server as deploy user, issues SSL cert via webroot (zero downtime) -8. Creates Garage media bucket via admin API (create, allow media-key, enable website) -9. Commits and pushes hantim-server (triggers deploy) -10. Triggers initial build of the app repo via Gitea API -11. Polls `https://www.example.com` until it responds (up to 3 minutes) - -After the site is live, clone the app repo and customize: - -```bash -git clone git@git.timothykim.net:hantim/example.com.git -cd example.com -# edit static/index.html, etc. -git add . && git commit -m "initial content" && git push -``` +**Expected outcome:** +- DNS A records created for bare + www +- Gitea repo `hantim/example.com` created from template +- Config files added: compose, nginx conf, deploy workflow +- SSL cert issued (zero downtime) +- Garage media bucket created with media-key access +- First build triggered +- `https://www.example.com` responds within 3 minutes +- Site ready for customization via git clone + push ## 4. Update app code -Push a change to the app repo (e.g., `example.com`): +**Trigger:** push to an app repo. -```bash -cd example.com -# make changes -git add . && git commit -m "update content" && git push -``` - -**What happens:** -1. Gitea Actions runs `build.yml`: builds Docker image, pushes to registry -2. SSHes into server, runs `deploy-example.com` -3. `deploy.sh` pulls latest hantim-server, pulls new image, restarts container - -No manual steps needed. +**Expected outcome:** +- Docker image built and pushed to registry +- Container restarted on server with new image +- No manual steps ## 5. Update Docker Compose config -Edit a compose file in this repo and push: +**Trigger:** push compose file change to this repo. -```bash -vim docker/example.com/compose.yml -git add docker/example.com/compose.yml -git commit -m "update example.com config" -git push -``` - -**What happens:** -1. `deploy-example.com.yml` workflow triggers (path match) -2. `deploy.sh` pulls latest code, pulls image, runs `docker compose up -d` +**Expected outcome:** +- Path-matched workflow triggers +- Container restarted with updated config ## 6. Update nginx config -Edit an nginx config in this repo and push: +**Trigger:** push nginx conf change to this repo. -```bash -vim docker/nginx/conf.d/example.com.conf -git add docker/nginx/conf.d/example.com.conf -git commit -m "update nginx config" -git push -``` +**Expected outcome:** +- Config tested with `nginx -t` before applying +- If test fails, running nginx is untouched +- If new certs needed, issued before applying config +- nginx reloaded with new config -**What happens:** -1. `deploy-nginx.yml` workflow triggers (path match) -2. `deploy.sh` pulls latest code -3. If any conf files reference certs that don't exist yet, nginx is briefly - stopped and certs are issued via standalone certbot (auto-restarts on failure) -4. Tests nginx config with `nginx -t` -- if it fails, deploy aborts and the - running nginx is untouched -5. Runs `docker compose up -d` and reloads nginx +## 7. Re-run configure -**Note:** Each conf file must contain server blocks for only one domain. The -cert name is derived from the filename (e.g. `example.com.conf` -> cert -`example.com`). Redirect-only domains get their own conf file. +**Trigger:** push change to `scripts/configure.sh`, or run manually. -## 7. Re-run configure on existing server - -Safe to do at any time, manually or via CI: - -```bash -cd /opt/hantim -git pull -bash scripts/configure.sh -``` - -Or push a change to `scripts/configure.sh` — the `provision.yml` workflow -triggers automatically. - -- All steps are idempotent -- Existing certs are skipped -- Brief nginx downtime (seconds) while certbot checks run -- Services are restarted +**Expected outcome:** +- All steps idempotent — safe to run anytime +- Existing certs skipped +- Services restarted ## 8. Upload media files -Media files (images, videos) are stored in Garage and served at `/media/` -on each domain. +**Trigger:** `aws s3 cp` or `aws s3 sync` to `s3://example.com/`. -```bash -# One-time setup -aws configure # key ID + secret from bws, region: garage, output: json -# Then add to ~/.aws/config under [default]: -# endpoint_url = https://s3.hantim.net - -# Upload a single file -aws s3 cp photo.jpg s3://example.com/photo.jpg - -# Sync a directory -aws s3 sync static/media/ s3://example.com/ -``` - -**What happens:** -1. File is uploaded to the Garage S3 API via `s3.hantim.net` -2. Garage replicates the file to both nodes (hantim + argento) -3. File is accessible at `https://www.example.com/media/photo.jpg` - -**Local development:** Store media in `static/media/` (gitignored). The app's -nginx serves these locally, matching the `/media/` path used in production. - -**Reference in HTML:** Use absolute paths like `/media/photo.jpg`. +**Expected outcome:** +- File uploaded via S3 API (`s3.hantim.net`) +- Replicated to both Garage nodes (hantim + argento) +- Accessible at `https://www.example.com/media/` +- Local development: `static/media/` (gitignored) serves at `/media/`