hantim-server/ARCHITECTURE.md

# Architecture

Technical details for the hantim infrastructure. See README.md for usage
instructions, USECASES.md for expected behaviors.

## Servers

### Hantim (Vultr, Rocky Linux 9)

Production web server. This repo is cloned to `/opt/hantim`.

- Runs nginx reverse proxy + all app containers via Docker Compose
- Certbot runs on the host (not in Docker) and manages Let's Encrypt certs
- The `deploy` user receives SSH commands from CI to trigger deploys
- Garage storage node for media replication

### Argento (home server)

- Gitea instance at `git.timothykim.net`
- Docker container registry (`git.timothykim.net/hantim/<app>`)
- Gitea Actions CI runner (registered at site level)
- Second Garage storage node (replication partner)
- Garage web UI for administration

### Bitwarden Secrets Manager (cloud)

All secrets stored in project `hantim`, fetched via `bws` CLI.

Machine accounts:
- `hantim-server` — used by `bootstrap.sh` and `configure.sh` (token at `/etc/bws-token`)
- `hantim-ci` — reserved for future CI use

Gitea org-level secrets (`hantim`) duplicate some bws secrets for direct
CI use without installing `bws`: `DEPLOY_SSH_KEY`, `CI_REGISTRY_TOKEN`,
`DEPLOY_HOST` (variable).

## Network topology

```
                  Internet
                     |
              [Hantim Server]
                     |
            nginx (port 80/443)
           /    |         |     \
  example_com  app_two  garage  ...
         (shared Docker network)
```

- nginx is the only container with host port bindings (80, 443)
- All app containers are on the `shared` Docker network
- nginx proxies `https://www.<domain>` to the app container by name
- The `shared` network is created by the nginx compose file and referenced
  as `external: true` by all other compose files

## Deploy flow

### App code change (push to app repo)

```
Developer pushes to app repo (e.g., example.com)
  -> build.yml runs on Gitea runner
    -> docker build + push to registry
    -> SSH to deploy@hantim as deploy-<domain>
      -> deploy.sh: git pull, docker compose pull, docker compose up -d
```

### Infrastructure change (push to hantim-server)

```
Developer pushes to hantim-server
  -> deploy-<app>.yml triggers (path match on docker/<app>/)
    -> SSH to deploy@hantim as deploy-<app>
      -> deploy.sh: git pull, docker compose pull, docker compose up -d
      -> nginx: auto-issues certs for new domains, tests config with nginx -t
```

### Setup change (push to hantim-server)

```
Developer pushes change to scripts/configure.sh
  -> provision.yml triggers
    -> SSH to deploy@hantim as provision
      -> deploy.sh: git pull, runs configure.sh
```

## SSH deploy protocol

The `deploy` user's `authorized_keys` uses `command=` restrictions. The SSH
client sends a command string (e.g., `deploy-example.com`), and `deploy.sh`
parses it to determine the action:

- `deploy-<app>` — deploy a specific app
- `deploy-nginx` — deploy nginx (with cert handling)
- `cert-<domain>` — issue a cert only
- `provision` — run `configure.sh`

## SSL certificate handling

Cert names are derived from nginx conf filenames (e.g., `example.com.conf` →
cert `example.com`). All `server_name` values in the conf are included in the
cert as SANs.

**New domain (nginx not yet running):**
1. `deploy.sh` detects missing cert files
2. Stops nginx temporarily
3. Issues cert via `certbot --standalone`
4. Starts nginx
5. On error, cleans up the temp conf file (trap)

**New domain (nginx already running):**
1. Issues cert via `certbot --webroot` using the ACME challenge directory
2. Zero downtime

**Renewal:** certbot timer/cron runs daily, deploy hook reloads nginx.

## `.env.keys` mechanism

Services needing secrets declare them in `.env.keys`:

```
ENV_VAR=bws-secret-name
```

During deploy, `deploy.sh`:
1. Reads each line from `.env.keys`
2. Fetches the secret value from bws: `bws secret get <uuid> --access-token ...`
3. Writes `ENV_VAR=value` to `.env`
4. Docker Compose reads `.env` on startup

If any secret is missing, deploy fails with an error (not a warning).

## Naming conventions

- **App name = domain** (e.g., `example.com`)
- **Container name** = domain with dots → underscores (e.g., `example_com`)
- **Image name** = `git.timothykim.net/hantim/<domain>:latest`
- **Cert name** = nginx conf filename without `.conf`
- **One domain per conf file** — `deploy.sh` derives the cert name from the
  filename and collects all `server_name` values for that cert

## URL routing

- All HTTP → HTTPS redirect
- Bare domain HTTPS → `https://www.<domain>` redirect
- Each domain gets its own conf file (even redirect-only domains)

## Garage (S3-compatible storage)

Two-node cluster: hantim + argento, `replication_factor = 2`.

**Components:**
- Garage v2.2.0 (`dxflrs/garage:v2.2.0`)
- Port 3900: S3 API (proxied via `s3.hantim.net`)
- Port 3901: RPC (inter-node, firewall opened)
- Port 3902: Web endpoint (serves files via `Host: <bucket>.web.garage`)
- Port 3903: Admin API (proxied via `garage.hantim.net`)

**Media serving path:**
```
Client -> nginx (443) -> /media/ location -> garage:3902
  (proxy_set_header Host <domain>.web.garage)
```

**Config generation:** `garage.toml` uses `envsubst` placeholders (`${RPC_SECRET}`,
`${ARGENTO_NODE_ID}`, `${ADMIN_TOKEN}`). The Dockerfile copies the template and
runs envsubst at container start. Values come from `.env` generated by the
`.env.keys` mechanism.

**Bucket setup** (done by `new-app.sh` via admin API v2):
1. `POST /v2/CreateBucket` with `globalAlias: <domain>`
2. `POST /v2/AllowBucketKey` to grant media-key read/write access
3. `POST /v2/UpdateBucket` to enable website access

## App repos

Every site repo follows the same pattern (created from `static-site-template`):

```
Dockerfile              # Copies static/ into nginx:alpine, applies nginx.conf
nginx.conf              # Per-app HTTP config (port 80, routing, cache headers)
static/                 # HTML/CSS/JS content
static/media/           # Local media files (gitignored)
.gitea/workflows/build.yml  # Build image, push to registry, SSH deploy
```

The template uses Gitea template variables (`${REPO_NAME}`) so the image tag
and deploy command match the repo name automatically. Workflow files must
escape `$$` to prevent template variable consumption.