diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 4e7a282..db136ea 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -59,8 +59,9 @@ configs, deploy scripts, and Gitea Actions workflows. scripts/bootstrap.sh # First-time server setup (manual, uses Bitwarden) scripts/configure.sh # Idempotent server config (CI-safe) scripts/deploy.sh # Receives deploy + cert + provision commands via SSH -scripts/new-app.sh # Scaffolds + deploys a new static site (single command) -scripts/new-service.sh # Scaffolds a new Docker Compose service +tools/new-app.sh # Scaffolds + deploys a new static site (run from dev machine) +tools/new-service.sh # Scaffolds a new Docker Compose service (run from dev machine) +tools/remove-app.sh # Removes a static site (run from dev machine) docker/nginx/ # nginx reverse proxy compose.yml # nginx:alpine container nginx.conf # Main nginx config @@ -82,7 +83,7 @@ workflow that builds/pushes a Docker image and triggers a deploy. Uses Gitea template variables (`${REPO_NAME}`) so the image tag and deploy command are automatically set to the repo name. -### App repos (e.g., timothykim.net) +### App repos (e.g., example.com) Each app is its own repo created from `static-site-template`. Contains the app source code and a `build.yml` workflow. @@ -96,7 +97,7 @@ app source code and a `build.yml` workflow. | nginx (port 80/443) | | - timothykim_net hcsuzuki_net ... + example_com example_org ... (shared Docker network) ``` @@ -111,7 +112,7 @@ app source code and a `build.yml` workflow. ### App code change (push to app repo) ``` -Developer pushes to app repo (e.g., timothykim.net) +Developer pushes to app repo (e.g., example.com) -> build.yml workflow runs on Gitea runner -> docker build + push to git.timothykim.net registry -> SSH to deploy@hantim as deploy- @@ -140,9 +141,9 @@ Developer pushes change to scripts/configure.sh ## Conventions -- **App name = domain** (e.g., `hcsuzuki.net`) +- **App name = domain** (e.g., `example.com`) - **Container name** = domain with dots replaced by underscores (e.g., - `hcsuzuki_net`) + `example_com`) - **Cert name** = config filename without `.conf` (stored at `/etc/letsencrypt/live//`) - **One domain per conf file** -- `deploy.sh` derives the cert name from the conf filename and collects all `server_name` values for that cert. Putting unrelated diff --git a/CLAUDE.md b/CLAUDE.md index 9d45ed5..0f1a391 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -12,8 +12,9 @@ Actions workflows. - `scripts/bootstrap.sh` -- First-time server setup (manual, uses Bitwarden) - `scripts/configure.sh` -- Idempotent server config (firewall, certbot, start services; CI-safe) - `scripts/deploy.sh` -- SSH-triggered deploy + cert issuance + provision (deploy-*, cert-*, provision commands) -- `scripts/new-app.sh` -- Single-command new static site (DNS + Gitea repo + cert + deploy + verify) -- `scripts/new-service.sh` -- Scaffold a new Docker Compose service (compose + workflow) +- `tools/new-app.sh` -- Single-command new static site (DNS + Gitea repo + cert + deploy + verify) +- `tools/new-service.sh` -- Scaffold a new Docker Compose service (compose + workflow) +- `tools/remove-app.sh` -- Remove a static site (local files + Gitea repo) - `docker/nginx/` -- Reverse proxy (nginx:alpine + certbot on host) - `docker//compose.yml` -- Per-app compose files - `.gitea/workflows/deploy-.yml` -- Per-app deploy workflows @@ -21,8 +22,8 @@ Actions workflows. ## Conventions -- **App name = domain** (e.g., `hcsuzuki.net`) -- **Container name** = domain with dots replaced by underscores (e.g., `hcsuzuki_net`) +- **App name = domain** (e.g., `example.com`) +- **Container name** = domain with dots replaced by underscores (e.g., `example_com`) - **Image** = `git.timothykim.net/hantim/:latest` - **Cert name** = config filename without `.conf` (at `/etc/letsencrypt/live//`) - **One domain per conf file** -- deploy.sh derives cert name from filename @@ -51,13 +52,13 @@ Gitea org-level (`hantim`) secrets/variables: ### Adding a new static site -Run `./scripts/new-app.sh ` from dev machine. Handles everything: +Run `./tools/new-app.sh ` from dev machine. Handles everything: DNS, Gitea repo, SSL cert, commit/push, build trigger, and verification. See USECASES.md for full details. ### Adding a new Docker service -Run `./scripts/new-service.sh ` to scaffold the compose file and deploy +Run `./tools/new-service.sh ` to scaffold the compose file and deploy workflow, then edit the compose file and commit. ### Editing scripts @@ -85,6 +86,6 @@ No build step. No tests. Verify changes by: ## Commit style -- Short, lowercase commit messages (e.g., "add hcsuzuki.net", "fix deploy command") +- Short, lowercase commit messages (e.g., "add example.com", "fix deploy command") - Do NOT include "Co-Authored-By" lines - Commit only when explicitly asked diff --git a/README.md b/README.md index 4e26348..3faba21 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,9 @@ scripts, nginx configs, and Gitea Actions workflows for each app. docker/ nginx/ # Reverse proxy (nginx:alpine) + SSL config nginx.conf # Main nginx config - conf.d/ # Per-app server blocks (e.g. timothykim.net.conf) + conf.d/ # Per-app server blocks (e.g. example.com.conf) compose.yml - timothykim.net/ # timothykim.net static site + / # Per-app static site (e.g. example.com) compose.yml garage/ # Garage S3-compatible object storage compose.yml @@ -21,8 +21,10 @@ scripts/ bootstrap.sh # First-time server setup (manual, uses Bitwarden) configure.sh # Idempotent server config (CI-safe) deploy.sh # Deploy + cert issuance + provision (called via SSH) - new-app.sh # Scaffolding script to add a new static site - new-service.sh # Scaffolding script to add a new Docker service +tools/ + new-app.sh # Add a new static site (run from dev machine) + new-service.sh # Add a new Docker service (run from dev machine) + remove-app.sh # Remove a static site (run from dev machine) .gitea/workflows/ # Per-app deploy workflows triggered by path changes ``` @@ -59,19 +61,19 @@ bash /opt/hantim/scripts/bootstrap.sh ## Adding a new app ```bash -./scripts/new-app.sh +./tools/new-app.sh ``` -Example: `./scripts/new-app.sh hcsuzuki.net` +Example: `./tools/new-app.sh example.com` This single command handles everything: creates DNS records on Vultr, creates the Gitea repo from `static-site-template`, generates all config files, issues the SSL cert (zero downtime), commits and pushes, triggers the first build, and verifies the site is live. -**Prerequisites:** domain nameservers pointed to Vultr, `bw`/`jq`/`dig` -installed. Uses the deploy SSH key from Bitwarden -- no personal server -access needed. See USECASES.md for full details. +**Prerequisites:** domain nameservers pointed to Vultr, `bws`/`jq`/`dig` +installed. Uses the deploy SSH key from Bitwarden Secrets Manager -- no +personal server access needed. See USECASES.md for full details. After the site is live, clone the app repo and customize it: @@ -84,7 +86,7 @@ git add . && git commit -m "initial content" && git push ## Deploying changes -**App code change** (push to an app repo like `timothykim.net`): +**App code change** (push to an app repo like `example.com`): - The app's `build.yml` builds the Docker image, pushes to the registry, and SSHes into the server to trigger a deploy diff --git a/USECASES.md b/USECASES.md index 38f3051..44cc60e 100644 --- a/USECASES.md +++ b/USECASES.md @@ -41,52 +41,53 @@ fresh server. From your dev machine (single command): ```bash -./scripts/new-app.sh hcsuzuki.net +./tools/new-app.sh example.com ``` **Prerequisites:** -- Bitwarden vault must contain `hantim-new-app-script`, `hantim-vultr-api-key`, - and `hantim-server-deploy` +- Bitwarden Secrets Manager must contain `hantim-new-app-script`, + `hantim-vultr-api-key`, and `hantim-deploy-ssh-private-key` - Domain nameservers must be pointed to Vultr (configured on directnic.com) -- Dependencies: `bw`, `jq`, `dig` +- Dependencies: `bws`, `jq`, `dig` +- `BWS_ACCESS_TOKEN` env var or token saved at `~/.config/hantim/bws-token` **What happens (fully automated):** 1. Resolves server IP from `hantim.net` -2. Fetches Gitea API token, Vultr API key, and deploy SSH key from Bitwarden +2. Fetches Gitea API token, Vultr API key, and deploy SSH key from Bitwarden Secrets Manager 3. Creates DNS zone on Vultr (if needed) and A records for bare + www 4. Waits for DNS to propagate -5. Creates Gitea repo `hantim/hcsuzuki.net` from `static-site-template` +5. Creates Gitea repo `hantim/example.com` from `static-site-template` 6. Creates local files: - - `docker/hcsuzuki.net/compose.yml` (container name: `hcsuzuki_net`) - - `.gitea/workflows/deploy-hcsuzuki.net.yml` - - `docker/nginx/conf.d/hcsuzuki.net.conf` (HTTP->HTTPS, bare->www, proxy) + - `docker/example.com/compose.yml` (container name: `example_com`) + - `.gitea/workflows/deploy-example.com.yml` + - `docker/nginx/conf.d/example.com.conf` (HTTP->HTTPS, bare->www, proxy) 7. SSHes to server as deploy user, issues SSL cert via webroot (zero downtime) 8. Commits and pushes hantim-server (triggers deploy) 9. Triggers initial build of the app repo via Gitea API -10. Polls `https://www.hcsuzuki.net` until it responds (up to 3 minutes) +10. Polls `https://www.example.com` until it responds (up to 3 minutes) After the site is live, clone the app repo and customize: ```bash -git clone git@git.timothykim.net:hantim/hcsuzuki.net.git -cd hcsuzuki.net +git clone git@git.timothykim.net:hantim/example.com.git +cd example.com # edit static/index.html, etc. git add . && git commit -m "initial content" && git push ``` ## 4. Update app code -Push a change to the app repo (e.g., `timothykim.net`): +Push a change to the app repo (e.g., `example.com`): ```bash -cd timothykim.net +cd example.com # make changes git add . && git commit -m "update content" && git push ``` **What happens:** 1. Gitea Actions runs `build.yml`: builds Docker image, pushes to registry -2. SSHes into server, runs `deploy-timothykim.net` +2. SSHes into server, runs `deploy-example.com` 3. `deploy.sh` pulls latest hantim-server, pulls new image, restarts container No manual steps needed. @@ -96,14 +97,14 @@ No manual steps needed. Edit a compose file in this repo and push: ```bash -vim docker/timothykim.net/compose.yml -git add docker/timothykim.net/compose.yml -git commit -m "update timothykim.net config" +vim docker/example.com/compose.yml +git add docker/example.com/compose.yml +git commit -m "update example.com config" git push ``` **What happens:** -1. `deploy-timothykim.yml` workflow triggers (path match) +1. `deploy-example.com.yml` workflow triggers (path match) 2. `deploy.sh` pulls latest code, pulls image, runs `docker compose up -d` ## 6. Update nginx config @@ -111,8 +112,8 @@ git push Edit an nginx config in this repo and push: ```bash -vim docker/nginx/conf.d/timothykim.net.conf -git add docker/nginx/conf.d/timothykim.net.conf +vim docker/nginx/conf.d/example.com.conf +git add docker/nginx/conf.d/example.com.conf git commit -m "update nginx config" git push ``` @@ -127,9 +128,8 @@ git push 5. Runs `docker compose up -d` and reloads nginx **Note:** Each conf file must contain server blocks for only one domain. The -cert name is derived from the filename (e.g. `hcsuzuki.net.conf` -> cert -`hcsuzuki.net`). Redirect-only domains like `hcsuzukiviolin.com` get their -own conf file. +cert name is derived from the filename (e.g. `example.com.conf` -> cert +`example.com`). Redirect-only domains get their own conf file. ## 7. Re-run configure on existing server diff --git a/scripts/configure.sh b/scripts/configure.sh index 28988be..ae1be80 100755 --- a/scripts/configure.sh +++ b/scripts/configure.sh @@ -38,7 +38,7 @@ echo "==> Issuing SSL certificates..." # Stop nginx if running so certbot can bind to port 80 docker stop nginx 2>/dev/null || true for conf in "$REPO_DIR"/docker/nginx/conf.d/*.conf; do - # Derive cert name from config filename (e.g. timothykim.net.conf -> timothykim.net) + # Derive cert name from config filename (e.g. example.com.conf -> example.com) cert_name=$(basename "$conf" .conf) domains=$(grep -oP 'server_name\s+\K[^;]+' "$conf" | tr ' ' '\n' | sort -u) if [ -z "$domains" ]; then diff --git a/scripts/new-app.sh b/tools/new-app.sh similarity index 85% rename from scripts/new-app.sh rename to tools/new-app.sh index f96d9c6..d09c760 100755 --- a/scripts/new-app.sh +++ b/tools/new-app.sh @@ -17,10 +17,10 @@ spin_wait() { } if [ -z "${1:-}" ]; then - echo "Usage: ./scripts/new-app.sh " - echo " Example: ./scripts/new-app.sh hcsuzuki.net" + echo "Usage: ./tools/new-app.sh " + echo " Example: ./tools/new-app.sh example.com" echo "" - echo "Requires: bw, jq, dig" + echo "Requires: bws, jq, dig" exit 1 fi @@ -47,9 +47,9 @@ fi # --- Check dependencies --- -if ! command -v bw &>/dev/null; then - echo "Error: Bitwarden CLI (bw) is not installed." - echo " npm install -g @bitwarden/cli" +if ! command -v bws &>/dev/null; then + echo "Error: Bitwarden Secrets Manager CLI (bws) is not installed." + echo " https://github.com/bitwarden/sdk-sm/releases" exit 1 fi @@ -65,41 +65,46 @@ if ! command -v dig &>/dev/null; then exit 1 fi -# --- Get secrets (cached in file, or fetch from Bitwarden) --- +# --- Get bws access token --- -SECRETS_CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/hantim-secrets" +BWS_TOKEN_FILE="${XDG_CONFIG_HOME:-$HOME/.config}/hantim/bws-token" -if [ -f "$SECRETS_CACHE" ]; then - echo "Using cached secrets." - # shellcheck source=/dev/null - source "$SECRETS_CACHE" - GITEA_TOKEN="$HANTIM_GITEA_TOKEN" - VULTR_API_KEY="$HANTIM_VULTR_API_KEY" - DEPLOY_KEY="$HANTIM_DEPLOY_KEY" -else - if bw status 2>/dev/null | grep -q '"status":"unauthenticated"'; then - echo "Log in to Bitwarden:" - bw login +if [ -z "${BWS_ACCESS_TOKEN:-}" ]; then + if [ -f "$BWS_TOKEN_FILE" ]; then + BWS_ACCESS_TOKEN=$(cat "$BWS_TOKEN_FILE") + else + echo "==> Bitwarden Secrets Manager access token required." + echo " Generate one for your machine account at:" + echo " https://vault.bitwarden.com/#/sm/machine-accounts" + read -rp " Paste access token: " BWS_ACCESS_TOKEN + if [ -z "$BWS_ACCESS_TOKEN" ]; then + echo "ERROR: Access token cannot be empty." + exit 1 + fi + mkdir -p "$(dirname "$BWS_TOKEN_FILE")" + echo "$BWS_ACCESS_TOKEN" > "$BWS_TOKEN_FILE" + chmod 600 "$BWS_TOKEN_FILE" fi - - if bw status 2>/dev/null | grep -q '"status":"locked"'; then - BW_SESSION=$(bw unlock --raw) - export BW_SESSION - fi - - bw sync --session "${BW_SESSION:-}" - GITEA_TOKEN=$(bw get notes hantim-new-app-script --session "${BW_SESSION:-}") - VULTR_API_KEY=$(bw get notes hantim-vultr-api-key --session "${BW_SESSION:-}") - DEPLOY_KEY=$(bw get item hantim-server-deploy --session "${BW_SESSION:-}" | jq -r '.sshKey.privateKey') - - mkdir -p "$(dirname "$SECRETS_CACHE")" - cat > "$SECRETS_CACHE" </dev/null | jq -r --arg name "$name" '.[] | select(.key == $name) | .value') + if [ -z "$value" ]; then + echo "ERROR: Secret '$name' not found in Bitwarden Secrets Manager." >&2 + exit 1 + fi + echo "$value" +} + +echo "==> Fetching secrets from Bitwarden Secrets Manager..." +GITEA_TOKEN=$(bws_get "hantim-new-app-script") +VULTR_API_KEY=$(bws_get "hantim-vultr-api-key") +DEPLOY_KEY=$(bws_get "hantim-deploy-ssh-private-key") DEPLOY_KEY_FILE=$(mktemp) echo "$DEPLOY_KEY" > "$DEPLOY_KEY_FILE" diff --git a/scripts/new-service.sh b/tools/new-service.sh similarity index 93% rename from scripts/new-service.sh rename to tools/new-service.sh index 1521356..3771aac 100755 --- a/scripts/new-service.sh +++ b/tools/new-service.sh @@ -2,8 +2,8 @@ set -euo pipefail if [ -z "${1:-}" ]; then - echo "Usage: ./scripts/new-service.sh " - echo " Example: ./scripts/new-service.sh garage" + echo "Usage: ./tools/new-service.sh " + echo " Example: ./tools/new-service.sh garage" echo "" echo "Creates docker//compose.yml and .gitea/workflows/deploy-.yml" exit 1 diff --git a/scripts/remove-app.sh b/tools/remove-app.sh similarity index 51% rename from scripts/remove-app.sh rename to tools/remove-app.sh index de9d43e..70992b1 100755 --- a/scripts/remove-app.sh +++ b/tools/remove-app.sh @@ -5,8 +5,8 @@ GITEA_URL="https://git.timothykim.net" GITEA_ORG="hantim" if [ -z "${1:-}" ]; then - echo "Usage: ./scripts/remove-app.sh " - echo " Example: ./scripts/remove-app.sh hcsuzuki.net" + echo "Usage: ./tools/remove-app.sh " + echo " Example: ./tools/remove-app.sh example.com" echo "" echo "Removes: local files (compose, workflow, nginx conf), Gitea repo." echo "Does NOT remove: DNS records, SSL certs." @@ -23,33 +23,42 @@ fi # --- Check dependencies --- -if ! command -v bw &>/dev/null; then - echo "Error: Bitwarden CLI (bw) is not installed." +if ! command -v bws &>/dev/null; then + echo "Error: Bitwarden Secrets Manager CLI (bws) is not installed." + echo " https://github.com/bitwarden/sdk-sm/releases" exit 1 fi -# --- Get secrets (cached in file, or fetch from Bitwarden) --- +# --- Get bws access token --- -SECRETS_CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/hantim-secrets" +BWS_TOKEN_FILE="${XDG_CONFIG_HOME:-$HOME/.config}/hantim/bws-token" -if [ -f "$SECRETS_CACHE" ]; then - echo "Using cached secrets." - # shellcheck source=/dev/null - source "$SECRETS_CACHE" - GITEA_TOKEN="$HANTIM_GITEA_TOKEN" -else - if bw status 2>/dev/null | grep -q '"status":"unauthenticated"'; then - echo "Log in to Bitwarden:" - bw login +if [ -z "${BWS_ACCESS_TOKEN:-}" ]; then + if [ -f "$BWS_TOKEN_FILE" ]; then + BWS_ACCESS_TOKEN=$(cat "$BWS_TOKEN_FILE") + else + echo "==> Bitwarden Secrets Manager access token required." + echo " Generate one for your machine account at:" + echo " https://vault.bitwarden.com/#/sm/machine-accounts" + read -rp " Paste access token: " BWS_ACCESS_TOKEN + if [ -z "$BWS_ACCESS_TOKEN" ]; then + echo "ERROR: Access token cannot be empty." + exit 1 + fi + mkdir -p "$(dirname "$BWS_TOKEN_FILE")" + echo "$BWS_ACCESS_TOKEN" > "$BWS_TOKEN_FILE" + chmod 600 "$BWS_TOKEN_FILE" fi +fi +export BWS_ACCESS_TOKEN - if bw status 2>/dev/null | grep -q '"status":"locked"'; then - BW_SESSION=$(bw unlock --raw) - export BW_SESSION - fi +# --- Fetch secrets from Bitwarden Secrets Manager --- - bw sync --session "${BW_SESSION:-}" - GITEA_TOKEN=$(bw get notes hantim-new-app-script --session "${BW_SESSION:-}") +echo "==> Fetching secrets from Bitwarden Secrets Manager..." +GITEA_TOKEN=$(bws secret list 2>/dev/null | jq -r '.[] | select(.key == "hantim-new-app-script") | .value') +if [ -z "$GITEA_TOKEN" ]; then + echo "ERROR: Secret 'hantim-new-app-script' not found in Bitwarden Secrets Manager." + exit 1 fi # --- Delete Gitea repo ---