diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 4c9a94c..c5a7999 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -29,6 +29,7 @@ Bitwarden free tier is used as the secret store. Stores: - `hantim-ci-registry-push` -- Gitea API token for registry access (Secure Note) - `hantim-server-deploy` -- deploy user SSH key pair (SSH Key) - `hantim-new-app-script` -- Gitea API token for creating repos (Secure Note) +- `hantim-vultr-api-key` -- Vultr API key for DNS management (Secure Note) The Bitwarden CLI (`bw`) is used in `setup.sh` (on the server) and `new-app.sh` (on dev machines) to fetch secrets at runtime. @@ -45,7 +46,7 @@ configs, deploy scripts, and Gitea Actions workflows. ``` setup.sh # Server provisioning (idempotent) scripts/deploy.sh # Receives deploy commands via SSH -scripts/new-app.sh # Scaffolds a new app (Gitea repo + local files) +scripts/new-app.sh # Scaffolds + deploys a new app (single command) docker/nginx/ # nginx reverse proxy compose.yml # nginx:alpine container nginx.conf # Main nginx config diff --git a/CLAUDE.md b/CLAUDE.md index 0faf761..e29ed87 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,7 +11,7 @@ Actions workflows. - `setup.sh` -- Server provisioning (idempotent, run as root) - `scripts/deploy.sh` -- SSH-triggered deploy (validates command, pulls, restarts) -- `scripts/new-app.sh` -- Scaffolds a new static site (Gitea repo + compose + workflow + nginx conf) +- `scripts/new-app.sh` -- Single-command new app (DNS + Gitea repo + cert + deploy + verify) - `docker/nginx/` -- Reverse proxy (nginx:alpine + certbot on host) - `docker//compose.yml` -- Per-app compose files - `.gitea/workflows/deploy-.yml` -- Per-app deploy workflows @@ -32,6 +32,7 @@ Stored in Bitwarden (free tier), fetched via `bw` CLI: - `hantim-ci-registry-push` -- Secure Note (Gitea registry token) - `hantim-server-deploy` -- SSH Key (deploy user key pair) - `hantim-new-app-script` -- Secure Note (Gitea API token for creating repos) +- `hantim-vultr-api-key` -- Secure Note (Vultr API key for DNS management) Gitea org-level (`hantim`) secrets/variables: - `DEPLOY_HOST` -- Variable (server IP) @@ -42,8 +43,9 @@ Gitea org-level (`hantim`) secrets/variables: ### Adding a new app -Run `./scripts/new-app.sh ` from dev machine, then issue SSL cert on -server **before** pushing. See USECASES.md for full steps. +Run `./scripts/new-app.sh ` from dev machine. Handles everything: +DNS, Gitea repo, SSL cert, commit/push, build trigger, and verification. +See USECASES.md for full details. ### Editing scripts diff --git a/README.md b/README.md index 1da162f..abbc92e 100644 --- a/README.md +++ b/README.md @@ -53,26 +53,22 @@ before running setup. Example: `./scripts/new-app.sh hcsuzuki.net` -This creates: +This single command handles everything: creates DNS records on Vultr, creates +the Gitea repo from `static-site-template`, generates all config files, issues +the SSL cert (zero downtime), commits and pushes, triggers the first build, +and verifies the site is live. -- A Gitea repo (`hantim/`) from the `static-site-template` -- `docker//compose.yml` -- app compose file -- `.gitea/workflows/deploy-.yml` -- deploy workflow -- `docker/nginx/conf.d/.conf` -- nginx proxy config (HTTP->HTTPS, - bare domain->www) +**Prerequisites:** domain nameservers pointed to Vultr, `bw`/`jq`/`dig` +installed, SSH access to the server. See USECASES.md for full details. -After running the script: +After the site is live, clone the app repo and customize it: -1. Issue the SSL cert on the server (**before** pushing): - ```bash - docker stop nginx - certbot certonly --standalone --non-interactive --agree-tos \ - --register-unsafely-without-email --cert-name \ - -d -d www. - docker start nginx - ``` -2. Commit and push this repo to deploy -3. Clone the new app repo, customize it, and push to build/deploy +```bash +git clone git@git.timothykim.net:hantim/.git +cd +# edit content +git add . && git commit -m "initial content" && git push +``` ## Deploying changes @@ -110,8 +106,9 @@ The following are configured in the `hantim` Gitea org settings: | `DEPLOY_SSH_KEY` | Secret | SSH private key for the deploy user | | `CI_REGISTRY_TOKEN` | Secret | Gitea token for Docker registry login | -The following is stored in Bitwarden and used by `new-app.sh`: +The following are stored in Bitwarden and used by `new-app.sh`: | Bitwarden item | Type | Purpose | |---|---|---| | `hantim-new-app-script` | Secure Note | Gitea API token for creating repos | +| `hantim-vultr-api-key` | Secure Note | Vultr API key for DNS management | diff --git a/USECASES.md b/USECASES.md index f00152b..e8014b8 100644 --- a/USECASES.md +++ b/USECASES.md @@ -36,31 +36,35 @@ fresh server. ## 3. Add a new static site -From your dev machine: +From your dev machine (single command): ```bash ./scripts/new-app.sh hcsuzuki.net ``` -Then on the server, issue the SSL cert **before** pushing: +**Prerequisites:** +- Bitwarden vault must contain `hantim-new-app-script` and `hantim-vultr-api-key` +- Domain nameservers must be pointed to Vultr (configured on directnic.com) +- SSH access to the server as your user with passwordless sudo +- Dependencies: `bw`, `jq`, `dig` -```bash -docker stop nginx -certbot certonly --standalone --non-interactive --agree-tos \ - --register-unsafely-without-email --cert-name hcsuzuki.net \ - -d hcsuzuki.net -d www.hcsuzuki.net -docker start nginx -``` +**What happens (fully automated):** +1. Resolves server IP from `hantim.net` +2. Fetches Gitea API token and Vultr API key from Bitwarden +3. Creates DNS zone on Vultr (if needed) and A records for bare + www +4. Waits for DNS to propagate +5. Creates Gitea repo `hantim/hcsuzuki.net` from `static-site-template` +6. Creates local files: + - `docker/hcsuzuki.net/compose.yml` (container name: `hcsuzuki_net`) + - `.gitea/workflows/deploy-hcsuzuki.net.yml` + - `docker/nginx/conf.d/hcsuzuki.net.conf` (HTTP->HTTPS, bare->www, proxy) +7. SSHes to server: writes temporary HTTP-only nginx config, issues SSL cert + via webroot (zero downtime) +8. Commits and pushes hantim-server (triggers deploy) +9. Triggers initial build of the app repo via Gitea API +10. Polls `https://www.hcsuzuki.net` until it responds (up to 3 minutes) -Then commit and push this repo: - -```bash -git add docker/hcsuzuki.net/ .gitea/workflows/deploy-hcsuzuki.net.yml docker/nginx/conf.d/hcsuzuki.net.conf -git commit -m "add hcsuzuki.net" -git push -``` - -Finally, clone the new app repo, customize it, and push: +After the site is live, clone the app repo and customize: ```bash git clone git@git.timothykim.net:hantim/hcsuzuki.net.git @@ -69,12 +73,6 @@ cd hcsuzuki.net git add . && git commit -m "initial content" && git push ``` -**What the script creates:** -- Gitea repo `hantim/hcsuzuki.net` (from `static-site-template`) -- `docker/hcsuzuki.net/compose.yml` (container name: `hcsuzuki_net`) -- `.gitea/workflows/deploy-hcsuzuki.net.yml` -- `docker/nginx/conf.d/hcsuzuki.net.conf` (HTTP->HTTPS, bare->www, proxy) - ## 4. Update app code Push a change to the app repo (e.g., `timothykim.net`): diff --git a/scripts/new-app.sh b/scripts/new-app.sh index 8abab14..ef7da91 100755 --- a/scripts/new-app.sh +++ b/scripts/new-app.sh @@ -7,12 +7,23 @@ TEMPLATE_REPO="static-site-template" if [ -z "${1:-}" ]; then echo "Usage: ./scripts/new-app.sh " - echo "Example: ./scripts/new-app.sh hcsuzuki.net" + echo " Example: ./scripts/new-app.sh hcsuzuki.net" + echo "" + echo "Requires environment variables:" + echo " DEPLOY_USER - SSH username (default: \$USER)" exit 1 fi APP="$1" CONTAINER_NAME="${APP//./_}" +SERVER_IP=$(dig +short hantim.net | head -1) + +if [ -z "$SERVER_IP" ]; then + echo "Error: Could not resolve hantim.net to get server IP." + exit 1 +fi + +SSH_TARGET="${DEPLOY_USER:-$USER}@$SERVER_IP" if ! [[ "$APP" =~ ^[a-zA-Z0-9][a-zA-Z0-9._-]*$ ]]; then echo "Error: app name must be alphanumeric (hyphens, dots, underscores allowed)." @@ -39,6 +50,13 @@ if ! command -v jq &>/dev/null; then exit 1 fi +if ! command -v dig &>/dev/null; then + echo "Error: dig is not installed." + echo " On macOS: brew install bind" + echo " On Linux: dnf install bind-utils" + exit 1 +fi + # --- Get Gitea API token from Bitwarden --- if bw status 2>/dev/null | grep -q '"status":"unauthenticated"'; then @@ -53,6 +71,73 @@ fi bw sync --session "${BW_SESSION:-}" GITEA_TOKEN=$(bw get notes hantim-new-app-script --session "${BW_SESSION:-}") +VULTR_API_KEY=$(bw get notes hantim-vultr-api-key --session "${BW_SESSION:-}") + +# --- Set up DNS records on Vultr --- + +VULTR_API="https://api.vultr.com/v2" +VULTR_AUTH="Authorization: Bearer $VULTR_API_KEY" + +echo "==> Checking DNS zone for $APP on Vultr..." +ZONE_CHECK=$(curl -s -o /dev/null -w "%{http_code}" \ + "$VULTR_API/domains/$APP" \ + -H "$VULTR_AUTH") + +if [ "$ZONE_CHECK" != "200" ]; then + echo " Creating DNS zone for $APP..." + HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" \ + -X POST "$VULTR_API/domains" \ + -H "$VULTR_AUTH" \ + -H "Content-Type: application/json" \ + -d "{\"domain\": \"$APP\"}") + if [ "$HTTP_CODE" != "200" ] && [ "$HTTP_CODE" != "201" ]; then + echo "Error: Failed to create DNS zone (HTTP $HTTP_CODE)" + exit 1 + fi + echo " Created DNS zone for $APP" +fi + +echo "==> Checking A records for $APP..." +RECORDS=$(curl -s "$VULTR_API/domains/$APP/records" -H "$VULTR_AUTH") + +# Check/create root A record +ROOT_EXISTS=$(echo "$RECORDS" | jq -r ".records[] | select(.type == \"A\" and .name == \"\" and .data == \"$SERVER_IP\") | .id") +if [ -z "$ROOT_EXISTS" ]; then + echo " Creating A record: $APP -> $SERVER_IP" + curl -sf -X POST "$VULTR_API/domains/$APP/records" \ + -H "$VULTR_AUTH" \ + -H "Content-Type: application/json" \ + -d "{\"name\": \"\", \"type\": \"A\", \"data\": \"$SERVER_IP\", \"ttl\": 3600}" > /dev/null +else + echo " A record for $APP already exists." +fi + +# Check/create www A record +WWW_EXISTS=$(echo "$RECORDS" | jq -r ".records[] | select(.type == \"A\" and .name == \"www\" and .data == \"$SERVER_IP\") | .id") +if [ -z "$WWW_EXISTS" ]; then + echo " Creating A record: www.$APP -> $SERVER_IP" + curl -sf -X POST "$VULTR_API/domains/$APP/records" \ + -H "$VULTR_AUTH" \ + -H "Content-Type: application/json" \ + -d "{\"name\": \"www\", \"type\": \"A\", \"data\": \"$SERVER_IP\", \"ttl\": 3600}" > /dev/null +else + echo " A record for www.$APP already exists." +fi + +echo "==> Waiting for DNS to propagate..." +for i in $(seq 1 30); do + RESOLVED=$(dig +short "$APP" @ns1.vultr.com 2>/dev/null) + if [ "$RESOLVED" = "$SERVER_IP" ]; then + echo " DNS is live." + break + fi + if [ "$i" = "30" ]; then + echo "Error: DNS for $APP did not resolve to $SERVER_IP after 5 minutes." + echo " If this is a new domain, make sure nameservers are set to Vultr on directnic.com." + exit 1 + fi + sleep 10 +done # --- Create Gitea repo from template --- @@ -172,18 +257,57 @@ server { } NGINX -echo "" -echo "Done! Files created:" -echo " - Gitea repo: $GITEA_URL/$GITEA_ORG/$APP" -echo " - docker/$APP/compose.yml" -echo " - .gitea/workflows/deploy-$APP.yml" -echo " - docker/nginx/conf.d/$APP.conf" -echo "" -echo "Next steps:" -echo " 1. Issue SSL cert on the server (BEFORE pushing nginx config):" -echo " docker stop nginx" -echo " certbot certonly --standalone --non-interactive --agree-tos --register-unsafely-without-email --cert-name $APP -d $APP -d www.$APP" -echo " docker start nginx" -echo " 2. Commit and push hantim-server to deploy" -echo " 3. Clone $GITEA_URL/$GITEA_ORG/$APP and customize it" -echo " 4. Edit docker/$APP/compose.yml if needed" +# --- Issue SSL cert (zero downtime) --- + +echo "==> Writing temporary HTTP-only config on server..." +TEMP_CONF="server { + listen 80; + listen [::]:80; + server_name $APP www.$APP; + + location /.well-known/acme-challenge/ { + root /var/www/certbot; + } + + location / { + return 444; + } +}" +echo "$TEMP_CONF" | ssh "$SSH_TARGET" "sudo tee /opt/hantim/docker/nginx/conf.d/$APP.conf > /dev/null" +ssh "$SSH_TARGET" "sudo docker exec nginx nginx -t && sudo docker exec nginx nginx -s reload" + +echo "==> Issuing SSL certificate..." +ssh "$SSH_TARGET" "sudo certbot certonly --webroot -w /opt/hantim/docker/nginx/certbot/www --non-interactive --agree-tos --register-unsafely-without-email --cert-name $APP -d $APP -d www.$APP" + +# --- Commit and push --- + +echo "==> Committing and pushing hantim-server..." +cd "$REPO_ROOT" +git add "docker/$APP/compose.yml" ".gitea/workflows/deploy-$APP.yml" "docker/nginx/conf.d/$APP.conf" +git commit -m "add $APP" +git push + +# --- Trigger initial build --- + +echo "==> Triggering initial build for $APP..." +curl -sf -X POST "$GITEA_URL/api/v1/repos/$GITEA_ORG/$APP/contents/.deploy-trigger" \ + -H "Authorization: token $GITEA_TOKEN" \ + -H "Content-Type: application/json" \ + -d "{\"content\": \"$(echo -n "initial deploy" | base64)\", \"message\": \"trigger initial build\"}" > /dev/null + +# --- Verify --- + +echo "==> Waiting for deployment (up to 3 minutes)..." +for i in $(seq 1 18); do + if curl -sf "https://www.$APP" > /dev/null 2>&1; then + echo "Site is live at https://www.$APP" + exit 0 + fi + sleep 10 +done + +echo "WARNING: https://www.$APP is not responding after 3 minutes." +echo " Check that:" +echo " - DNS for $APP and www.$APP points to $SERVER_IP" +echo " - The build completed: $GITEA_URL/$GITEA_ORG/$APP/actions" +echo " - The container is running: ssh $SSH_TARGET sudo docker ps"