self-heal standalone cert renewal configs on provision

- reorder app.sh to run cert before commit/push (fixes workflow failure on first run)
- quote $SSH_ORIGINAL_COMMAND in authorized_keys to prevent command injection
- add port validation in service.sh nginx command
- pin host key in deploy-beszel.yml workflow
- fix append vs overwrite for known_hosts in deploy-nginx.yml

.gitea/workflows/deploy-beszel.yml

@@ -13,6 +13,7 @@ jobs:
       - name: Deploy via SSH
         run: |
           mkdir -p ~/.ssh
+          echo "${{ vars.DEPLOY_HOST_KEY }}" > ~/.ssh/known_hosts
           echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key
           chmod 600 ~/.ssh/deploy_key
-          ssh -o StrictHostKeyChecking=accept-new -i ~/.ssh/deploy_key deploy@${{ vars.DEPLOY_HOST }} deploy-beszel
+          ssh -o StrictHostKeyChecking=yes -i ~/.ssh/deploy_key deploy@${{ vars.DEPLOY_HOST }} deploy-beszel

.gitea/workflows/deploy-carolpreschool.com.yml

@@ -0,0 +1,19 @@
+name: Deploy carolpreschool.com
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'docker/carolpreschool.com/**'
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Deploy via SSH
+        run: |
+          mkdir -p ~/.ssh
+          echo "${{ vars.DEPLOY_HOST_KEY }}" > ~/.ssh/known_hosts
+          echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key
+          chmod 600 ~/.ssh/deploy_key
+          ssh -o StrictHostKeyChecking=yes -i ~/.ssh/deploy_key deploy@${{ vars.DEPLOY_HOST }} deploy-carolpreschool.com

.gitea/workflows/deploy-nginx.yml

@@ -15,8 +15,7 @@ jobs:
       - name: Deploy via SSH
         run: |
           mkdir -p ~/.ssh
-          echo "${{ vars.DEPLOY_HOST_KEY }}" >> ~/.ssh/known_hosts
+          echo "${{ vars.DEPLOY_HOST_KEY }}" > ~/.ssh/known_hosts
-          chmod 644 ~/.ssh/known_hosts
           echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key
           chmod 600 ~/.ssh/deploy_key
           ssh -o StrictHostKeyChecking=yes -i ~/.ssh/deploy_key deploy@${{ vars.DEPLOY_HOST }} deploy-nginx

ARCHITECTURE.md

@@ -105,7 +105,12 @@ cert as SANs.
 1. Run `app.sh cert <domain>` (uses `certbot --webroot`, zero downtime)
 2. Deploy nginx config — `deploy.sh` verifies certs exist, fails if missing
 
-**Renewal:** certbot timer/cron runs daily, deploy hook reloads nginx.
+**Renewal:** certbot timer/cron runs daily, deploy hook reloads nginx. All certs
+must use the `webroot` authenticator so nginx can keep port 80; certs issued by
+the old `--standalone` flow (removed in `de1d60b`) can never renew because nginx
+holds port 80. `configure.sh` self-heals this on every `provision`: it scans
+`/etc/letsencrypt/renewal/*.conf` and reissues any `standalone` cert via the
+`cert-<domain>` webroot path (best-effort; warns and exits non-zero on failure).
 
 ## `.env.keys` mechanism
 
@@ -133,6 +138,19 @@ If any secret is missing, deploy fails with an error (not a warning).
 - **One domain per conf file** — `deploy.sh` derives the cert name from the
   filename and collects all `server_name` values for that cert
 
+## GoatCounter (analytics)
+
+GoatCounter runs at `goatcounter.hantim.net` for privacy-friendly web analytics.
+
+**Script injection:** nginx injects the GoatCounter tracking script into all
+HTML responses via `sub_filter`. The shared include `conf.d/goatcounter.inc`
+is added to each site's main HTTPS server block. New sites get it automatically
+via `app.sh`.
+
+**Config:** `docker/goatcounter/compose.yaml` uses the `baethon/goatcounter`
+image. Password is stored in Bitwarden (`hantim-goatcounter-password`) and
+injected via the `.env.keys` mechanism.
+
 ## URL routing
 
 - All HTTP → HTTPS redirect

CLAUDE.md

@@ -8,11 +8,12 @@ for technical deep-dives, USECASES.md for expected behaviors.
 - `scripts/bootstrap.sh` -- First-time server setup (manual, uses Bitwarden)
 - `scripts/configure.sh` -- Idempotent server config (firewall, certbot, start services; CI-safe)
 - `scripts/deploy.sh` -- SSH-triggered deploy + cert issuance + provision (deploy-*, cert-*, provision commands)
-- `tools/app.sh` -- App provisioning by subcommand (dns, repo, files, cert, garage, build, verify, all)
+- `tools/app.sh` -- App provisioning by subcommand (dns, repo, files, cert, garage, build, verify, monitor, all)
 - `tools/service.sh` -- Manage Docker services by subcommand (dns, files, nginx, cert, all)
 - `tools/remove-app.sh` -- Remove a static site (local files + Gitea repo)
 - `docker/nginx/conf.d/` -- Per-app nginx server blocks
 - `docker/<domain>/compose.yml` -- Per-app compose files
+- `docker/goatcounter/` -- GoatCounter analytics (compose.yaml, .env.keys)
 - `docker/garage/` -- Garage S3 object storage (Dockerfile, garage.toml, .env.keys)
 - `.gitea/workflows/` -- Per-app deploy workflows + provision.yml
 

README.md

@@ -38,6 +38,7 @@ docker/
     compose.yml
   <domain>/            # Per-app compose files (e.g. example.com)
     compose.yml
+  goatcounter/         # GoatCounter analytics
   garage/              # Garage S3-compatible object storage
     compose.yml
     Dockerfile
@@ -163,19 +164,55 @@ reconnects automatically.
 
 ## Testing
 
-Tests live in `tests/` and run with pytest:
+Tests live in `tests/` and run with [uv](https://docs.astral.sh/uv/):
 
 ```bash
-pip install pytest
+# Install uv: curl -LsSf https://astral.sh/uv/install.sh | sh
-pytest tests/ -v
+
+# Live site tests (security headers, TLS, default server)
+uvx pytest tests/ -v --ignore=tests/test_app_e2e.py
+
+# E2E tests for tools/app.sh (creates real resources)
+uvx pytest tests/test_app_e2e.py -v -x
 ```
 
-Tests auto-discover sites from `docker/nginx/conf.d/*.conf` — no list to
+### Live site tests
-maintain when adding new sites.
 
-The `deploy-nginx.yml` workflow runs tests automatically after each nginx
+These run against live sites and verify infrastructure invariants:
+
+- **`test_security_headers.py`** — Checks that all sites including
+  `security-headers.inc` return the expected headers (HSTS, X-Frame-Options,
+  etc.). Sites are auto-discovered from `docker/nginx/conf.d/*.conf`.
+- **`test_tls.py`** — Verifies TLS 1.3 and 1.2 work, TLS 1.1 is rejected.
+- **`test_default_server.py`** — Verifies unknown Host headers get dropped
+  (connection reset) on both HTTP and HTTPS.
+
+The `deploy-nginx.yml` workflow runs these automatically after each nginx
 deploy. If tests fail, Gitea sends an email notification.
 
+### E2E tests
+
+`test_app_e2e.py` tests `tools/app.sh` against real services (Vultr, Gitea,
+Garage, UptimeRobot). Uses `kgfamily.com` as a throwaway test domain.
+
+**Prerequisites:**
+- CLI tools: `bws`, `jq`, `dig`, `ssh`, `curl`, `git`
+- `BWS_ACCESS_TOKEN` env var or token at `~/.config/hantim/bws-token`
+- Vultr API key must allow requests from the machine's IP
+  (check https://my.vultr.com/settings/#settingsapi)
+
+**Structure:**
+
+1. **TestAppSubcommands** — Runs each `app.sh` subcommand individually
+   (dns, repo, files, cert, garage, build, monitor), verifies side effects
+   via API, then cleans up.
+2. **TestAppAll** — Runs `app.sh all kgfamily.com`, verifies the full flow
+   including site liveness and git commit, then cleans up.
+
+Cleanup runs before and after each phase via APIs (Vultr, Gitea, Garage,
+UptimeRobot) + local file deletion. Does not use `remove-app.sh` to avoid
+coupling.
+
 ## Secrets
 
 ### Bitwarden Secrets Manager
@@ -195,6 +232,7 @@ Project: `hantim`. Fetched via `bws` CLI on the server.
 | `hantim-garage-media-key-id` | S3 access key ID for media uploads | `app.sh`, `aws` CLI |
 | `hantim-garage-media-secret-key` | S3 secret key for media uploads | `aws` CLI |
 | `hantim-uptimerobot-api-key` | UptimeRobot API key for monitors | `app.sh` |
+| `hantim-goatcounter-password` | GoatCounter admin password | `deploy.sh` |
 
 Machine accounts: `hantim-server` (token at `/etc/bws-token`), `hantim-ci` (reserved).
 

USECASES.md

@@ -102,7 +102,17 @@ for stateless static sites.
 - SSL cert issued for `<name>.hantim.net` (no www variant)
 - Service ready for customization (edit compose.yml image, add `.env.keys` if needed)
 
-## 9. Upload media files
+## 9. Analytics tracking
+
+**Trigger:** automatic — nginx injects the GoatCounter script into all HTML responses.
+
+**Expected outcome:**
+- Every site includes the GoatCounter tracking script (injected via `sub_filter`)
+- Page views are recorded at `goatcounter.hantim.net`
+- New sites provisioned with `app.sh` include tracking automatically
+- GoatCounter dashboard itself is not tracked
+
+## 10. Upload media files
 
 **Trigger:** `aws s3 cp` or `aws s3 sync` to `s3://example.com/`, or `./media.sh push` from a site repo.
 
@@ -111,7 +121,7 @@ for stateless static sites.
 - Replicated to both Garage nodes (hantim + argento)
 - Accessible at `https://www.example.com/media/<path>`
 
-## 10. Local development with media files
+## 11. Local development with media files
 
 **Trigger:** `./media.sh pull` from a site repo.
 

docker/beszel/.gitignore

@@ -2,4 +2,6 @@
 beszel_data/
 # !create
 beszel_socket/
+# !create
+beszel_agent_data/
 

docker/carolpreschool.com/compose.yml

@@ -0,0 +1,11 @@
+services:
+  app:
+    image: git.timothykim.net/hantim/carolpreschool.com:latest
+    restart: unless-stopped
+    container_name: carolpreschool_com
+    networks:
+      - shared
+
+networks:
+  shared:
+    external: true

docker/goatcounter/.env.keys

@@ -0,0 +1,2 @@
+# hantim-goatcounter-password
+GOATCOUNTER_PASSWORD=1b02fe2e-7acb-4111-918a-b42701546299

docker/goatcounter/.gitignore

@@ -1,2 +1,3 @@
 # !create
 db/
+.env

docker/goatcounter/compose.yaml

@@ -3,11 +3,10 @@ services:
     image: baethon/goatcounter
     restart: unless-stopped
     container_name: goatcounter
-    command: >-
+    env_file: .env
-      goatcounter serve
+    environment:
-      -listen :8080
+      GOATCOUNTER_DOMAIN: goatcounter.hantim.net
-      -tls none
+      GOATCOUNTER_EMAIL: timothykim@fastmail.fm
-      -db sqlite3+/goatcounter/db/goatcounter.sqlite3
     volumes:
       - ./db:/goatcounter/db
     networks:

docker/nginx/conf.d/beszel.hantim.net.conf

@@ -28,5 +28,6 @@ server {
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_hide_header X-Frame-Options;
     }
 }

docker/nginx/conf.d/carolpreschool.com.conf

@@ -0,0 +1,54 @@
+# Redirect HTTP to HTTPS, bare domain to www
+server {
+    listen 80;
+    listen [::]:80;
+    server_name carolpreschool.com www.carolpreschool.com;
+
+    location /.well-known/acme-challenge/ {
+        root /var/www/certbot;
+    }
+
+    location / {
+        return 301 https://www.carolpreschool.com$request_uri;
+    }
+}
+
+# Redirect bare HTTPS domain to www
+server {
+    listen 443 ssl;
+    listen [::]:443 ssl;
+    http2 on;
+    server_name carolpreschool.com;
+
+    ssl_certificate     /etc/letsencrypt/live/carolpreschool.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/carolpreschool.com/privkey.pem;
+
+    return 301 https://www.carolpreschool.com$request_uri;
+}
+
+# Main site
+server {
+    listen 443 ssl;
+    listen [::]:443 ssl;
+    http2 on;
+    server_name www.carolpreschool.com;
+
+    ssl_certificate     /etc/letsencrypt/live/carolpreschool.com/fullchain.pem;
+    ssl_certificate_key /etc/letsencrypt/live/carolpreschool.com/privkey.pem;
+
+    include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
+
+    location /media/ {
+        proxy_pass http://garage:3902/;
+        proxy_set_header Host carolpreschool.com.web.garage;
+    }
+
+    location / {
+        proxy_pass http://carolpreschool_com:80;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+    }
+}

docker/nginx/conf.d/garage.hantim.net.conf

@@ -31,5 +31,6 @@ server {
         deny all;
 
         proxy_pass http://garage:3903;
+        proxy_hide_header X-Frame-Options;
     }
 }

docker/nginx/conf.d/goatcounter.hantim.net.conf

@@ -28,5 +28,6 @@ server {
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_hide_header X-Frame-Options;
     }
 }

docker/nginx/conf.d/goatcounter.inc

@@ -0,0 +1,2 @@
+sub_filter '</body>' '<script>window.goatcounter={path:function(p){return location.host+p}}</script><script data-goatcounter="https://goatcounter.hantim.net/count" async src="//goatcounter.hantim.net/count.js"></script></body>';
+sub_filter_once on;

docker/nginx/conf.d/haanmind.net.conf

@@ -37,6 +37,7 @@ server {
     ssl_certificate_key /etc/letsencrypt/live/haanmind.net/privkey.pem;
 
     include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
 
     location /media/ {
         proxy_pass http://garage:3902/;

docker/nginx/conf.d/hantim.net.conf

@@ -37,6 +37,7 @@ server {
     ssl_certificate_key /etc/letsencrypt/live/hantim.net/privkey.pem;
 
     include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
 
     location /media/ {
         proxy_pass http://garage:3902/;

docker/nginx/conf.d/hcsuzuki.music.conf

@@ -37,6 +37,7 @@ server {
     ssl_certificate_key /etc/letsencrypt/live/hcsuzuki.music/privkey.pem;
 
     include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
 
     location /media/ {
         proxy_pass http://garage:3902/;

docker/nginx/conf.d/hcsuzuki.net.conf

@@ -37,6 +37,7 @@ server {
     ssl_certificate_key /etc/letsencrypt/live/hcsuzuki.net/privkey.pem;
 
     include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
 
     location /media/ {
         proxy_pass http://garage:3902/;

docker/nginx/conf.d/s3.hantim.net.conf

@@ -31,5 +31,6 @@ server {
         proxy_set_header Host $host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_hide_header X-Frame-Options;
     }
 }

docker/nginx/conf.d/thekims.family.conf

@@ -37,6 +37,7 @@ server {
     ssl_certificate_key /etc/letsencrypt/live/thekims.family/privkey.pem;
 
     include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
 
     location /media/ {
         proxy_pass http://garage:3902/;

docker/nginx/conf.d/timothykim.net.conf

@@ -37,6 +37,7 @@ server {
     ssl_certificate_key /etc/letsencrypt/live/timothykim.net/privkey.pem;
 
     include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
 
     location /media/ {
         proxy_pass http://garage:3902/;

scripts/bootstrap.sh

@@ -99,7 +99,7 @@ fi
 echo "==> Setting up deploy SSH key..."
 mkdir -p /home/deploy/.ssh
 chmod 700 /home/deploy/.ssh
-echo "command=\"sudo /opt/hantim/scripts/deploy.sh \$SSH_ORIGINAL_COMMAND\",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty $DEPLOY_PUBKEY" > /home/deploy/.ssh/authorized_keys
+echo "command=\"sudo /opt/hantim/scripts/deploy.sh \\\"\$SSH_ORIGINAL_COMMAND\\\"\",no-port-forwarding,no-agent-forwarding,no-X11-forwarding,no-pty $DEPLOY_PUBKEY" > /home/deploy/.ssh/authorized_keys
 chmod 600 /home/deploy/.ssh/authorized_keys
 chown -R deploy:deploy /home/deploy/.ssh
 

scripts/configure.sh

@@ -37,6 +37,25 @@ fi
 echo "==> Deploying services..."
 # Deploy nginx first (creates the shared network)
 bash "$REPO_DIR/scripts/deploy.sh" deploy-nginx
+
+# Self-heal renewal config drift: certs issued by the old --standalone flow
+# (removed in de1d60b) can never renew because nginx now holds port 80. Reissue
+# any such cert via the canonical webroot path. nginx must be up (above) to
+# serve the ACME challenge. Best-effort: a single failure warns but does not
+# block the rest of the provision.
+echo "==> Healing standalone cert renewal configs..."
+heal_failed=false
+for conf in /etc/letsencrypt/renewal/*.conf; do
+    [ -e "$conf" ] || continue
+    grep -q '^authenticator = standalone' "$conf" || continue
+    cert_name=$(basename "$conf" .conf)
+    echo "  $cert_name uses standalone; reissuing via webroot..."
+    if ! bash "$REPO_DIR/scripts/deploy.sh" "cert-$cert_name"; then
+        echo "  WARNING: failed to reissue $cert_name; leaving standalone config in place"
+        heal_failed=true
+    fi
+done
+
 # Deploy all other apps
 for app in "$REPO_DIR"/docker/*/; do
     app_name=$(basename "$app")
@@ -46,3 +65,8 @@ for app in "$REPO_DIR"/docker/*/; do
 done
 
 echo "==> Done."
+
+if [ "$heal_failed" = true ]; then
+    echo "ERROR: one or more standalone certs could not be reissued (see warnings above)."
+    exit 1
+fi

tests/conftest.py

@@ -138,7 +138,7 @@ def cleanup(secrets, revert_git=False):
             headers=auth,
         )
         if status == 200 and data:
-            for m in data.get("monitors", []):
+            for m in data.get("data", []):
                 if m.get("url") == f"https://www.{TEST_DOMAIN}":
                     api("DELETE", f"{UPTIMEROBOT_API}/monitors/{m['id']}", headers=auth)
     except Exception as e:
@@ -149,10 +149,10 @@ def cleanup(secrets, revert_git=False):
         token = secrets["hantim-garage-admin-token"]
         auth = {"Authorization": f"Bearer {token}", "Content-Type": "application/json"}
         status, data = api(
-            "POST",
+            "GET",
-            f"{GARAGE_API}/v2/GetBucketInfo",
+            f"{GARAGE_API}/v2/GetBucketInfo?"
+            + urllib.parse.urlencode({"globalAlias": TEST_DOMAIN}),
             headers=auth,
-            data={"globalAlias": TEST_DOMAIN},
         )
         if status == 200 and data and data.get("id"):
             bid = data["id"]
@@ -169,14 +169,8 @@ def cleanup(secrets, revert_git=False):
                         "permissions": {"read": True, "write": True, "owner": False},
                     },
                 )
-            # Remove global alias
+            # Delete bucket directly (Garage rejects alias removal when
-            api(
+            # it's the bucket's only alias, so skip RemoveBucketAlias)
-                "POST",
-                f"{GARAGE_API}/v2/RemoveBucketAlias",
-                headers=auth,
-                data={"bucketId": bid, "globalAlias": TEST_DOMAIN},
-            )
-            # Delete bucket (id is a query param, not body)
             api(
                 "POST",
                 f"{GARAGE_API}/v2/DeleteBucket?"
@@ -250,14 +244,21 @@ def cleanup(secrets, revert_git=False):
         except Exception as e:
             errors.append(f"Local {path}: {e}")
 
-    # --- Vultr DNS zone ---
+    # --- Vultr DNS records (keep zone to avoid slow re-creation) ---
     try:
         key = secrets["hantim-vultr-api-key"]
-        api(
+        auth = {"Authorization": f"Bearer {key}"}
-            "DELETE",
+        status, data = api(
-            f"{VULTR_API}/domains/{TEST_DOMAIN}",
+            "GET", f"{VULTR_API}/domains/{TEST_DOMAIN}/records", headers=auth
-            headers={"Authorization": f"Bearer {key}"},
         )
+        if status == 200 and data:
+            for r in data.get("records", []):
+                if r["type"] == "A" and r["name"] in ("", "www"):
+                    api(
+                        "DELETE",
+                        f"{VULTR_API}/domains/{TEST_DOMAIN}/records/{r['id']}",
+                        headers=auth,
+                    )
     except Exception as e:
         errors.append(f"Vultr: {e}")
 

tests/test_app_e2e.py

@@ -5,8 +5,8 @@ Run: uvx pytest tests/test_app_e2e.py -v -x
 """
 
 import os
-import ssl
 import subprocess
+import time
 import urllib.error
 import urllib.parse
 import urllib.request
@@ -104,19 +104,17 @@ class TestAppSubcommands:
 
         auth = {
             "Authorization": f"Bearer {secrets['hantim-garage-admin-token']}",
-            "Content-Type": "application/json",
         }
         status, data = api(
-            "POST",
+            "GET",
-            f"{GARAGE_API}/v2/GetBucketInfo",
+            f"{GARAGE_API}/v2/GetBucketInfo?"
+            + urllib.parse.urlencode({"globalAlias": TEST_DOMAIN}),
             headers=auth,
-            data={"globalAlias": TEST_DOMAIN},
         )
         assert status == 200, f"Bucket not found (HTTP {status})"
         assert data["id"], "Bucket has no ID"
         assert TEST_DOMAIN in data.get("globalAliases", []), "Global alias not set"
-        website = data.get("websiteAccess", {})
+        assert data.get("websiteAccess") is True, f"Website access not enabled: {data.get('websiteAccess')}"
-        assert website.get("enabled") is True, f"Website access not enabled: {website}"
 
     def test_build(self, secrets):
         result = run_app("build", timeout=180)
@@ -126,16 +124,24 @@ class TestAppSubcommands:
         result = run_app("monitor")
         assert result.returncode == 0, f"app.sh monitor failed:\n{result.stdout}\n{result.stderr}"
 
+        # UptimeRobot API may have brief eventual consistency after create
         auth = {"Authorization": f"Bearer {secrets['hantim-uptimerobot-api-key']}"}
-        status, data = api(
+        target = f"https://www.{TEST_DOMAIN}"
-            "GET",
+        for attempt in range(3):
-            f"{UPTIMEROBOT_API}/monitors?"
+            status, data = api(
-            + urllib.parse.urlencode({"search": TEST_DOMAIN}),
+                "GET",
-            headers=auth,
+                f"{UPTIMEROBOT_API}/monitors",
+                headers=auth,
+            )
+            assert status == 200, f"UptimeRobot API returned {status}"
+            urls = [m["url"] for m in data.get("data", [])]
+            if target in urls:
+                break
+            time.sleep(2)
+        assert target in urls, (
+            f"Monitor not found. URLs: {urls}\n"
+            f"app.sh output:\n{result.stdout}"
         )
-        assert status == 200, f"UptimeRobot API returned {status}"
-        urls = [m["url"] for m in data.get("monitors", [])]
-        assert f"https://www.{TEST_DOMAIN}" in urls, f"Monitor not found. URLs: {urls}"
 
 
 class TestAppAll:
@@ -191,35 +197,40 @@ class TestAppAll:
         # --- Garage bucket ---
         garage_auth = {
             "Authorization": f"Bearer {secrets['hantim-garage-admin-token']}",
-            "Content-Type": "application/json",
         }
         status, data = api(
-            "POST",
+            "GET",
-            f"{GARAGE_API}/v2/GetBucketInfo",
+            f"{GARAGE_API}/v2/GetBucketInfo?"
+            + urllib.parse.urlencode({"globalAlias": TEST_DOMAIN}),
             headers=garage_auth,
-            data={"globalAlias": TEST_DOMAIN},
         )
         assert status == 200, "Garage bucket not found"
 
         # --- Site is live ---
-        ctx = ssl.create_default_context()
+        # Use server IP from verified DNS records to bypass negative cache
-        req = urllib.request.Request(f"https://www.{TEST_DOMAIN}", method="HEAD")
+        server_ip = next(
-        try:
+            r["data"] for r in records if r["type"] == "A" and r["name"] == ""
-            resp = urllib.request.urlopen(req, context=ctx, timeout=10)
+        )
-            assert resp.status in (200, 301, 302)
+        curl = subprocess.run(
-        except urllib.error.HTTPError:
+            ["curl", "-sf", "--resolve", f"www.{TEST_DOMAIN}:443:{server_ip}",
-            pass  # any HTTP response means the site is up
+             "-o", "/dev/null", "-w", "%{http_code}",
-        except urllib.error.URLError as e:
+             f"https://www.{TEST_DOMAIN}"],
-            pytest.fail(f"Site not reachable: {e}")
+            capture_output=True, text=True, timeout=10,
+        )
+        assert curl.returncode == 0, f"Site not reachable (HTTP {curl.stdout})"
 
         # --- UptimeRobot monitor ---
         ur_auth = {"Authorization": f"Bearer {secrets['hantim-uptimerobot-api-key']}"}
-        status, data = api(
+        target = f"https://www.{TEST_DOMAIN}"
-            "GET",
+        for attempt in range(3):
-            f"{UPTIMEROBOT_API}/monitors?"
+            status, data = api(
-            + urllib.parse.urlencode({"search": TEST_DOMAIN}),
+                "GET",
-            headers=ur_auth,
+                f"{UPTIMEROBOT_API}/monitors",
-        )
+                headers=ur_auth,
-        assert status == 200
+            )
-        urls = [m["url"] for m in data.get("monitors", [])]
+            assert status == 200
-        assert f"https://www.{TEST_DOMAIN}" in urls
+            urls = [m["url"] for m in data.get("data", [])]
+            if target in urls:
+                break
+            time.sleep(2)
+        assert target in urls

tests/test_tls.py

@@ -26,6 +26,7 @@ def test_tls12_works():
         assert s.version() == "TLSv1.2"
 
 
+@pytest.mark.filterwarnings("ignore::DeprecationWarning")
 def test_tls11_rejected():
     ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
     ctx.minimum_version = ssl.TLSVersion.TLSv1_1

tools/app.sh

@@ -356,6 +356,7 @@ server {
     ssl_certificate_key /etc/letsencrypt/live/$APP/privkey.pem;
 
     include /etc/nginx/conf.d/security-headers.inc;
+    include /etc/nginx/conf.d/goatcounter.inc;
 
     location /media/ {
         proxy_pass http://garage:3902/;
@@ -403,10 +404,8 @@ cmd_garage() {
 
     BUCKET_ID=$(echo "$BUCKET_RESP" | jq -r '.id')
     if [ -z "$BUCKET_ID" ] || [ "$BUCKET_ID" = "null" ]; then
-        BUCKET_ID=$(curl -s -X POST "$GARAGE_API/v2/GetBucketInfo" \
+        BUCKET_ID=$(curl -s "$GARAGE_API/v2/GetBucketInfo?globalAlias=$APP" \
-            -H "$GARAGE_AUTH" \
+            -H "$GARAGE_AUTH" | jq -r '.id')
-            -H "Content-Type: application/json" \
-            -d "{\"globalAlias\": \"$APP\"}" | jq -r '.id')
         if [ -z "$BUCKET_ID" ] || [ "$BUCKET_ID" = "null" ]; then
             echo "Error: Failed to create or find bucket for $APP"
             echo "$BUCKET_RESP"
@@ -499,7 +498,7 @@ cmd_monitor() {
         --data-urlencode "search=$APP")
 
     MATCH=$(echo "$EXISTING" | jq -r --arg url "$MONITOR_URL" \
-        '.monitors[]? | select(.url == $url) | .id')
+        '.data[]? | select(.url == $url) | .id')
     if [ -n "$MATCH" ]; then
         echo "  Monitor already exists (id: $MATCH), skipping."
         return
@@ -528,6 +527,7 @@ cmd_monitor() {
 cmd_all() {
     cmd_dns
     cmd_repo
+    cmd_cert
     cmd_files
 
     echo "==> Committing and pushing hantim-server..."
@@ -540,7 +540,6 @@ cmd_all() {
         git push
     fi
 
-    cmd_cert
     cmd_garage
     cmd_build
     cmd_verify

tools/service.sh

@@ -256,6 +256,11 @@ cmd_nginx() {
     local port="$2"
     local fqdn="$name.hantim.net"
 
+    if ! [[ "$port" =~ ^[0-9]+$ ]]; then
+        echo "Error: port must be a number."
+        exit 1
+    fi
+
     if [ -e "$REPO_ROOT/docker/nginx/conf.d/$fqdn.conf" ]; then
         echo "  docker/nginx/conf.d/$fqdn.conf already exists, skipping."
         return
@@ -293,6 +298,7 @@ server {
         proxy_set_header Host \$host;
         proxy_set_header X-Real-IP \$remote_addr;
         proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
+        proxy_hide_header X-Frame-Options;
     }
 }
 NGINX