car-tracker/docs/DEPLOYMENT.md

# Manual Homelab Deployment

## Scope
The next milestone is one manual **collection -> raw files -> PostgreSQL import**
job on the homelab. Development stays on the laptop; build and run from a repository
checkout on the existing Debian VM, alongside Gitea, its runner, and PostgreSQL 17.
Use the existing database and role, currently both `car_tracker`.
Do not provision another production PostgreSQL server or recreate those objects.

`ops/app/compose.yaml` is the one-shot application stack, separate from
`ops/ci-runner`. There is no scheduling, restart loop, or automatic CI deployment.
Run commands from the homelab checkout's root with Docker Engine and Compose
available. Review the revision and `searches.toml`; do not run these in CI.
After these changes are published, update the homelab checkout before building.

## Private Runtime Configuration
Use an external, nonsecret Compose env file, for example
`/srv/car-tracker/app.env`. These absolute paths are examples, not secrets.
Inspect existing files first: never overwrite an existing env file or `.pgpass`.
Use `ops/app/.env.example` as the reference when creating a new file locally:

```dotenv
APP_UID=1000
APP_GID=1000
RAW_DATA_DIR=/srv/car-tracker/raw
DB_CONFIG_DIR=/srv/car-tracker/db
PGHOST=postgres.your-lan.example
PGPORT=5432
PGDATABASE=car_tracker
PGUSER=car_tracker
PGSSLMODE=require
APP_SUBNET=172.30.17.0/28
```

Replace the hostname with the existing VM's reachable LAN/VPN hostname and choose
the numeric `APP_UID`/`APP_GID` that will own the mounted files. Keep the application
user non-root. Both bind directories must already exist; Compose refuses to create
missing source directories. After inspecting `/srv` and any existing application
directories, create only missing directories and apply permissions deliberately:

```sh
sudo install -d -m 0700 -o 1000 -g 1000 /srv/car-tracker/raw /srv/car-tracker/db
```

Use your chosen UID/GID instead of `1000` in all ownership commands. Both
`RAW_DATA_DIR` and `DB_CONFIG_DIR` must have mode `0700` and the correct ownership.
The raw directory must be writable by the container user; the DB directory must be
traversable by that user even though its container mount is read-only.

Create or carefully edit `/srv/car-tracker/db/pgpass` locally, using a private
editor/session. Its matching entry has this form (do not use the placeholder):

```text
postgres.your-lan.example:5432:car_tracker:car_tracker:<password entered locally>
```

Escape `:` and `\` in the password according to libpq password-file rules.
Do not put passwords in commands, the env file, Git, logs, or chat. Do not send
tokens or credentials to anyone to complete this setup. Preserve existing password
files and entries rather than replacing them blindly. Protect the dedicated file:

```sh
sudo chown 1000:1000 /srv/car-tracker/db/pgpass
sudo chmod 0600 /srv/car-tracker/db/pgpass
```

The password is supplied only through `/run/db/pgpass`, mounted from
`DB_CONFIG_DIR`. Incorrect ownership prevents reads; libpq ignores a password file
with overly permissive permissions. Keep this directory dedicated to app DB files,
not unrelated credentials.

Compose interpolates the external env file but passes only the runtime whitelist:
`PGHOST`, `PGPORT`, `PGDATABASE`, `PGUSER`, and `PGSSLMODE`, plus fixed
`PGPASSFILE=/run/db/pgpass` and `PGSSLROOTCERT=/run/db/root.crt`.
It does not pass through `DATABASE_URL`, `PGPASSWORD`, or arbitrary host variables.
Shell variables can override Compose env-file interpolation; unset stale matching
variables before using this configuration. Always specify `--env-file` explicitly.

## TLS And Network Access
`PGSSLMODE` defaults to `require`: require encryption, but do not treat this as
server identity verification. Prefer `verify-full` with the trusted CA certificate
at `/srv/car-tracker/db/root.crt`, readable by the application user. The existing
directory mount exposes it as `/run/db/root.crt`; `PGHOST` must match the server
certificate. Configure PostgreSQL server TLS before running. Never fix production
connectivity by disabling TLS or adding `trust` authentication.

`localhost` inside the container is not the VM's PostgreSQL server. Prefer the VM's
LAN/VPN hostname. Compose also provides `host.docker.internal` through
`extra_hosts: host-gateway`, but PostgreSQL must listen on that gateway address if
you use it. A listener only on loopback or the LAN address does not necessarily
accept gateway connections. Certificate hostname matching still applies.

The app has a dedicated bridge subnet, default `172.30.17.0/28`. Check existing
Docker, LAN, and VPN routes for conflicts before starting; configure `APP_SUBNET`
to a non-overlapping range if needed. Coordinate any network changes with the
existing runner and services. Do not expose PostgreSQL publicly.

PostgreSQL may see a container client address different from the laptop's address.
The laptop's working HBA rule is therefore not proof this job can connect. Inspect
the actual client address in PostgreSQL logs and existing firewall/HBA rules. A new
narrow rule may be needed for the default app subnet:

```text
hostssl  car_tracker  car_tracker  172.30.17.0/28  scram-sha-256
```

Use the configured subnet and actual observed addressing, not a blindly copied
rule. HBA rules are first-match; preserve unrelated rules and permit only the
necessary source/port through the firewall. Check `pg_hba_file_rules` for errors
and reload authentication changes with `SELECT pg_reload_conf();` as an admin.
Changing `listen_addresses` requires a PostgreSQL restart: coordinate downtime for
the shared VM's clients rather than restarting casually. See
[PostgreSQL troubleshooting](POSTGRESQL.md#connection-troubleshooting) for details;
that document's `car_traker` names are examples, not this VM's `car_tracker` names.

## Build And Smoke Run
Validate without printing interpolated configuration, then build on the homelab:

```sh
docker compose --env-file /srv/car-tracker/app.env -f ops/app/compose.yaml config --quiet
docker compose --env-file /srv/car-tracker/app.env -f ops/app/compose.yaml build app
docker compose --env-file /srv/car-tracker/app.env -f ops/app/compose.yaml run --rm app car-traker-db check
```

`check` is read-only connectivity testing, not a schema or migration-privilege
check. The pipeline additionally checks access to `silver.collection_runs` before
opening the browser. No new migrations are required for this milestone. If the
existing packaged migrations have not yet been applied, apply them explicitly:

```sh
docker compose --env-file /srv/car-tracker/app.env -f ops/app/compose.yaml run --rm app car-traker-db migrate
```

Migrations are not automatic. Once DB access/schema are ready, run the image's
default bounded headed pipeline: Saab 900, one page per selected search, at most
one unique detail request for the run, with the default five-second request delay.

```sh
docker compose --env-file /srv/car-tracker/app.env -f ops/app/compose.yaml run --rm app
```

Inspect the reported manifest and import outcome before expanding. After a usable
smoke run, this is the explicit larger manual command:

```sh
docker compose --env-file /srv/car-tracker/app.env -f ops/app/compose.yaml run --rm app \
  car-traker-pipeline --headed --config /app/searches.toml --output /data \
  --family saab-900 --max-pages 3 --max-details 5
```

Limits can intentionally produce partial coverage. Do not loop or retry a blocked
source automatically; stop and investigate access failures before another scrape.

## Outcomes And Recovery
The pipeline prints `Manifest: /data/<date>/<run-id>/manifest.json`, reports a JSON
summary after successful import, and imports
that exact finalized run, including partial or failed collection runs. Host files
are under `/srv/car-tracker/raw/<date>/<run-id>/`. Import success does not imply
complete source coverage. Keep finalized raw files and manifests immutable.

| Pipeline exit | Meaning |
| --- | --- |
| 0 | Complete collection imported, without rejected records |
| 1 | Incomplete collection imported, without rejected records |
| 2 | Import or replay has rejected records; inspect validation issues |
| 3 | Operational, configuration, locking, or database failure |
| 130 | Cancellation/interruption |

`pipeline.json` is a separate operational sidecar, not a rewrite of raw coverage.
It records `collection_status`, `import_status`, and an import summary on success.
An interrupted import or lost DB response can leave `import_status: unknown`:
the transaction may already have committed. Recover by idempotently replaying the
exact manifest with the DB CLI, not by rerunning the scrape. Substitute the exact
date/run ID printed by the pipeline:

```sh
docker compose --env-file /srv/car-tracker/app.env -f ops/app/compose.yaml run --rm app \
  car-traker-db import '/data/<date>/<run-id>/manifest.json'
```

An identical replay reports `already_imported` without duplicating observations.
The standalone DB CLI does not update `pipeline.json`; retain its replay result
separately. Its exit codes differ; see [import documentation](POSTGRESQL.md#migrate-and-import).
Do not alter raw files to force a replay past integrity checks.

`browser_cleanup.json` may record a nonfatal browser teardown error after raw
finalization; the pipeline can still import that run. Allow graceful cancellation
to finish. A hard kill, OOM, or expired stop grace can leave a `running` manifest,
no sidecar, or stale `running` import state. A `running` manifest is not imported;
investigate rather than marking it complete or assuming a DB rollback occurred.

The output-root `.pipeline.lock` covers the whole manual pipeline job, including
collection and import. Do not delete an active lock file; the file can remain after
exit and its presence alone does not mean a job is active. Standalone collection
and DB CLIs do not honor this lock. Never overlap them with a pipeline or other
manual job using the same output root, including during recovery.

## Runtime Boundaries

Compose runs non-root with a read-only root filesystem, all capabilities dropped,
and `no-new-privileges`. There is no Docker socket mount or published port.
Runtime limits are 1.5 CPUs, 2 GiB memory, 256 PIDs, 512 MiB shared memory,
512 MiB `/tmp` tmpfs, a 1 MiB X11 socket tmpfs, and a 30-second stop grace period. These do not constrain
image builds. Keep load low on the VM shared with Gitea, its runner, and PostgreSQL.

The image installs Chromium matching the locked Playwright version. Its custom
Python entrypoint starts Xvfb for headed operation, forwards SIGTERM to the child,
and waits for child cleanup before removing remaining processes and the display.
No external GUI or noVNC service is needed.

Chromium's sandbox remains disabled by Playwright's default launch behavior; an
explicit sandbox trial failed under host security restrictions. These container
settings are not a claim of secure browser isolation. Do not add privileged mode,
`SYS_ADMIN`, or `seccomp=unconfined` to work around this. The browser container can
read mounted DB credentials and shares the VM kernel. Prefer a separate VM for
future browser isolation rather than weakening the shared host's protections.

## Verification And Retention

Live source access from this homelab has not been verified here. All 109 tests
passed inside the built non-root image with its read-only filesystem and resource
limits, using isolated PostgreSQL 17 and synthetic browser responses. An actual
Docker stop also allowed the child to finish cleanup and exit 130 before the
display was removed. These tests do not guarantee live source access.
For optional testing, see [PostgreSQL tests](POSTGRESQL.md#tests) and
`tests/test_browser_pipeline.py`. Browser tests require `RUN_BROWSER_TESTS=1` and
`TEST_DATABASE_URL`; use only a disposable database explicitly ending in `_test`,
never homelab production. Tests truncate data. Default CI skips browser tests.

Retention is still manual: start with 30 days for raw files and private database
copies, including descriptions and raw payloads, and account for relevant backups.
Deleting raw files does not delete DB copies. No retention or backup automation is
provided here; arrange and verify private backups manually before relying on the
data. Keep raw files, DB copies, and backups private and outside Git/CI artifacts.