199 lines
10 KiB
Markdown
199 lines
10 KiB
Markdown
# Gitea-triggered application jobs
|
|
|
|
> **Retired:** the active Run app workflow now uses restricted SSH. Follow
|
|
> [SSH_RUN.md](SSH_RUN.md), keep the reconcile timer disabled, and do not use
|
|
> the installation/activation steps below. Retained for historical recovery context.
|
|
|
|
## Flow
|
|
|
|
The app is a one-shot collector/importer, not a continuously running service.
|
|
|
|
1. Push to **master** (the actual default branch). CI runs lint, offline tests,
|
|
disposable PostgreSQL 17 integration tests and package checks.
|
|
2. After tests pass, CI builds and publishes
|
|
`gitea.vandeplanque.com/tonio/car-tracker:<full-commit-SHA>`. Reruns skip an
|
|
existing SHA tag. Tags are still mutable at the registry; requests pin a digest.
|
|
3. Choose **Actions → Run app → Run workflow**, select workflow branch **master**,
|
|
and enter a published full SHA or a Git tag pointing to it. Branch names and
|
|
`latest` are not release selectors. The release must be in master's history.
|
|
4. The workflow pulls the image on the builder VM, verifies its revision label,
|
|
and commits `deploy/request.json` to the dedicated **gitops** branch. It creates
|
|
that branch from master on the first request. Every dispatch gets a new UUID,
|
|
even for the same image. Concurrent non-fast-forward pushes fail; no force push.
|
|
5. A production user-systemd timer reads the request history using a separate bare
|
|
Git checkout and runs requests serially, once each. It uses the image digest and
|
|
its bounded default command. Searches come from that image, not a host checkout.
|
|
|
|
**Green Run app means queued, not successfully collected/imported.** The production
|
|
journal, local request state and raw manifest/sidecar report the actual outcome.
|
|
There is no SSH deployment, incoming deployment API, automatic migration, daily
|
|
collection schedule, or automatic retry after collection starts.
|
|
|
|
## Runner and network configuration
|
|
|
|
`ops/ci-runner/compose.yaml` and `config.yaml` match the new setup at
|
|
`~/containers/gitea-runner/`: project `gitea-runner`, `./data` bind,
|
|
`python-ci:docker://node:22-bookworm`, capacity 1, timeout 45 minutes, and
|
|
`container.docker_host: ""` for job socket access. The build job installs Docker
|
|
CLI/Buildx; the daemon runs on the builder VM. No privileged jobs or arbitrary
|
|
workflow host bind mounts are enabled.
|
|
|
|
Socket access grants jobs root-equivalent control of the **builder VM**, including
|
|
other jobs and runner registration. Run only trusted workflows. A label is routing,
|
|
not a security boundary. No production passwords, backups or raw data on that VM.
|
|
GitOps avoids exposing production credentials to CI, but does not make arbitrary
|
|
images trustworthy: the approved image can read the mounted DB password in production.
|
|
|
|
Keep runner configuration deployment manual. Inspect before copying repository
|
|
files over deployed files; preserve `data/.runner`. Do not copy the old runner's
|
|
named volume, unregister the working new runner, or use `down -v` during migration.
|
|
|
|
Runner/job `extra_hosts` map Gitea to `192.168.1.56` with HTTPS verification intact.
|
|
**Docker Engine also needs working registry DNS on the builder VM**; container
|
|
`extra_hosts` does not affect that daemon. If DNS still returns the wrong public
|
|
address, carefully add this entry to the builder VM's `/etc/hosts`:
|
|
|
|
```text
|
|
192.168.1.56 gitea.vandeplanque.com
|
|
```
|
|
|
|
Reserve that LAN address in DHCP. Production Docker also needs a working registry
|
|
path. Never configure an insecure registry or disable TLS verification.
|
|
|
|
## Gitea Actions secrets
|
|
|
|
Create these repository secrets in Settings → Actions → Secrets:
|
|
|
|
| Secret | Purpose |
|
|
| --- | --- |
|
|
| `REGISTRY_USER` | `tonio`: owner of the current package namespace |
|
|
| `REGISTRY_TOKEN` | Token with `write:package` (includes package reads) |
|
|
| `GITOPS_USER` | Account allowed to write the repository's gitops branch |
|
|
| `GITOPS_TOKEN` | Separate token with `write:repository` |
|
|
|
|
For the current user-owned `tonio/car-tracker` package, publishing must authenticate
|
|
as `tonio`. A bot added as a repository collaborator cannot publish to that user's
|
|
package namespace. Use separate narrowly scoped tokens; Gitea 1.23 token scopes
|
|
are category-wide, not restricted to one repository. Private user-owned packages
|
|
also require the owner's identity for reads; production uses a separate
|
|
`read:package` token. Public user profiles can expose their packages publicly,
|
|
regardless of linked repository visibility.
|
|
|
|
Gitea 1.23 does not enforce GitHub-style
|
|
job `permissions` as a security boundary: set token scopes and account/branch access
|
|
in Gitea itself. Protect master; restrict the state writer to gitops through branch
|
|
protection where possible. Never rewrite the gitops branch's audit log/queue.
|
|
Verify the package owner's visibility; packages do not inherit repository
|
|
access controls.
|
|
|
|
Enter tokens only in the Gitea UI, not chat, commands, committed files or logs.
|
|
The workflows use stdin for registry login and a temporary Git askpass helper,
|
|
not tokens in command arguments or URLs. The built-in Actions token is not assumed
|
|
to have write privileges on this older release. No production password or SSH
|
|
deployment key belongs in Actions secrets.
|
|
|
|
## Install the production consumer (manual, no run yet)
|
|
|
|
Follow [host runtime configuration](DEPLOYMENT.md) first: private directories,
|
|
`app.env`, `pgpass`, TLS and HBA rule must exist. Migrations stay manual. The chosen
|
|
stack root here is `~/containers/car-tracker`, not the `/srv` example.
|
|
|
|
**Disable the old SHA-polling agent before installing this version.** It checked
|
|
out releases in the development repository and retried failed scrapes. Neither
|
|
behavior is used here. The obsolete `deploy/prod.tag` is removed; do not activate
|
|
the old agent. It will see no desired tag after these changes are published.
|
|
|
|
```sh
|
|
systemctl --user disable --now car-tracker-reconcile.timer
|
|
systemctl --user status car-tracker-reconcile.service
|
|
```
|
|
|
|
If a service/job is active, let it finalize before replacing it; investigate uncertain
|
|
imports. From the updated repository root, inspect destination files first, then:
|
|
|
|
```sh
|
|
install -m 0700 ops/gitops/reconcile.py ~/containers/car-tracker/reconcile.py
|
|
install -m 0600 ops/app/compose.runtime.yaml ~/containers/car-tracker/compose.runtime.yaml
|
|
mkdir -p ~/.config/systemd/user
|
|
install -m 0644 ops/gitops/car-tracker-reconcile.service ~/.config/systemd/user/
|
|
install -m 0644 ops/gitops/car-tracker-reconcile.timer ~/.config/systemd/user/
|
|
systemctl --user daemon-reload
|
|
```
|
|
|
|
Create a **separate** bare checkout only if `desired.git` doesn't already exist:
|
|
|
|
```sh
|
|
git clone --bare ssh://git@debian.tailf52e5b.ts.net:2222/tonio/car-tracker.git \
|
|
~/containers/car-tracker/desired.git
|
|
chmod 0700 ~/containers/car-tracker/desired.git
|
|
```
|
|
|
|
Prefer a read-only repository deploy key. Verify SSH host trust/access interactively
|
|
before enabling the timer. The agent never checks out or resets the development repo.
|
|
|
|
For private registry pulls, run `docker login gitea.vandeplanque.com` interactively
|
|
on production with a separate **read-only package token**, not the publisher token.
|
|
Protect Docker's credential file or use a credential helper. Credentials remain local.
|
|
|
|
Both Compose stacks use `car-tracker-app` and the same subnet/storage. If the old
|
|
pre-rename `car-traker-app` network occupies the subnet, remove it through the old
|
|
project only when no job is active; do not delete unrelated networks.
|
|
|
|
Before dispatch, validate with a real published digest (never this placeholder):
|
|
|
|
```sh
|
|
CAR_TRACKER_IMAGE='gitea.vandeplanque.com/tonio/car-tracker@sha256:<published-digest>' \
|
|
docker compose --env-file ~/containers/car-tracker/app.env \
|
|
-f ~/containers/car-tracker/compose.runtime.yaml config --quiet
|
|
```
|
|
|
|
When ready:
|
|
|
|
```sh
|
|
sudo loginctl enable-linger tonio
|
|
systemctl --user enable --now car-tracker-reconcile.timer
|
|
```
|
|
|
|
Review gitops history first: enabling can execute all outstanding requests. If the
|
|
branch does not exist yet, fetching fails and no app starts; later polls try again.
|
|
Repository updates do not automatically replace deployed agent/Compose files.
|
|
|
|
## Outcomes and recovery
|
|
|
|
The timer polls approximately every minute after the preceding cycle finishes.
|
|
All request commits are processed in order, so two button presses between polls
|
|
are not silently reduced to the latest image.
|
|
|
|
```sh
|
|
systemctl --user list-timers car-tracker-reconcile.timer
|
|
journalctl --user -u car-tracker-reconcile.service -n 100 --no-pager
|
|
ls ~/containers/car-tracker/state/requests/
|
|
```
|
|
|
|
Each `state/requests/<uuid>.json` records a request and pipeline exit code. `finished`
|
|
means exited, **not** complete source coverage. See [pipeline exit meanings](DEPLOYMENT.md#outcomes-and-recovery).
|
|
Raw files and `pipeline.json` stay private on production, never CI artifacts.
|
|
|
|
An attempt is atomically recorded **before launching** the collector. It is never
|
|
rerun automatically, regardless of exit code. A crash/timeout may leave
|
|
`status: attempted`, exit unknown. This blocks further requests until investigated.
|
|
This favors a missed run over duplicate scraping; it is not an exactly-once guarantee.
|
|
|
|
Inspect `car-tracker-run-<uuid>`, the raw manifest and database outcome. Recover an
|
|
uncertain import by replaying the exact finalized manifest, not scraping again.
|
|
After investigation, retain the record and explicitly set its local status to
|
|
`resolved` to unblock future requests. Do not delete state to force retries. Back
|
|
up `state/requests`, raw files and the DB: losing the ledger can replay old jobs.
|
|
|
|
Registry/config failures **before** recording an attempt can retry because no
|
|
collection started. No retries after the marker. Never overlap standalone CLI or
|
|
recovery operations with a pipeline on the same output root.
|
|
|
|
For an intentional repeat or an older image, dispatch **Run app again**: a new UUID
|
|
is recorded. Reverting a state commit containing a consumed UUID does not repeat
|
|
it. "Rollback" here selects an older version for a new run; it does not undo DB data.
|
|
|
|
Build/image retention remains manual on the builder VM. Check `docker system df`
|
|
and prune unused build cache deliberately; never prune volumes to free build space.
|
|
No live collection is performed while implementing/testing these files.
|