commit 7df453a8c392265c295c7b899ac4f935537007ba Author: Chris Smith Date: Mon Jun 29 05:04:05 2026 -0500 initial commit diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..a9fd6c6 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,515 @@ +# AGENTS.md — smithpc + +Operational reference for the gaming + homelab rig. Used to orient AI coding assistants (Codex, Claude Code, etc.) when +working on this machine. + +## Communication Style (READ FIRST) + +How Chris wants agents to respond on this system: + +- **Be concise.** Default to short, direct answers. No walls of text, no exhaustive option dumps unless asked. Lead with + the answer, then minimal supporting detail. Expand only when the task genuinely requires it or Chris asks for depth. +- **No filler.** Skip empty praise, validation, motivational language, and hedging. Don't agree to be polite. +- **Be brutally honest and logical.** Challenge assumptions, question reasoning, and point out flaws, contradictions, or + unrealistic ideas directly. Prioritize accuracy over comfort. If an idea is bad, say so and explain why. +- **Push back when warranted.** Disagreement with reasons is more useful than agreement. Never give empty approval. +- This style is subordinate to the Operating Principles below — being concise never means skipping a safety check or a + required confirmation. + +## Operating Principles (READ FIRST) + +These rules apply to every action on this system. They exist because every one of them has cost real time or risked real +harm at some point. + +### 1. VERIFY ASSUMPTIONS BEFORE CHANGING ANYTHING + +Do not assume state. Check it. Examples of assumptions that have caused problems on this system: + +- "The container is running" → check with `docker ps` +- "The mount is active" → check with `mount | grep ` +- "The fstab entry is correct" → check with `cat /etc/fstab` +- "The bucket exists" → check with the actual cloud provider +- "The cron job ran successfully" → check with `journalctl` or the log file +- "The previous edit took effect" → check with `cat` on the file or the running config +- "The drive letter is /dev/sdc" → check with `lsblk` (letters shift between reboots) +- "The compose file was reloaded" → `docker compose restart` does NOT re-read compose; only `down` + `up` does + +**Rule:** Before suggesting a fix, run the check that would prove the assumed cause is correct. Before making a change, +run the check that would prove the change is safe. If a check returns nothing, **say so** — do not silently assume "no +output = success". + +### 2. DESTRUCTIVE OPERATIONS REQUIRE EXPLICIT CONFIRMATION + +The following operations need a one-line "yes I want to do X to Y" confirmation from Chris before running. Do NOT +proceed based on prior implied consent: + +- `rm -rf` of anything outside `/tmp` +- `zpool destroy`, `zfs destroy`, `zfs rollback` +- `restic forget --prune`, `restic forget `, `restic init` against an existing repo +- `docker compose down -v` (the `-v` removes named volumes) +- `dd of=/dev/...` +- `mkfs.*`, `fdisk`, `parted` with write operations +- `nvme format`, `hdparm --security-erase`, `cryptsetup luksErase` +- Anything that modifies `/boot/grub`, `/etc/fstab`, or kernel cmdline +- Editing a Postgres data dir directly +- Direct SQL modifications to Jellyfin/Immich production databases +- Force-pushing changes that delete things in B2 (`b2 rm --versions`) +- Disabling smartd, ZFS scrubs, restic timer, or other safety nets + +**Rule:** Show the exact command, identify exactly what it will affect, and wait for confirmation. "Are you sure?" is +not enough — restate what's being destroyed. + +### 3. SECRETS NEVER GO IN CHAT, COMMANDS, OR COMPOSE FILES + +Past incidents on this system: Mail-in-a-Box password leaked twice, Immich DB password leaked once, B2 keys discussed in +plaintext. Treat as compromised, rotate immediately. + +**Rules:** + +- Never paste secrets into chat with an AI agent (the conversation may be logged or retained) +- Never write secrets directly in compose files — use `.env` files referenced via `env_file:` with `chmod 600 root:root` +- For host scripts, secrets live in `/etc//credentials` or `/etc//env`, root-owned, mode 600 +- If a secret is exposed anywhere (chat, screenshot, accidental git commit, log file), **rotate it before doing anything + else** +- Restic's repo password must be in a password manager too — without it, backups are permanently unrecoverable +- When an agent suggests using a password, it should reference the file path, never the value + +### 4. CHANGE BLAST RADIUS — THIS IS PRODUCTION + +There is no staging environment. There is no test environment. This single host runs: + +- The family's photo library (~490 GB, irreplaceable) +- The family's audiobook collection (purchased content) +- The family's movie/TV server (Kym + kids use this regularly) +- Chris's home directory, SSH keys, gaming setup +- Public-facing services via Caddy + +A bad change here = real consequences. Approach changes like production deployments: + +- Have a rollback plan before starting +- Take a ZFS snapshot first when reasonable +- Verify after the change that the thing still works +- Don't batch unrelated changes — one logical change at a time + +### 5. DATABASE OPERATIONS HAVE STRICT RULES + +This system has been bitten by all three of these. Don't repeat. + +- **SQLite (Jellyfin, etc.):** Service MUST be stopped before any file-level copy. `rsync`/`cp` of a running SQLite DB + corrupts it. Use `sqlite3 .dump` or `.backup` for live exports. Recovery is via `sqlite3 .recover` but lossy. +- **Postgres (Immich):** Never `rsync` the data directory between hosts. Use `pg_dumpall` from a running instance, + restore via `psql` on the target. Keep `DB_PASSWORD` identical across migration or the restored roles won't + authenticate. +- **In general:** A database file appearing intact at the byte level says nothing about whether it's actually + consistent. Always test reads after migration before deleting the source. + +### 6. ROLLBACK PLAN BEFORE DESTRUCTION + +ZFS snapshots are free, instant, and copy-on-write. Use them. + +Before any risky operation on `/data/*`: + +```bash +sudo zfs snapshot tank/@pre- +``` + +To roll back: + +```bash +sudo zfs rollback tank/@pre- +``` + +To clean up later: + +```bash +sudo zfs destroy tank/@pre- +``` + +For non-ZFS data (`/etc`, `/home/chris`, `/mnt/media`): take a restic snapshot or `cp -a` to `/tmp` first. The 30 +seconds spent on a snapshot saves hours of recovery work. + +### 7. CONTAINER LIFECYCLE NUANCES + +These have caused confusion on this system. Know the difference: + +| Command | What it does | When to use | +|-----------------------------------------------|--------------------------------------------------------------------|---------------------------------------------------------| +| `docker compose restart` | Restarts running containers. **Does NOT re-read compose changes.** | When a container is misbehaving but config is unchanged | +| `docker compose down && docker compose up -d` | Stops, removes containers, re-reads compose, recreates | After editing compose.yaml | +| `docker compose up -d --force-recreate` | Recreates even without compose changes | Diagnostic, rare | +| `docker compose down -v` | Adds volume removal — **DESTRUCTIVE**, loses named volume data | Confirm explicitly first | +| `docker compose pull && docker compose up -d` | Updates to latest images | Maintenance | + +**Rule:** When changing a compose file, the next step is ALWAYS `down` then `up -d`. Not `restart`. + +### 8. NETWORK CHANGES ARE COUPLED + +These three things must move together: + +1. **Caddy** routes public DNS hostnames to local services +2. **Mail-in-a-Box DNS** answers public DNS queries for sometimescode.com subdomains +3. **Router port forwards** point 80/443 at smithpc's LAN IP + +Changing one without the others = external services break silently. Examples: + +- Adding a new subdomain → update Caddyfile AND add DNS record AND make sure port forwards still point here +- Moving a service to a new port → update Caddyfile only (port forward + DNS unchanged) +- Changing smithpc's LAN IP → update router port forwards AND DHCP reservation (DNS unchanged because it points to + public IP) + +The `update-dns.sh` script handles record creation/update for hardcoded hostnames. If you add a new public subdomain, +add it to the script's `RECORDS` array too. + +### 9. SERVICES KYM AND THE KIDS USE + +Tier-1 (don't break during family-use hours): + +- **Jellyfin** — kids watch shows, Kym watches movies. Evening + weekend especially. +- **Audiobookshelf** — Kym uses for audiobooks during workout/commute +- **Immich** — Kym accesses for family photos + +If maintenance requires downtime on any of these, give a heads-up first. Don't take them down silently during peak +hours. + +### 10. THIS IS A DUAL-PURPOSE BOX + +smithpc is both a homelab AND Chris's gaming PC. Implications: + +- Reboots interrupt Chris's gaming sessions and any running streams +- High GPU load from a game competes with Immich ML and Jellyfin transcoding +- Steam updates pull bandwidth that affects other services +- Chris reboots into Windows for nothing (no dual-boot), but a hung gaming session can require power-cycling +- 8BitDo controllers may need Steam Big Picture running to stay polled + +When recommending actions, factor in that this isn't a server in a closet — it's the living-room machine. + +### 11. SCOPE DISCIPLINE — "WORKS, DON'T TOUCH" + +Many things on this system are intentionally "good enough" rather than "optimal": + +- Backups are working, even if they emit cache warnings +- Jellyfin posters are mostly fine after recovery, even if a few are still missing artwork +- The dummy plug works as-is, no EDID override needed +- UFW is intentionally disabled (router NAT is sufficient) +- Steam library lives on NVMe (not ZFS) on purpose + +When an agent notices something "could be improved," **ask first** before optimizing. The migration is done. Drift from +working state without consent is not welcome. + +### 12. WHEN TO STOP AND ASK CHRIS + +Don't proceed independently when: + +- The next step would be in the "destructive operations" list (rule 2) +- A check reveals the system state doesn't match what was expected +- An error message references something not covered in this doc +- The fix would touch Tier-1 services (rule 9) during likely-use hours +- The operation would take >30 minutes of downtime +- Secrets would need to be displayed, regenerated, or moved +- Multiple plausible fixes exist and the right one isn't obvious +- The agent finds itself constructing increasingly complex workarounds (sign of a wrong assumption upstream — go back to + rule 1) + +Asking takes 30 seconds. Recovering from a bad assumption takes hours. + +## System Identity + +- **Hostname:** `smithpc` +- **LAN IP:** `192.168.2.189` (static, took over from old OptiPlex slot) +- **OS:** Ubuntu 26.04 LTS (Server install + `ubuntu-desktop-minimal` on top) +- **Owner:** chris (UID 1000, GID 1000) +- **Editor preference:** vim. Prefer `sudoedit` over `sudo vim`. `update-alternatives` set so `crontab -e`, `visudo`, + etc. use vim. +- **Timezone:** Europe/Berlin +- **Notification email:** cgsmith105@gmail.com (Gmail, personal) +- **Outbound mail relay:** `notify@fivedevs.com` via msmtp → box.fivedevs.com (Mail-in-a-Box) + +## Hardware + +- **CPU:** AMD Ryzen 5 7500F (AM5, 6c/12t, Zen 4) +- **Cooler:** Thermalright Peerless Assassin 120 SE +- **Motherboard:** ASRock B850M Pro-A WiFi (mATX) +- **RAM:** 2× 8GB Kingston FURY Beast DDR5-6000 CL30, EXPO enabled (running 6000 MT/s) +- **GPU:** Asus DUAL OC GeForce RTX 5060 8GB (driver 575+, Nvidia Container Toolkit installed) +- **NVMe:** Kingston NV3 500GB (OS) +- **HDDs:** 2× Seagate IronWolf 4TB (ZFS mirror) + 1× Seagate Barracuda 8TB SMR (ST8000DM004, media) +- **PSU:** MSI MAG A750GL 750W Gold +- **Case:** Cooler Master Silencio S400 (dampened, 24/7 operation) +- **Optical:** Pioneer BDC-202 (BD-ROM / DVD-writer combo, SATA) — reads BD/DVD/CD, used for ripping. Device `/dev/sr0` + (also `/dev/cdrom`, `/dev/sg4`), group `cdrom`. NOTE: this is a Blu-ray *reader*, not a writer — earlier doc claimed + an LG BH16NS40, which is not the installed unit. Reading is all ARM needs, so no impact on ripping. +- **Controllers:** 4× 8BitDo Ultimate C 2.4GHz (X-input mode, used with Steam Big Picture for Lego Party couch co-op) +- **Display dongle:** Vestel 4K HDMI dummy plug on HDMI-A-1 (keeps GPU active when monitor sleeps; reports + `55UHD_LCD_TV`) + +### BIOS Settings of Note + +- EXPO Profile 1 enabled (RAM at 6000 MT/s) +- Restore on AC/Power Loss: **Power On** (auto-boot after outage) +- No UPS (3yr stable power history, ZFS handles unclean shutdowns) + +## Storage Layout + +### ZFS Mirror (pool `tank`, mounted at `/data`) + +- 2× IronWolf 4TB in mirror (3.51 TiB usable after overhead) +- `ashift=12`, `compression=lz4`, `atime=off`, `xattr=sa`, `acltype=posixacl` +- Monthly scrub timer enabled (`zfs-scrub-monthly@tank.timer`) +- Daily health check at `/etc/cron.daily/zfs-health-check` (emails on degradation only) + +**Datasets:** + +``` +tank → /data +tank/immich → /data/immich +tank/audiobookshelf → /data/audiobookshelf +tank/docker → /data/docker (Docker data-root) +tank/jellyfin → /data/jellyfin (config only — media on Barracuda) +tank/backups → /data/backups +``` + +All `/data/*` owned by `chris:chris`. + +### Barracuda 8TB + +- Single drive, ext4, mounted at `/mnt/media` +- Contains Jellyfin media (`/mnt/media/jellyfin/{movies,shows,music,books,inbound}`) +- **Mounted via /etc/fstab** with `x-systemd.device-timeout=10` option (persists across reboots). Device currently + `/dev/sdd` but fstab should use UUID for stability. +- **Note:** The currently-active mount shows `x-system.device-timeout=10` (typo, missing `d`). The kernel ignores + unknown options silently, so the mount works but the timeout isn't actually applied. If a reboot ever hangs on missing + drive, fix this typo in `/etc/fstab` to `x-systemd.device-timeout=10`. +- **No redundancy** — replaceable media, not backed up. SMR drive (slow writes, fine for reads). + +### NVMe (`/`) + +- Ubuntu LVM on `/dev/nvme0n1p3` (462.7 GB) +- `/boot/efi` 1G, `/boot` 2G, root LVM uses the rest +- Steam library currently on NVMe (~200-300 GB allocation) + +## Network & Public Access + +- **Domain:** sometimescode.com, DNS at Mail-in-a-Box (`box.fivedevs.com`) +- **Public subdomains** (all reverse-proxied through Caddy): + - `immich.sometimescode.com` → port 2283 + - `audio.sometimescode.com` → port 13378 + - `jellyfin.sometimescode.com` → port 8096 + - `paperless.sometimescode.com`, `actual.sometimescode.com`, `copyparty.sometimescode.com` (referenced in Caddyfile + but services NOT migrated — return 502) +- **Router:** forwards 80/443 to 192.168.2.189 +- **Firewall:** UFW **disabled** (relying on home router NAT) +- **DNS update script:** `/usr/local/bin/update-dns.sh` runs via `/etc/cron.d/update-dns` every 15 min — pushes current + public IP to MIAB DNS records (`immich`, `smithpc`, `jellyfin`, `audio`) + +## Display Server + +- **Wayland** (default in Ubuntu 26.04 with Nvidia 575+) +- Previous attempt to force X11 via `/etc/gdm3/custom.conf` did not stick on this kernel/driver combo +- Auto-suspend disabled: `sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target` +- Screen blank disabled via GNOME settings (Nvidia + Wayland + lock screen had wake-up issues — workaround: never blank + the screen, lock manually with Super+L) + +## SSH + +- Key auth only, password auth disabled (`/etc/ssh/sshd_config.d/99-hardening.conf`) +- Root login disabled +- User SSH key: `~/.ssh/smithhome_id_ed25519` (passphrase-protected, cached via ssh-agent or GNOME Keyring) + +## Docker Stack + +- Docker installed via official `get.docker.com` script +- Data root: `/data/docker` (set via `/etc/docker/daemon.json`) +- **Nvidia Container Toolkit installed** — GPU passthrough works in containers (verified via + `docker run --rm --gpus all nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi`) +- Compose stacks live at `/data/stacks//compose.yaml` +- Managed via **Dockge** (UI at `http://smithpc:5001`, also reads `/data/stacks/`) + +### Running Stacks (all stable as of latest setup) + +| Stack | Port | Purpose | Notes | +|-------------------|--------|-----------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `dockge` | 5001 | Stack manager | Reads `/data/stacks/` | +| `caddy` | 80/443 | Reverse proxy | `network_mode: host`, logs to `/data/caddy/logs/` | +| `immich` | 2283 | Photo library | Uses Postgres + Valkey, GPU-accelerated ML (`-cuda` image), bind mounts `/home/chris/Pictures` for external library, `DB_STORAGE_TYPE=HDD` | +| `audiobookshelf` | 13378 | Audiobooks + podcasts | Data at `/data/audiobookshelf/{audiobooks,podcasts,config,metadata}` | +| `jellyfin` | 8096 | Movies/TV/music | `network_mode: host`, `user: "1000:1000"`, config merged from Debian package layout (`/etc/jellyfin` + `/var/lib/jellyfin` → `/config`), GPU NVENC enabled. **Note:** DB was recovered via `sqlite3 .recover` after migration corruption — `jellyfin.db.broken.bak` preserved in `/data/jellyfin/config/data/` | +| `mailinabox-ddns` | n/a | DNS auto-update | Pushes current public IP to MIAB | + +### Important Compose Conventions + +- All stacks use absolute paths (no `./relative` paths since stack files and data are separate) +- All include `x-dockge.urls` block for clickable links in Dockge UI +- TZ set to `Europe/Berlin` in all stacks +- Container names match service names (`immich_server`, `jellyfin`, `audiobookshelf`, etc.) +- GPU-using services include the `deploy.resources.reservations.devices` Nvidia block + +## Monitoring + +### smartd + +- Configured at `/etc/smartd.conf` +- Monitors 2× IronWolf + NVMe +- Short tests daily 03:00, long tests Sundays 03:00 +- Emails errors only (no test mode) to cgsmith105@gmail.com +- Uses custom script `/usr/local/bin/smartd-notify` → msmtp + +### ZFS Health Check + +- `/etc/cron.daily/zfs-health-check` — emails only if pool not `ONLINE` or errors detected +- No periodic "all is well" emails — silence is success + +### msmtp (mail relay) + +- Config: `/etc/msmtprc` (chmod 600, root:root) +- Account: `notify@fivedevs.com` on `box.fivedevs.com:587` (STARTTLS) +- Logfile: `/var/log/msmtp.log` + +## Backups + +### Restic + Backblaze B2 + +- **Repo:** `b2:cgsmith-personal-backups:smithpc` +- **Credentials:** `/etc/restic/env` (chmod 600, root:root) — contains B2 keyID, applicationKey, repo password, cache + dir, HOME export +- **Script:** `/usr/local/bin/restic-backup.sh` +- **Cache:** `/var/cache/restic` (HOME=/root set in env file to avoid systemd "no $HOME" warning) +- **Schedule:** systemd timer `restic-backup.timer` runs daily at 00:00 ±30min +- **Service:** `restic-backup.service` with `Nice=10`, `IOSchedulingClass=idle`, failure email via msmtp +- **Logs:** `/var/log/restic-backup.log` (logrotate weekly, keep 12) + +**Backed up paths:** + +``` +/data/immich +/data/audiobookshelf +/data/stacks +/home/chris/Pictures +/home/chris/Documents +/home/chris/Desktop +/home/chris/.ssh +/home/chris/.config +/etc +/usr/local/bin +``` + +**Excludes:** `pg_wal`, `*.tmp`, `*.cache`, `node_modules`, `.git/objects/pack` + +**Retention:** 7 daily / 4 weekly / 12 monthly / 5 yearly + 2% random repo check per run. + +**Not backed up (intentional):** `/data/docker` (containers ephemeral), `/mnt/media` (replaceable Jellyfin content), +Steam library. + +### B2 Bucket Configuration + +- **Bucket type:** Private +- **Object Lock:** Disabled +- **Lifecycle rule:** **"Keep only the latest version"** is CONFIGURED but problematic — restic operations may produce + 403 errors on hidden lock/config files. If issues recur, switch to "Keep all versions" or custom "hide=0 days, + delete=1 day". +- **Daily download bandwidth cap:** Set to non-zero ($1/day or higher). $0.00 cap will cause 403 on any restic read + operation. + +## Custom Scripts + +- `/usr/local/bin/restic-backup.sh` — backup + prune + check (root, 700) +- `/usr/local/bin/update-dns.sh` — push IP to MIAB DNS (root, 700) +- `/usr/local/bin/smartd-notify` — smartd email formatter (root, 755) + +## Recovery Notes / Migration Scars + +These are landmines learned the hard way. Avoid repeating them. + +1. **SQLite databases (Jellyfin, etc.) must be stopped before copying.** Rsync of a running SQLite DB corrupts it. Use + `sqlite3 .dump` or stop the service first. The Jellyfin DB on this box was recovered with + `sqlite3 jellyfin.db.broken.bak ".recover"` after exactly this mistake during migration from the OptiPlex. + +2. **Postgres data (Immich) survives clean container shutdown.** `docker compose down` is safe. Use `pg_dumpall` for + migration between hosts — never copy `pg_data` directly. + +3. **B2 + Restic gotchas:** + - $0.00 daily download cap = 403 errors on all operations + - "Keep only latest version" lifecycle = lock files get hidden mid-operation, breaks restic + - App key needs `deleteFiles` capability (not just read/write) + +4. **Wayland + Nvidia + lock screen** = monitor won't wake reliably. Workaround: never blank screen, lock manually with + Super+L. + +5. **sudo doesn't see user's ~/.ssh/config.** For rsync over SSH as sudo, either: + - Use `-e "ssh -F /home/chris/.ssh/config -i /home/chris/.ssh/"` + - Or pre-create destination dirs with chown to your user, then run rsync without sudo + +6. **8BitDo controllers + Linux:** May disconnect during idle. Mitigations: keep Steam Big Picture open (polls + controllers), use USB 2.0 ports (USB 3 has 2.4GHz RF interference), USB extension cables, update firmware on Windows. + +## Things That Don't Yet Exist (TODOs / Wishlist) + +These are planned but not implemented. Don't assume they're present. + +- **CrowdSec** for log monitoring of Caddy access logs +- **Copyparty + FileBot Node** for friend-uploads → Jellyfin pipeline +- **MeTube + Beets** for YouTube music ripping +- **Automatic Ripping Machine (ARM)** for DVD/Blu-ray/CD ripping — not installed. Manual workflows exist instead: `cd-ripping-workflow.md` (audio CDs → MP3 → Jellyfin via abcde + beets), `movie-ripping-workflow.md` (DVD/Blu-ray → H.265 MKV → Jellyfin via HandBrakeCLI + MakeMKV). Interactive helper script: `disc-scan.py` (scans disc, prompts for settings, executes rip). +- **Tailscale** for remote access (currently using public DNS + Caddy) +- **Paperless, Actual Budget, copyparty** — referenced in Caddyfile but services not on smithpc yet + +## Conventions for AI Agents Working on This System + +- **Always use absolute paths** in compose files (`/data/...`, not `./...`) +- **Never embed secrets in compose files or shell history** — use files in `/etc//` with `chmod 600 root:root` + and source/mount them +- **Stack files in `/data/stacks//compose.yaml`** — Dockge will auto-detect +- **Bind-mount media as `:ro` only when intentional** — Jellyfin needs write access for artwork; restrict via filesystem + permissions instead +- **Match in-container paths to original install paths** when migrating services (e.g., Jellyfin's DB references + `/media/share/chris/jellyfin/`, so the bind-mount destination matches even though host path is `/mnt/media`) +- **GPU-accelerated containers** need: + ```yaml + deploy: + resources: + reservations: + devices: + - driver: nvidia + count: all # or specific number + capabilities: [gpu] + ``` +- **All emails route through msmtp** — scripts that need to email should pipe to `msmtp cgsmith105@gmail.com`, never + invoke other mail clients +- **Restic operations:** always source `/etc/restic/env` first. Run as root (sudo). Use `restic unlock` if stale locks + block operations. +- **Before any rsync of a database file:** stop the service first. No exceptions. +- **Owner of all user-facing dirs in `/data`:** `chris:chris` (1000:1000). Containers run as `1000:1000` where possible. + +## Service-Specific Quirks + +### Immich + +- ML container uses `:release-cuda` image (not vanilla `:release`) +- `IMMICH_MACHINE_LEARNING_HARDWARE_ACCELERATION=cuda` in `.env` +- `DB_STORAGE_TYPE=HDD` in compose (Postgres on spinning ZFS) +- External library at `/home/chris/Pictures` (read-only bind mount) +- DB password kept identical between OptiPlex and smithpc to preserve auth across pg_dumpall restore — rotated + post-migration + +### Jellyfin + +- Bind-mount destination INSIDE container is `/media/share/chris/jellyfin` (matches OptiPlex paths so DB references + resolve) +- Host source path is `/mnt/media/jellyfin` +- Mount is read-write (Jellyfin writes artwork next to media files) +- Config merged from Debian package: `/var/lib/jellyfin` → `/config`, `/etc/jellyfin` → `/config/config`, + `/var/cache/jellyfin` → `/cache` +- NVENC hardware transcoding enabled +- Music library managed by **beets** (`~/.config/beets/config.yaml`): imports to `/mnt/media/jellyfin/music`, path format `$albumartist/$album/$track - $title`. After any beets import, trigger a Jellyfin library scan manually. +- `/mnt/media/jellyfin/inbound` is used as a staging area for new media (including CD rips) before final import + +### Caddy + +- `network_mode: host` (matches existing `localhost:PORT` references from OptiPlex Caddyfile) +- Certs migrated from OptiPlex at `/data/caddy/data/caddy/` (no Let's Encrypt re-issue) +- JSON access logs to `/data/caddy/logs/access.log` (for future CrowdSec integration) + +### Dockge + +- Reads compose files from `/data/stacks/` (must be EXACT same path inside container — Dockge passes through file paths) +- Container stack at `/data/stacks/dockge/compose.yaml` +- Bind mounts `/var/run/docker.sock` for control diff --git a/cd-ripping-workflow.md b/cd-ripping-workflow.md new file mode 100644 index 0000000..93fb795 --- /dev/null +++ b/cd-ripping-workflow.md @@ -0,0 +1,90 @@ +# CD Ripping Workflow (abcde + beets → Jellyfin) + +## Tools + +| Tool | Purpose | +|-----------|---------------------------------------------------| +| `abcde` | Rips CD and encodes to MP3 via cdparanoia | +| `lame` | MP3 encoder used by abcde | +| `mid3v2` | ID3 tagger used by abcde (part of python3-mutagen)| +| `beets` | MusicBrainz metadata match + library import | + +Install once: +```bash +sudo apt install -y abcde lame beets python3-mutagen +``` + +## Config Files + +### `~/.abcde.conf` +``` +CDDBMETHOD=musicbrainz +OUTPUTTYPE=mp3 +LAMEOPTS="-b 320" +OUTPUTDIR=/mnt/media/jellyfin/inbound +ACTIONS=cddb,read,encode,tag,move,clean +NOGAP=y +TAGGER=mid3v2 +``` + +### `~/.config/beets/config.yaml` +```yaml +directory: /mnt/media/jellyfin/music +library: ~/.config/beets/musiclibrary.db + +import: + move: yes + write: yes + +paths: + default: $albumartist/$album/$track - $title + singleton: Non-Album/$artist - $title + comp: Compilations/$album/$track - $title +``` + +## Workflow + +**1. Insert the CD, then rip:** +```bash +abcde -d /dev/sr0 +``` +- Looks up the disc on MusicBrainz, prompts to confirm the match +- Encodes to MP3 320kbps +- Stages files to `/mnt/media/jellyfin/inbound///` +- Takes ~5–10 min depending on disc + +**2. Import with beets:** +```bash +beet import -A "/mnt/media/jellyfin/inbound/" +``` +- `-A` runs non-interactively and auto-selects the best MusicBrainz match +- On success, moves files to `/mnt/media/jellyfin/music/// - .mp3` +- If auto-match fails (wrong album, low confidence), drop `-A` and run interactively from a real terminal — beet will prompt you to search or enter a MusicBrainz ID manually. **Cannot be run interactively from a Claude Code session.** + +**3. Trigger Jellyfin rescan:** +Jellyfin dashboard → Libraries → Music → "Scan Library Files" + +## Device Info + +- Drive: Pioneer BDC-202 (BD-ROM / DVD-writer combo) +- Device: `/dev/sr0` (also `/dev/cdrom`) +- `chris` is in the `cdrom` group — no sudo needed to read the disc + +## Troubleshooting + +- **MusicBrainz lookup fails in abcde:** Install `python3-musicbrainzngs` separately. +- **abcde errors "mid3v2 is not in your path":** `sudo apt install python3-mutagen` — mid3v2 is part of that package. +- **beet import shows " - " as artist/album, files land in `__/` folder:** abcde failed to write ID3 tags. Verify with `mid3v2 -l file.mp3`. If blank, tag manually and re-import: + ```bash + mid3v2 -a "Artist" -A "Album" -t "Title" -T "TRACKNUM/TOTAL" -y YEAR --TPOS "DISC/TOTALDISCS" file.mp3 + ``` +- **Multi-disc album: disc number missing in Jellyfin:** beets doesn't set `TPOS` automatically. Tag each disc after import: + ```bash + # CD1 + for f in /mnt/media/jellyfin/music/Artist/Album/0{1..9}-*.mp3 ...; do mid3v2 --TPOS "1/2" "$f"; done + # CD2 + for f in /mnt/media/jellyfin/music/Artist/Album/0{1..9}-*.mp3 ...; do mid3v2 --TPOS "2/2" "$f"; done + ``` + Then trigger a Jellyfin rescan. +- **beet import can't find files:** Check the exact staged path after abcde finishes — album artist name may differ from expected. +- **Jellyfin doesn't pick up music:** Trigger a manual library scan after import. diff --git a/disc-scan.py b/disc-scan.py new file mode 100644 index 0000000..3fea367 --- /dev/null +++ b/disc-scan.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""Scan an optical disc, prompt for rip settings, and optionally execute HandBrakeCLI.""" + +import datetime +import os +import re +import subprocess +import sys + +DEVICE = sys.argv[1] if len(sys.argv) > 1 else "/dev/sr0" + +# ── Scan ────────────────────────────────────────────────────────────────────── + +print(f"Scanning {DEVICE} (takes ~20-30 s)...\n", flush=True) + +proc = subprocess.run( + ["HandBrakeCLI", "-i", DEVICE, "--scan", "-t", "0"], + capture_output=True, text=True, stdin=subprocess.DEVNULL +) +lines = proc.stderr.splitlines() + +titles = {} + +# Pass 1: log lines → duration, audio, subtitles +current = None +checking_audio = checking_sub = False + +for line in lines: + m = re.search(r'scan: scanning title (\d+)', line) + if m: + current = m.group(1) + titles.setdefault(current, {"num": current, "duration": "", "video": "", "audio": [], "subs": []}) + checking_audio = checking_sub = False + continue + + if current is None: + continue + + t = titles[current] + + m = re.search(r'duration is (\d+:\d+:\d+)', line) + if m: + t["duration"] = m.group(1) + continue + + if re.search(r'scan: ignoring title', line): + current = None + continue + + if re.search(r'checking audio', line): + checking_audio, checking_sub = True, False + continue + if re.search(r'checking subtitle', line): + checking_audio, checking_sub = False, True + continue + + m = re.search(r'id=0x[0-9a-f]+, lang=(.+?),\s*3cc=(\w+)', line) + if m: + lang_full = m.group(1).strip() + code = m.group(2) + if checking_audio: + codec_m = re.search(r'\(([^)]+)\)', lang_full) + codec = codec_m.group(1) if codec_m else "?" + name = re.sub(r'\s*\([^)]*\)', '', lang_full).strip() + t["audio"].append(f"{name} ({codec}) [{code}]") + elif checking_sub: + t["subs"].append(code) + +# Pass 2: summary block → video size +current = None +for line in lines: + m = re.match(r'^\s*\+ title (\d+):', line) + if m: + current = m.group(1) + continue + if current and current in titles: + m = re.search(r'\+ size: (\d+x\d+)', line) + if m: + titles[current]["video"] = m.group(1) + +titles = [t for t in titles.values() if t["duration"] and t["duration"] != "00:00:00"] + +if not titles: + print("No titles found. Is a disc inserted and HandBrakeCLI installed?") + sys.exit(1) + +# ── Display ─────────────────────────────────────────────────────────────────── + +def fmt_duration(hms): + try: + h, m, s = (int(x) for x in hms.split(":")) + total_min = h * 60 + m + round(s / 60) + return f"{hms} (ca {total_min} min.)" + except ValueError: + return hms + +longest = max(titles, key=lambda x: x["duration"]) +titles_by_num = {t["num"]: t for t in titles} + +col_dur, col_vid, col_a = 26, 10, 50 +print(f"{'#':<5} {'Duration':<{col_dur}} {'Video':<{col_vid}} {'Audio Tracks':<{col_a}} Subtitles") +print("─" * (5 + col_dur + col_vid + col_a + 20)) + +for t in titles: + marker = " ◀ likely main feature" if t is longest and len(titles) > 1 else "" + audio_str = " / ".join(t["audio"]) if t["audio"] else "—" + subs_str = " ".join(t["subs"]) if t["subs"] else "—" + print(f"{t['num']:<5} {fmt_duration(t['duration']):<{col_dur}} {t['video']:<{col_vid}} {audio_str:<{col_a}} {subs_str}{marker}") + +# ── Interactive setup ───────────────────────────────────────────────────────── + +def ask(question, default=""): + suffix = f" [{default}]" if default else "" + try: + answer = input(f"\n{question}{suffix}: ").strip() + except (EOFError, KeyboardInterrupt): + print("\nAborted.") + sys.exit(0) + return answer if answer else default + +def get_disc_label(): + r = subprocess.run(["blkid", "-o", "value", "-s", "LABEL", DEVICE], + capture_output=True, text=True, stdin=subprocess.DEVNULL) + label = r.stdout.strip() + if not label: + return "" + # TRANSFORMERS_RISE_OF_THE_BEASTS → Transformers Rise of the Beasts + words = label.replace("_", " ").split() + small = {"of", "the", "a", "an", "and", "in", "on", "at", "to", "for", "with", "from"} + return " ".join( + w.capitalize() if i == 0 or w.lower() not in small else w.lower() + for i, w in enumerate(words) + ) + +def get_disc_year(): + r = subprocess.run(["findmnt", "-o", "TARGET", "--noheadings", "-S", DEVICE], + capture_output=True, text=True, stdin=subprocess.DEVNULL) + mount_point = r.stdout.strip() + if mount_point: + try: + mtime = os.path.getmtime(mount_point) + return str(datetime.datetime.fromtimestamp(mtime).year) + except Exception: + pass + return "" + +print() +print("─" * 40) +print("Setup") +print("─" * 40) + +# Title number +while True: + num = ask("Title number", longest["num"]) + if num in titles_by_num: + selected = titles_by_num[num] + break + print(f" Title {num!r} not found. Valid: {', '.join(t['num'] for t in titles)}") + +# Confirm the selected title +t = selected +audio_str = " / ".join(t["audio"]) if t["audio"] else "—" +subs_str = " ".join(t["subs"]) if t["subs"] else "—" +print(f"\n Selected: title #{t['num']} {fmt_duration(t['duration'])} {t['video']}") +print(f" Audio: {audio_str}") +print(f" Subs: {subs_str}") + +# Movie name + year +default_name = get_disc_label() +default_year = get_disc_year() +movie_name = ask("Movie name", default_name) +year = ask("Year", default_year) + +# Audio / output +audio_langs = ask("Audio languages (comma-separated, order = priority)", "deu,eng") +out_base = ask("Output directory", "/mnt/media/jellyfin/movies") + +# ── Confirmation ────────────────────────────────────────────────────────────── + +full_name = f"{movie_name} ({year})" if year else movie_name +out_dir = os.path.join(out_base, full_name) +out_file = os.path.join(out_dir, f"{full_name}.mkv") + +cmd = [ + "HandBrakeCLI", + "-i", DEVICE, + "-t", num, + "-o", out_file, + "--preset=H.265 MKV 1080p30", + "--audio-lang-list", audio_langs, + "--all-audio", + "--aencoder", "copy", + "--audio-fallback", "aac", + "--all-subtitles", +] + +print() +print("─" * 40) +print("Ready to rip") +print("─" * 40) +print(f" Title: #{num} {fmt_duration(t['duration'])} {t['video']}") +print(f" Movie: {full_name}") +print(f" Output: {out_file}") +print(f" Audio: {audio_langs} (all matching tracks, pass-through → AAC fallback)") +print(f" Subs: all tracks ({len(t['subs'])} found on this title)") +cmd_display = ( + f"HandBrakeCLI \\\n" + f" -i {DEVICE} -t {num} \\\n" + f" -o \"{out_file}\" \\\n" + f" --preset=\"H.265 MKV 1080p30\" \\\n" + f" --audio-lang-list {audio_langs} --all-audio --aencoder copy --audio-fallback aac \\\n" + f" --all-subtitles" +) +print() +print(" Command:") +print(" " + cmd_display) +print() + +try: + answer = input("Begin ripping? [Y/n]: ").strip().lower() +except (EOFError, KeyboardInterrupt): + print("\nAborted.") + sys.exit(0) + +if answer in ("", "y", "yes"): + os.makedirs(out_dir, exist_ok=True) + print() + subprocess.run(cmd) +else: + print("Aborted.") diff --git a/movie-ripping-workflow.md b/movie-ripping-workflow.md new file mode 100644 index 0000000..d04cea0 --- /dev/null +++ b/movie-ripping-workflow.md @@ -0,0 +1,148 @@ +# Movie Ripping Workflow (HandBrakeCLI + MakeMKV → Jellyfin) + +## Tools + +| Tool | Purpose | +|------|---------| +| `HandBrakeCLI` | Rip + encode DVDs to H.265 MKV directly from disc | +| `libdvd-pkg` | Builds libdvdcss to decrypt commercial DVDs | +| `makemkv-bin` / `makemkv-oss` | Decrypt Blu-rays to lossless MKV (required before HandBrake) | +| `disc-scan.py` | Helper script to identify the main feature title number | + +## Install (one-time) + +### DVD support +```bash +sudo apt install -y handbrake-cli libdvd-pkg +sudo dpkg-reconfigure libdvd-pkg # builds libdvdcss from source — follow the prompt +``` + +### Blu-ray support +```bash +sudo add-apt-repository ppa:heyarje/makemkv-beta +sudo apt update +sudo apt install -y makemkv-bin makemkv-oss +``` + +## Encode Settings (used for both DVD and Blu-ray) + +``` +Preset: H.265 MKV 1080p30 +Audio: German + English, all matching tracks, AC3/DTS passed through unchanged +Subs: ALL tracks (captures forced/foreign-language subs — see Subtitle Strategy below) +``` + +--- + +## DVD Workflow + +### One command +```bash +python3 ~/smithpc/disc-scan.py +``` + +The script handles everything interactively: +1. Scans the disc (~20-30 s) and prints a title table with duration, video, audio tracks, and subtitle languages +2. Prompts for title number (default: longest title), movie name (default: from disc label), year (default: from disc filesystem timestamp), audio languages, output directory +3. Shows a full confirmation including the exact HandBrakeCLI command that will run +4. Asks "Begin ripping? [Y/n]" before starting + +Takes ~30-90 min depending on disc. Output is typically 4–8 GB. + +### Manual command (reference / fallback) +```bash +HandBrakeCLI \ + -i /dev/sr0 \ + -t TITLE \ + -o "/mnt/media/jellyfin/movies/Movie Name (Year)/Movie Name (Year).mkv" \ + --preset="H.265 MKV 1080p30" \ + --audio-lang-list deu,eng --all-audio --aencoder copy --audio-fallback aac \ + --all-subtitles +``` + +--- + +## Blu-ray Workflow + +Blu-rays use AACS encryption that HandBrakeCLI cannot decrypt directly. +MakeMKV handles the decryption; HandBrakeCLI handles the encoding. + +### Step 1 — Create temp dir and scan +```bash +mkdir -p ~/makemkv-tmp +makemkvcon info disc:0 +``` +Look for the longest title by duration. Note its title number (zero-indexed in MakeMKV). + +### Step 2 — Decrypt to lossless MKV +Replace `TITLE` with the title number from Step 1 (MakeMKV uses 0-based indexing): + +```bash +TITLE=0 +makemkvcon mkv disc:0 "$TITLE" ~/makemkv-tmp/ +``` +Outputs a file like `~/makemkv-tmp/Title_t00.mkv`. No re-encoding, so this is fast (~10-20 min). + +### Step 3 — Encode with HandBrakeCLI +```bash +MOVIE="Movie Name (Year)" +OUTDIR="/mnt/media/jellyfin/movies/$MOVIE" +INPUT=$(ls ~/makemkv-tmp/*.mkv | head -1) + +mkdir -p "$OUTDIR" + +HandBrakeCLI \ + -i "$INPUT" \ + -o "$OUTDIR/$MOVIE.mkv" \ + --preset="H.265 MKV 1080p30" \ + --audio-lang-list deu,eng --all-audio --aencoder copy \ + --all-subtitles +``` + +### Step 4 — Clean up temp files +```bash +rm -rf ~/makemkv-tmp/ +``` + +--- + +## Jellyfin Naming Convention + +Jellyfin expects: +``` +/mnt/media/jellyfin/movies/ + Movie Name (Year)/ + Movie Name (Year).mkv +``` + +The `mkdir -p "$OUTDIR"` and output path in the commands above follow this convention automatically. + +--- + +## Subtitle Strategy + +`--all-subtitles` copies every subtitle track from the disc as soft subs (not burned in). + +This is critical for films with characters speaking constructed or foreign languages +(e.g., Dothraki in Game of Thrones, Elvish in LotR, alien dialogue). Those subtitles +often live on a separate "forced" track that `--subtitle-lang-list` would miss unless +explicitly included. `--all-subtitles` captures them all; Jellyfin lets you toggle +which track to display at playback time. + +--- + +## Trigger Jellyfin Rescan + +Jellyfin dashboard → Libraries → Movies → "Scan Library Files" + +--- + +## Troubleshooting + +- **`[warning] decavsub: track 0, invalid PTS` during encode:** Normal and harmless. VOBSUB subtitle packets on DVDs frequently have malformed presentation timestamps. HandBrake logs them but still copies the subtitle tracks correctly. +- **HandBrakeCLI: "CSS error" or no video:** libdvdcss isn't built yet — run `sudo dpkg-reconfigure libdvd-pkg` again and confirm at the prompt. +- **disc-scan.py shows no titles:** Check disc is inserted and spun up (`lsblk`). Try `lsdvd /dev/sr0` to confirm the drive sees it. +- **MakeMKV expired trial:** MakeMKV shows a nag but still works during the trial. For long-term use, purchase a license or build from source. +- **HandBrakeCLI audio: "copy not supported":** The preset or container rejected a codec. Add `--audio-fallback aac` to fall back to AAC instead of failing silently. +- **Wrong title ripped (bonus feature, trailer):** Re-run `disc-scan.py` and compare durations. Main feature is almost always the longest. +- **Blu-ray: MakeMKV "AACS error":** The disc uses a newer AACS version not yet in MakeMKV's key database — update MakeMKV to the latest beta via the PPA.