Initial commit

documentation/backend/database.md

@@ -0,0 +1,123 @@
+# Database Schema
+
+**Status:** ✅ Implemented
+
+PostgreSQL database storing users, audiobooks, requests, downloads, configuration, and jobs.
+
+**Setup:** Automatically created on container startup via `prisma db push` (syncs schema directly to DB without migration files).
+
+## Tables
+
+### Users
+- `id` (UUID PK), `plex_id` (unique), `plex_username`, `plex_email`, `role` ('user'|'admin')
+- `is_setup_admin` (bool, default false) - First admin created during setup, role protected from changes
+- `avatar_url`, `auth_token` (encrypted), `created_at`, `updated_at`, `last_login_at`
+- **Plex Home profile tracking:**
+  - `plex_home_user_id` (string, nullable) - Profile ID from Plex Home (null = main account, set = home profile)
+- **BookDate per-user preferences:**
+  - `bookdate_library_scope` ('full'|'rated', default 'full') - Library scope for recommendations
+  - `bookdate_custom_prompt` (text, optional, max 1000 chars) - Custom preferences for AI
+  - `bookdate_onboarding_complete` (bool, default false) - Whether user has completed BookDate onboarding
+- Indexes: `plex_id`, `role`
+
+### Audible_Cache
+- `id` (UUID PK), `asin` (unique, Audible ID), `title`, `author`, `narrator`, `description`
+- `cover_art_url`, `cached_cover_path` (local thumbnail path), `duration_minutes`, `release_date`, `rating`, `genres` (JSONB)
+- **Discovery:** `is_popular` (bool), `is_new_release` (bool), `popular_rank`, `new_release_rank`
+- `last_synced_at`, `created_at`, `updated_at`
+- Indexes: `asin`, `title`, `author`, `is_popular`, `is_new_release`, `popular_rank`, `new_release_rank`
+- **Purpose:** Cached Audible metadata (popular/new releases), thumbnails stored locally in `/app/cache/thumbnails`
+
+### Audiobooks
+- `id` (UUID PK), `audible_asin` (nullable), `title`, `author`, `narrator`, `description`
+- `cover_art_url`, `file_path`, `file_format`, `file_size_bytes`
+- `plex_guid` (nullable), `plex_library_id` (nullable)
+- `status` ('requested'|'downloading'|'processing'|'completed'|'failed')
+- `created_at`, `updated_at`, `completed_at`
+- Indexes: `audible_asin`, `plex_guid`, `title`, `author`, `status`
+- **Purpose:** User-requested audiobooks only (created on request)
+
+### Requests
+- `id` (UUID PK), `user_id` (FK), `audiobook_id` (FK)
+- `status` ('pending'|'searching'|'downloading'|'processing'|'downloaded'|'available'|'failed'|'cancelled'|'awaiting_search'|'awaiting_import'|'warn')
+  - Flow: pending → searching → downloading → processing → downloaded → available (when matched in Plex)
+- `progress` (0-100), `priority`, `error_message`
+- `search_attempts`, `download_attempts`, `import_attempts`, `max_import_retries` (default 5)
+- `last_search_at`, `last_import_at`, `created_at`, `updated_at`, `completed_at`
+- Unique: `(user_id, audiobook_id)`
+- Indexes: `user_id`, `audiobook_id`, `status`, `created_at DESC`
+
+### Download_History
+- `id` (UUID PK), `request_id` (FK), `indexer_name`, `torrent_name`, `torrent_hash`
+- `torrent_size_bytes`, `magnet_link`, `torrent_url`, `seeders`, `leechers`
+- `quality_score`, `selected` (bool), `download_client`, `download_client_id`
+- `download_status` ('queued'|'downloading'|'completed'|'failed'|'stalled')
+- `download_error`, `started_at`, `completed_at`, `created_at`
+- Indexes: `request_id`, `selected`, `created_at DESC`
+
+### Configuration
+- `id` (UUID PK), `key` (unique), `value`, `encrypted` (bool), `category`, `description`
+- `created_at`, `updated_at`
+- Indexes: `key`, `category`
+- Example keys: `plex.server_url`, `plex.auth_token`, `indexer.prowlarr_url`, `download_client.qbittorrent_password`, `paths.downloads`, `setup.completed`
+
+### Jobs
+- `id` (UUID PK), `bull_job_id`, `request_id` (FK nullable)
+- `type` ('search_indexers'|'monitor_download'|'organize_files'|'scan_plex'|'match_plex'|'plex_library_scan'|'plex_recently_added_check'|'audible_refresh'|'retry_missing_torrents'|'retry_failed_imports'|'cleanup_seeded_torrents'|'monitor_rss_feeds')
+- `status` ('pending'|'active'|'completed'|'failed'|'delayed'|'stuck')
+- `priority`, `attempts`, `max_attempts` (default 3)
+- `payload` (JSONB), `result` (JSONB), `error_message`, `stack_trace`
+- `started_at`, `completed_at`, `created_at`, `updated_at`
+- Indexes: `request_id`, `type`, `status`, `created_at DESC`
+
+### Job_Events
+- `id` (UUID PK), `job_id` (FK → Jobs, CASCADE delete)
+- `level` ('info'|'warn'|'error')
+- `context` (processor name: OrganizeFiles, FileOrganizer, MonitorDownload, etc.)
+- `message` (event description)
+- `metadata` (JSONB, optional structured data)
+- `created_at` (timestamp)
+- Indexes: `job_id`, `created_at`
+- **Purpose:** Store detailed event logs for job operations (shown in admin logs UI)
+
+## Relationships
+
+- User → Requests (1:many)
+- Audiobook → Requests (1:many)
+- Request → Download History (1:many)
+- Request → Jobs (1:many, nullable)
+- Job → Job Events (1:many, CASCADE delete)
+
+## Setup Strategy
+
+**Approach:** Schema sync via `prisma db push`
+- Prisma schema is source of truth
+- On startup: sync schema → database
+- Idempotent (safe to run multiple times)
+- No migration files needed
+- Generates Prisma client after sync
+
+## ORM: Prisma 6.x
+
+- Type-safe queries
+- Auto-generated types
+- Connection pooling
+- Client output: `src/generated/prisma`
+
+## Security
+
+**Encryption at Rest (AES-256):**
+- User auth tokens
+- API keys/passwords in Configuration
+- Download client credentials
+
+**SQL Injection:** Parameterized queries only via ORM
+
+**Access Control:** Row-level (users see only their requests), admins have full access
+
+## Tech Stack
+
+- PostgreSQL 16+
+- Prisma 6.x
+- `prisma db push` (schema sync)
+- Node.js crypto (encryption)

documentation/backend/middleware.md

@@ -0,0 +1,34 @@
+# Setup Middleware
+
+**Status:** ✅ Implemented | Edge middleware enforcing setup wizard completion
+
+## Overview
+Edge runtime middleware intercepts all non-API requests to gate access until the setup wizard finishes. It uses a lightweight API check so Prisma is never invoked inside the Edge sandbox.
+
+## Key Details
+- **Location:** `src/middleware.ts`
+- Skips: `/api/*`, `/_next/*`, `/static/*`, any path containing `.` (static assets)
+- Fetches `/api/setup/status` with header `x-middleware-request: true`
+- Redirects:
+  - Setup incomplete → `/setup` (unless already there)
+  - Setup complete → `/` when user visits `/setup`
+- Fetch origin priority:
+  1. `SETUP_CHECK_BASE_URL` env (optional override, e.g. `http://rmab-internal:3030`)
+  2. Incoming request origin (`request.nextUrl.origin`)
+  3. Loopback fallback `http://127.0.0.1:${PORT|3030}`
+- On repeated failures the middleware logs once per request but allows traffic to avoid blocking users
+
+## API/Interfaces
+```
+GET /api/setup/status
+Headers: x-middleware-request: true
+Response: { setupComplete: boolean }
+```
+
+## Critical Issues
+- Reverse proxies that terminate TLS on a hostname unreachable from inside the container should set `SETUP_CHECK_BASE_URL` to an internal origin (or rely on the loopback fallback if port exposure allows it).
+- Ensure the fallback port stays in sync with the app server port (`PORT` env, defaults to 3030 in Docker images).
+
+## Related
+- [documentation/setup-wizard.md](../setup-wizard.md)
+- [documentation/deployment/unified.md](../deployment/unified.md)

documentation/backend/services/auth.md

@@ -0,0 +1,174 @@
+# Authentication Service
+
+**Status:** ✅ Implemented | Plex OAuth + Plex Home profile support + JWT sessions + RBAC
+
+Handles authentication and authorization: Plex OAuth integration with Plex Home profile support, JWT session management, role-based access control.
+
+## Authentication: Plex OAuth
+
+- No password management needed
+- Users already have Plex accounts
+- Seamless integration
+- Automatic profile pictures/metadata
+- **Plex Home support:** Each profile = separate user
+
+## Session Management: JWT Tokens
+
+- Stateless authentication
+- No server-side session storage
+- Easy horizontal scaling
+- Includes user claims (role, permissions)
+
+## Access Control: RBAC
+
+**Roles:**
+1. **user** - Request audiobooks, view own requests, search
+2. **admin** - Full system access (settings, users, all requests)
+
+## OAuth Flow (with Plex Home Support)
+
+1. User clicks "Login with Plex"
+2. Redirect to Plex OAuth
+3. User authorizes app
+4. Redirect back with PIN code
+5. Exchange code for main account token
+6. Get main account user info
+7. **Verify user has access to configured Plex server** (uses stored machineIdentifier from config)
+8. **Check for Plex Home profiles:**
+   - If profiles exist → Redirect to profile selection page
+   - If no profiles → Continue with main account
+9. **Profile Selection (if applicable):**
+   - User selects profile from grid
+   - Enter PIN if profile is protected
+   - Switch to profile, get profile's auth token
+10. Create/update user in DB (with profile details)
+11. Generate JWT
+12. Return JWT to client
+13. Client includes JWT in subsequent requests
+
+## OAuth Endpoints
+
+**GET /api/auth/plex/login** - Redirect to Plex OAuth
+**GET /api/auth/plex/callback?pinId=...** - Exchange PIN, check for profiles, return JWT or redirect to profile selection
+**GET /api/auth/plex/home-users** - Get list of Plex Home profiles (requires X-Plex-Token header)
+**POST /api/auth/plex/switch-profile** - Switch to selected profile and complete authentication
+**POST /api/auth/refresh** - Get new access token (refresh token in header)
+**POST /api/auth/logout** - Clear client-side token
+**GET /api/auth/me** - Get current user (JWT in header)
+
+## JWT Structure
+
+**Access Token (1hr):**
+```json
+{
+  "sub": "user-uuid",
+  "plexId": "plex-user-id",
+  "username": "john_doe",
+  "role": "admin",
+  "iat": 1234567890,
+  "exp": 1234571490
+}
+```
+
+**Refresh Token (7d):**
+```json
+{
+  "sub": "user-uuid",
+  "type": "refresh",
+  "iat": 1234567890,
+  "exp": 1234971490
+}
+```
+
+**Storage:**
+- Access: HTTP-only cookie + localStorage
+- Refresh: HTTP-only secure cookie only
+- SameSite=Strict (CSRF protection)
+
+## Middleware
+
+**requireAuth()** - Verifies JWT exists/valid, adds user to request, returns 401 if invalid
+**requireAdmin()** - Checks `user.role === 'admin'`, returns 403 if not, chains after requireAuth
+
+## First User Setup
+
+- First user created during setup automatically promoted to admin
+- Marked as "setup admin" with `isSetupAdmin=true` flag
+- Setup admin role is **protected** - cannot be changed to prevent lockout
+- Ensures someone always has admin access after fresh install
+- Subsequent users default to 'user' role
+- Local admin uses username/password authentication (stored in `authToken` field as bcrypt hash)
+- `plexId` format: `local-{username}` for local admin accounts
+
+## Local Admin Authentication
+
+**Local Admin (Setup Admin):**
+- Created during setup wizard (step 2)
+- Username/password authentication (separate from Plex OAuth)
+- Password hashed with bcrypt (10 rounds) and stored in `authToken` field
+- Login: POST `/api/auth/admin/login` with username/password
+- Identified by: `isSetupAdmin=true` AND `plexId` starts with `local-`
+
+**Password Management:**
+- POST `/api/admin/settings/change-password` - Change local admin password
+- Requires: current password, new password (min 8 chars), confirmation
+- Security: Only accessible to local admin (verified via `requireLocalAdmin` middleware)
+- Validates current password before allowing change
+
+## Plex Home Profile Support
+
+**Overview:**
+- Plex Home accounts can have multiple profiles (managed users, family members)
+- Each profile has its own library ratings, watch history, restrictions
+- **Architecture:** Each profile = separate user in ReadMeABook system
+
+**Profile Selection Flow:**
+1. User authenticates with main Plex account
+2. System fetches list of profiles via `GET https://plex.tv/api/home/users`
+3. If profiles exist → Show profile selection page (`/auth/select-profile`)
+4. User selects their profile (enters PIN if protected)
+5. System switches to profile via `POST https://plex.tv/api/home/users/{id}/switch`
+6. Profile's auth token is stored (encrypted)
+7. User record created with profile's details
+
+**Profile Data Storage:**
+- `plexId`: Profile's unique ID (not main account ID)
+- `plexUsername`: Profile's friendlyName
+- `authToken`: Profile's auth token (encrypted)
+- `avatarUrl`: Profile's avatar
+- `plexHomeUserId`: Profile ID for reference (null = main account, set = home profile)
+
+**User Isolation:**
+- Each profile is a completely separate user
+- Separate requests, separate BookDate recommendations, separate ratings
+- Admin sees all profiles as independent users (no grouping)
+- Profile switching = logout and login again
+
+**Profile Protection:**
+- Protected profiles require PIN on login
+- PIN validated by Plex API during switch
+- PIN not stored (only needed at login)
+
+**Benefits:**
+- Accurate request attribution ("Requested by Dad" vs "Requested by Kids")
+- Personalized BookDate recommendations based on each profile's ratings
+- Separate "My Requests" per family member
+- Accurate logs and analytics
+
+## Security
+
+- Never log tokens
+- HTTPS only in production
+- Short access token expiry (1hr)
+- Optional refresh token rotation
+- Track valid tokens for revocation
+- **Server access verification**: Uses stored `machineIdentifier` from config (no API call needed)
+  - Only users with access to the configured Plex server can authenticate
+  - Prevents any Plex user from accessing the instance
+  - machineIdentifier stored during setup/settings configuration (architectural optimization)
+
+## Tech Stack
+
+- Custom Plex OAuth (direct API)
+- jsonwebtoken (npm)
+- Node.js crypto

documentation/backend/services/config.md

@@ -0,0 +1,117 @@
+# Configuration Service
+
+**Status:** ❌ Design Phase
+
+Manages application configuration with secure storage, encryption for sensitive values, clean API for read/write.
+
+## Storage: Database-Backed
+
+- Centralized management
+- Update via web UI without restart
+- Version history and audit trail
+- Encryption at rest for sensitive values
+- Survives container restarts
+
+## Encryption: AES-256-GCM
+
+- Industry standard symmetric encryption
+- Authenticated encryption (prevents tampering)
+- Built into Node.js crypto module
+- Encryption key: 32-byte random (env var `CONFIG_ENCRYPTION_KEY`)
+- Format: `iv:authTag:encryptedData` (base64)
+
+## Configuration Model
+
+- **Key** - Unique identifier (e.g., `plex.server_url`)
+- **Value** - Setting (string, JSON for complex types)
+- **Encrypted** - Boolean flag
+- **Category** - Logical grouping
+- **Description** - Human-readable explanation
+
+## Key Naming
+
+```
+{category}.{setting_name}
+
+Examples:
+plex.server_url
+plex.auth_token (encrypted)
+indexer.prowlarr.url
+indexer.prowlarr.api_key (encrypted)
+download_client.qbittorrent.url
+download_client.qbittorrent.password (encrypted)
+paths.downloads
+paths.media_library
+automation.check_interval_seconds
+system.setup_completed
+```
+
+## Service API
+
+```typescript
+interface ConfigService {
+  get(key: string): Promise<string | null>;
+  getOrDefault(key: string, defaultValue: string): Promise<string>;
+  getBoolean(key: string): Promise<boolean>;
+  getNumber(key: string): Promise<number>;
+  getJSON<T>(key: string): Promise<T | null>;
+
+  set(key: string, value: string, encrypted?: boolean): Promise<void>;
+  setMany(items: Array<{key, value, encrypted?}>): Promise<void>;
+
+  getCategory(category: string): Promise<Record<string, string>>;
+
+  // Helpers
+  getPlexConfig(): Promise<PlexConfig>;
+  getIndexerConfig(): Promise<IndexerConfig>;
+  getDownloadClientConfig(): Promise<DownloadClientConfig>;
+
+  isSetupCompleted(): Promise<boolean>;
+  testConnection(category: string): Promise<{success: boolean, message: string}>;
+}
+```
+
+## API Endpoints
+
+**GET /api/config/:category** - Get all config for category (admin auth, passwords masked)
+
+**PUT /api/config** - Update multiple values (admin auth)
+```json
+{
+  "updates": [
+    {"key": "plex.server_url", "value": "http://...", "encrypted": false},
+    {"key": "plex.auth_token", "value": "token", "encrypted": true}
+  ]
+}
+```
+
+**POST /api/config/test/:category** - Test connection (admin auth)
+
+**GET /api/config/setup-status** - Check setup completion (no auth)
+
+## Defaults
+
+```typescript
+const CONFIG_DEFAULTS = {
+  'automation.check_interval_seconds': '60',
+  'automation.max_search_attempts': '3',
+  'automation.preferred_format': 'm4b',
+  'system.setup_completed': 'false',
+  'system.log_level': 'info',
+  'paths.downloads': '/downloads',
+  'paths.media_library': '/media'
+};
+```
+
+## Required for App Function
+
+**Plex:** `server_url`, `library_id`, `auth_token`
+**Indexer:** `type`, `{type}.url`, `{type}.api_key`
+**Download Client:** `type`, `{type}.url`, credentials
+**Paths:** `downloads`, `media_library` (writable)
+
+## Tech Stack
+
+- Node.js crypto (encryption)
+- PostgreSQL (configuration table)
+- Zod (validation)

documentation/backend/services/environment.md

@@ -0,0 +1,199 @@
+# Environment Variables
+
+**Status:** ✅ Implemented | Centralized URL handling via getBaseUrl() utility
+
+Defines all environment variables used by ReadMeABook, configuration priority, and troubleshooting guide.
+
+## Public URL Configuration (OAuth Callbacks)
+
+**Critical for OAuth:** Plex OAuth and OIDC authentication require correct redirect URIs.
+
+**Priority Order:**
+1. `PUBLIC_URL` - **Primary** (documented standard)
+2. `NEXTAUTH_URL` - Legacy fallback (backward compatibility)
+3. `BASE_URL` - Alternative fallback
+4. `http://localhost:3030` - Development default
+
+**Format Requirements:**
+- Must start with `http://` or `https://`
+- No trailing slash (automatically normalized)
+- Must be publicly accessible for OAuth callbacks
+- Example: `https://readmeabook.example.com`
+
+**Docker Compose:**
+```yaml
+environment:
+  PUBLIC_URL: "https://readmeabook.example.com"
+```
+
+**Implementation:** `src/lib/utils/url.ts` → `getBaseUrl()`
+
+**Used By:**
+- OIDC OAuth redirect_uri: `{PUBLIC_URL}/api/auth/oidc/callback`
+- Plex OAuth redirect_uri: `{PUBLIC_URL}/api/auth/plex/callback`
+- Login error redirects: `{PUBLIC_URL}/login?error=...`
+
+## Database Configuration
+
+**Required:**
+- `DATABASE_URL` - PostgreSQL connection string
+  - **Auto-generated** by entrypoint in unified container
+  - Format: `postgresql://{user}:{password}@{host}:{port}/{database}`
+  - Example: `postgresql://readmeabook:password@localhost:5432/readmeabook`
+
+**PostgreSQL Settings (Unified Container):**
+- `POSTGRES_USER` - Default: `readmeabook`
+- `POSTGRES_PASSWORD` - Auto-generated on first run if not set
+- `POSTGRES_DB` - Default: `readmeabook`
+
+## Security & Secrets
+
+**Auto-generated on first run (Unified Container):**
+- `JWT_SECRET` - JWT access token signing key
+- `JWT_REFRESH_SECRET` - JWT refresh token signing key
+- `CONFIG_ENCRYPTION_KEY` - Config field encryption key (Plex tokens, etc.)
+- `POSTGRES_PASSWORD` - PostgreSQL password
+
+**Manual Override:** Set in docker-compose.yml before first run to use custom secrets.
+
+## File Ownership (Unified Container)
+
+**User/Group ID Mapping:**
+- `PUID` - Default: 1000 (your host user ID)
+- `PGID` - Default: 1000 (your host group ID)
+
+**How It Works:**
+- PostgreSQL: UID 103, GID={PGID}
+- Node/Redis: Fully remapped to PUID:PGID
+- See: documentation/deployment/unified.md
+
+## Plex Configuration
+
+**Optional Overrides:**
+- `PLEX_CLIENT_IDENTIFIER` - Default: auto-generated UUID
+- `PLEX_PRODUCT_NAME` - Default: `ReadMeABook`
+- `PLEX_OAUTH_CALLBACK_URL` - Custom OAuth callback (overrides PUBLIC_URL)
+
+## Logging
+
+**Optional:**
+- `LOG_LEVEL` - Default: `info`
+  - Values: `debug`, `info`, `warn`, `error`
+  - `debug` logs base URL resolution source
+
+**Debug Example:**
+```
+[URL Utility] Using base URL from PUBLIC_URL: https://example.com
+```
+
+## Setup Middleware
+
+**Internal Override:**
+- `SETUP_CHECK_BASE_URL` - Override base URL for setup status check
+  - Use case: Reverse proxies with TLS termination
+  - Default: Tries request origin, then loopback
+  - See: documentation/backend/middleware.md
+
+## Troubleshooting
+
+### Issue: OAuth Redirects to Localhost
+
+**Symptoms:**
+- OIDC/Plex OAuth redirects to `http://localhost:3030/api/auth/...`
+- Authentik/Identity Provider shows `localhost` redirect URI
+- "Redirect URI Error" or "Mismatching redirection URI"
+
+**Cause:** `PUBLIC_URL` not set (defaulting to localhost)
+
+**Fix:**
+```yaml
+# docker-compose.yml
+environment:
+  PUBLIC_URL: "https://your-actual-domain.com"  # No trailing slash
+```
+
+**Restart container after change.**
+
+### Issue: Invalid Redirect URI Format
+
+**Symptoms:**
+- Warning: `Invalid base URL format`
+- OAuth fails with malformed URL
+
+**Cause:** PUBLIC_URL missing protocol or has invalid format
+
+**Fix:**
+- ✅ Correct: `https://example.com`
+- ❌ Wrong: `example.com` (missing protocol)
+- ❌ Wrong: `https://example.com/` (trailing slash, auto-normalized but avoid)
+
+### Issue: Production Using Localhost
+
+**Symptoms:**
+- Warning: `Using localhost URL in production`
+- OAuth fails from external clients
+
+**Cause:** NODE_ENV=production but PUBLIC_URL not set
+
+**Fix:** Always set PUBLIC_URL in production deployments.
+
+### Issue: checks.state argument is missing (OIDC)
+
+**Symptoms:**
+- Error in URL after OIDC login: `error=TypeError: checks.state argument is missing`
+- Login redirects back to login page after Authentik authentication
+
+**Cause:** Missing state parameter in openid-client callback checks (fixed in latest version)
+
+**Fix:** Update to latest version with state parameter fix
+
+### Issue: OIDC login succeeds but redirects back to login page
+
+**Symptoms:**
+- OIDC authentication completes in Authentik
+- Redirect back to ReadMeABook succeeds
+- URL shows `/login?redirect=%2F`
+- Not actually logged in, no auth cookies visible
+
+**Cause:** httpOnly cookies prevent JavaScript from reading tokens (fixed in latest version)
+
+**Fix:**
+- Update to latest version
+- Callback now uses URL hash + accessible cookies (matches Plex OAuth pattern)
+- Tokens properly stored in localStorage
+
+**Authentik Configuration Requirements:**
+1. Go to Application/Provider → Scopes
+2. Add: `openid`, `profile`, `email`, `groups`
+3. Redirect URI: `https://your-domain.com/api/auth/oidc/callback`
+4. Save and retry login
+
+## Environment Variable Reference
+
+| Variable | Required | Default | Description |
+|----------|----------|---------|-------------|
+| `PUBLIC_URL` | Prod | localhost:3030 | Public URL for OAuth callbacks |
+| `NEXTAUTH_URL` | No | - | Legacy fallback for PUBLIC_URL |
+| `BASE_URL` | No | - | Alternative fallback for PUBLIC_URL |
+| `DATABASE_URL` | Yes | Auto-generated | PostgreSQL connection string |
+| `POSTGRES_USER` | No | readmeabook | PostgreSQL username |
+| `POSTGRES_PASSWORD` | No | Auto-generated | PostgreSQL password |
+| `POSTGRES_DB` | No | readmeabook | PostgreSQL database name |
+| `JWT_SECRET` | No | Auto-generated | JWT signing secret |
+| `JWT_REFRESH_SECRET` | No | Auto-generated | Refresh token secret |
+| `CONFIG_ENCRYPTION_KEY` | No | Auto-generated | Config encryption key |
+| `PUID` | No | 1000 | Host user ID for file ownership |
+| `PGID` | No | 1000 | Host group ID for file ownership |
+| `PLEX_CLIENT_IDENTIFIER` | No | Auto-generated | Plex API client ID |
+| `PLEX_PRODUCT_NAME` | No | ReadMeABook | Plex product name |
+| `PLEX_OAUTH_CALLBACK_URL` | No | - | Custom Plex OAuth callback |
+| `LOG_LEVEL` | No | info | Logging verbosity |
+| `SETUP_CHECK_BASE_URL` | No | - | Setup middleware override |
+| `NODE_ENV` | No | production | Environment mode |
+
+## Related
+
+- OAuth Implementation: documentation/backend/services/auth.md
+- OIDC Configuration: documentation/features/audiobookshelf-integration.md
+- Deployment: documentation/deployment/unified.md
+- Setup Middleware: documentation/backend/middleware.md

documentation/backend/services/jobs.md

@@ -0,0 +1,172 @@
+# Background Job System
+
+**Status:** ✅ Implemented
+
+Manages background job queue using Bull (Redis-backed) for async tasks: searching indexers, monitoring downloads, organizing files, scanning Plex.
+
+## Detailed Event Logging
+
+- **JobEvent table:** Stores timestamped event logs for all job operations
+- **JobLogger utility:** (`src/lib/utils/job-logger.ts`) provides structured logging
+- **Levels:** info, warn, error
+- **Context:** Processor name (e.g., OrganizeFiles, FileOrganizer, MonitorDownload)
+- **Metadata:** Optional JSON data for structured details
+- **UI:** Admin logs page shows detailed event logs, job results, and errors
+
+## Queue System: Bull + Redis
+
+- Redis-backed for persistence
+- Retry: 3 attempts, exponential backoff (2s, 4s, 8s)
+- Priority: High (10), Medium (5), Low (1)
+- Concurrency: 3 per job type
+- Jobs survive app restarts
+- Remove on complete: keep last 100
+- Remove on fail: keep last 200
+- MaxListeners: 20 on both Redis client and Bull queue (accommodates 12 job processors)
+
+## Job Types
+
+1. **search_indexers** - Search Prowlarr for torrents
+2. **monitor_download** - Poll progress (10s intervals)
+3. **organize_files** - Move to media library, set status to 'downloaded'
+4. **scan_plex** - Full scan of Plex library, match 'downloaded' requests
+5. **plex_recently_added_check** - Lightweight polling of recently added items (top 10)
+6. **match_plex** - Fuzzy match to Plex item (deprecated - now handled by scan_plex)
+
+## Special Behaviors
+
+**monitor_download:**
+- 3s initial delay before first check (avoids race condition with qBittorrent processing)
+- Retry logic: 3 attempts with exponential backoff (500ms, 1s, 2s) for getTorrent failures
+- Transient error handling: "torrent not found" errors don't mark request as failed during retries
+- Request stays in "downloading" status during all retry attempts
+- Only marks request as "failed" after all Bull retries (3 attempts) exhausted
+- 10s delay between checks (prevents excessive logging)
+- Only logs progress at 5% intervals or first 5%
+- Auto-reschedules until complete/failed
+
+**search_indexers:**
+- No torrents found → 'awaiting_search' status (not failed)
+- Allows automatic retry via scheduled job
+
+**organize_files:**
+- No audiobook files found → 'awaiting_import' status
+- Tracks `import_attempts` (max 5 default)
+- After max retries → 'warn' status for manual intervention
+- Success → 'downloaded' status (green, waiting for Plex scan)
+- No longer triggers immediate match_plex job
+
+**scan_plex:**
+- Scans Plex library and populates plex_library table
+- After scan, checks for requests with status 'downloaded'
+- Fuzzy matches downloaded requests against Plex library (70% threshold)
+- Matched requests → 'available' status with plexGuid linked
+
+## Job Payloads
+
+All payloads now include `jobId` (database job ID) automatically added by the job queue service.
+
+```typescript
+// search_indexers
+{jobId: string, requestId: string, audiobook: {id, title, author}}
+
+// monitor_download
+{jobId: string, requestId: string, downloadHistoryId: string, downloadClientId: string, downloadClient: 'qbittorrent'|'transmission'}
+
+// organize_files
+{jobId: string, requestId: string, audiobookId: string, downloadPath: string, targetPath: string}
+
+// scan_plex
+{jobId: string, libraryId: string, partial?: boolean, path?: string}
+
+// match_plex
+{jobId: string, requestId: string, audiobookId: string, title: string, author: string}
+```
+
+## Using JobLogger in Processors
+
+```typescript
+import { createJobLogger } from '../utils/job-logger';
+
+export async function processOrganizeFiles(payload: OrganizeFilesPayload) {
+  const { jobId, requestId, audiobookId } = payload;
+
+  // Create logger
+  const logger = jobId ? createJobLogger(jobId, 'OrganizeFiles') : null;
+
+  // Log events
+  await logger?.info('Processing request');
+  await logger?.warn('Warning message', { metadata: 'optional' });
+  await logger?.error('Error occurred');
+
+  // Pass to utilities
+  const organizer = getFileOrganizer();
+  await organizer.organize(path, metadata,
+    logger ? { jobId, context: 'FileOrganizer' } : undefined
+  );
+}
+```
+
+## Scheduled Job Tracking
+
+**Timer-triggered scheduled jobs** automatically:
+- Create Job records in database (via `ensureJobRecord()`)
+- Update `lastRun` timestamp in `scheduled_jobs` table
+- Generate JobEvent logs with full context
+- Display in system logs page
+
+**Manual-triggered jobs** (via "Trigger Now" button):
+- Go through `triggerJobNow()` → job queue methods → `addJob()`
+- Update `lastRun` timestamp in scheduler service
+- Create Job records with full tracking
+
+## Event Handling
+
+```typescript
+queue.on('completed', async (job, result) => {
+  await updateJobStatus(job.id, 'completed', result);
+});
+
+queue.on('failed', async (job, error) => {
+  await updateJobStatus(job.id, 'failed', null, error.message);
+});
+
+queue.on('stalled', async (job) => {
+  await updateJobStatus(job.id, 'stalled');
+});
+```
+
+## Concurrency Settings
+
+- **search_indexers:** 3 (avoid overwhelming indexers)
+- **monitor_download:** 5 (lightweight API calls)
+- **organize_files:** 2 (I/O intensive)
+- **scan_plex:** 1 (only one scan at a time)
+- **match_plex:** 3 (CPU bound)
+
+## Fixed Issues ✅
+
+- ✅ Monitor job logging excessively (~500x/s) → 10s delay
+- ✅ No retry for missing torrents → 'awaiting_search' status
+- ✅ No retry for failed imports → 'awaiting_import' + max retries
+- ✅ MaxListenersExceededWarning → increased maxListeners to 20 on both Redis client and Bull queue
+- ✅ Race condition causing "error" status on new downloads → 3s initial delay + retry with exponential backoff
+- ✅ Transient failures marking requests as "failed" prematurely → Distinguish transient vs permanent errors, only mark failed after all retries exhausted
+- ✅ Plex search error (400) immediately after file organization → Changed workflow: organize_files sets 'downloaded' status, scan_plex job handles matching during scheduled scans
+- ✅ System logs page incomplete and missing detailed events → Added JobEvent table, JobLogger utility, comprehensive event logging with timestamps and metadata
+- ✅ Scheduled jobs triggered by timer not appearing in system logs → Added ensureJobRecord() to create Job records for timer-triggered scheduled jobs
+- ✅ Scheduled jobs triggered by timer not updating lastRun timestamp → ensureJobRecord() now updates lastRun for timer-triggered jobs
+
+## API Endpoints
+
+**GET /api/admin/job-status/:id**
+- Get execution status of a specific job by database job ID
+- Returns: job status (pending, active, completed, failed, stuck)
+- Used by setup wizard to poll job completion
+- Requires admin auth
+
+## Tech Stack
+
+- Bull (npm)
+- Redis (ioredis)
+- PostgreSQL (jobs table for history)

documentation/backend/services/scheduler.md

@@ -0,0 +1,170 @@
+# Recurring Jobs Scheduler
+
+**Status:** ✅ Implemented
+
+Manages recurring/scheduled jobs providing automated tasks (Plex scans, Audible refresh) with scheduled (cron) execution and manual triggering.
+
+## Recent Updates
+
+- Config validation before job execution
+- Audible refresh persists to database
+- Enhanced error handling with clear messages
+- Schedule editing UI with toast notifications
+- Human-friendly schedule descriptions and editor (preset/custom/advanced modes)
+- Real-time cron expression preview
+
+## Scheduled Jobs
+
+1. **plex_library_scan** - Default: every 6 hours, full library scan, disabled by default (enable after setup)
+2. **plex_recently_added_check** - Default: every 5 minutes, lightweight polling of top 10 recently added items, enabled by default
+3. **audible_refresh** - Default: daily midnight, fetches 200 popular + 200 new releases, stores with rankings, disabled by default
+4. **retry_missing_torrents** - Default: daily midnight, re-searches 'awaiting_search' status (limit 50), enabled by default
+5. **retry_failed_imports** - Default: every 6 hours, re-attempts 'awaiting_import' status (limit 50), enabled by default
+6. **cleanup_seeded_torrents** - Default: every 30 mins, deletes torrents after seeding requirements met, respects `seeding_time_minutes` config (0 = never), enabled by default
+7. **monitor_rss_feeds** - Default: every 15 mins, checks RSS feeds from enabled indexers, matches against 'awaiting_search' requests (limit 100), triggers search jobs for matches, enabled by default
+
+## Architecture: Bull + Cron
+
+- Repeatable jobs with cron expressions (Bull's built-in scheduler)
+- Manual trigger capability
+- Job persistence and retry logic
+- Admin UI management
+- Automatic scheduling/unscheduling when jobs enabled/disabled
+- Schedule updates handled by unscheduling old job and scheduling new one
+
+## Human-Friendly Scheduling UI
+
+**Three Modes:**
+1. **Common Schedules** - Preset options (every 15min, hourly, daily, weekly, monthly)
+2. **Custom Schedule** - Visual builder with dropdowns for minutes/hours/daily/weekly/monthly
+3. **Advanced (Cron)** - Raw cron expression for power users
+
+**Features:**
+- Human-readable display: "Every 6 hours" instead of "0 */6 * * *"
+- Real-time preview of cron expressions
+- Visual schedule builder (no cron knowledge required)
+- Cron validation before saving
+- Shows both human text and cron expression in job list
+
+**Utility Functions** (`src/lib/utils/cron.ts`):
+- `cronToHuman(cron)` - Converts cron to readable text
+- `customScheduleToCron(schedule)` - Builds cron from visual inputs (auto-converts 24+ hour intervals to daily)
+- `cronToCustomSchedule(cron)` - Parses cron to visual inputs
+- `isValidCron(cron)` - Validates cron expression
+
+## Cron Expressions
+
+```
+* * * * *
+│ │ │ │ └─ day of week (0-7)
+│ │ │ └─── month (1-12)
+│ │ └───── day of month (1-31)
+│ └─────── hour (0-23)
+└───────── minute (0-59)
+```
+
+**Examples:**
+- `0 */6 * * *` - Every 6 hours
+- `0 0 * * *` - Daily midnight
+- `*/30 * * * *` - Every 30 mins
+
+## API Endpoints
+
+**GET /api/admin/jobs** - Get all scheduled jobs (admin auth)
+
+**POST /api/admin/jobs** - Create job (admin auth)
+```json
+{
+  "name": "Daily Audible Refresh",
+  "type": "audible_refresh",
+  "schedule": "0 0 * * *",
+  "enabled": true
+}
+```
+
+**PUT /api/admin/jobs/:id** - Update job (admin auth)
+
+**DELETE /api/admin/jobs/:id** - Delete job (admin auth)
+
+**POST /api/admin/jobs/:id/trigger** - Manually trigger job (admin auth)
+
+**GET /api/admin/jobs/:id/history?limit=50** - Job execution history (admin auth)
+
+## Data Model
+
+```typescript
+interface ScheduledJob {
+  id: string;
+  name: string;
+  type: JobType;
+  schedule: string; // cron
+  enabled: boolean;
+  lastRun: Date | null;
+  nextRun: Date | null;
+  payload: any;
+}
+```
+
+## Implementation Details
+
+**Scheduler Service (`scheduler.service.ts`):**
+- `start()`: Initializes scheduler, creates default jobs, schedules all enabled jobs
+- `scheduleJob()`: Adds job to Bull as repeatable job with cron expression
+- `unscheduleJob()`: Removes repeatable job from Bull
+- `updateScheduledJob()`: Unschedules old job, updates DB, schedules new job if enabled
+- `deleteScheduledJob()`: Unschedules job before deleting from DB
+
+**Job Queue Service (`job-queue.service.ts`):**
+- `addRepeatableJob()`: Registers job type with Bull's repeat scheduler
+- `removeRepeatableJob()`: Removes job from Bull's repeat scheduler
+- Processors for each scheduled job type call `scheduler.triggerJobNow()`
+- `setMaxListeners(20)`: Set on both Redis client and Bull queue to accommodate 12 job processors (6 regular + 6 scheduled)
+
+**Flow:**
+1. App starts → `scheduler.start()` → schedules all enabled jobs
+2. Bull triggers job at cron time → processor calls `triggerJobNow()`
+3. `triggerJobNow()` executes job-specific logic (Plex scan, Audible refresh, etc.)
+4. Updates `lastRun` timestamp in database
+
+## Audible Refresh Processor
+
+**Implementation:**
+1. Clear previous `isPopular`/`isNewRelease` flags
+2. Fetch 200 popular + 200 new releases (multi-page scraping)
+3. Download and cache cover thumbnails locally (stored in `/app/cache/thumbnails`)
+4. Store/update in DB with category flags, rankings (`popularRank`, `newReleaseRank`), and cached cover paths
+5. Record sync timestamp (`lastAudibleSync`)
+6. Clean up unused thumbnails (removes covers for audiobooks no longer in cache)
+7. Perform fuzzy matching (70% threshold) against Plex library
+8. Set `plexGuid` when match found (with duplicate protection)
+9. Update `availabilityStatus` to 'available' or 'unknown'
+
+**Duplicate PlexGuid Handling:** Since `plexGuid` has UNIQUE constraint, only first match gets assigned to prevent violations.
+
+**Thumbnail Caching:** Downloads cover images from Audible and stores them locally to reduce external requests. Cached thumbnails are served via `/api/cache/thumbnails/[filename]` endpoint. Unused thumbnails are automatically cleaned up after each sync.
+
+## Fixed Issues ✅
+
+- ✅ Jobs running without config validation
+- ✅ Default alert() popups → toast notifications
+- ✅ No UI for editing schedules → added edit modal
+- ✅ Audible data not persisting → saves to database
+- ✅ Download progress logging ~500x/s → 10s delay
+- ✅ Requests failing permanently (no torrents) → retry system with 'awaiting_search'
+- ✅ Requests failing permanently (no files) → retry system with max 5 retries + 'warn' status
+- ✅ Failed requests blocking re-requests → allow re-requesting failed/warn/cancelled
+- ✅ Files deleted immediately → kept until seeding requirements met
+- ✅ No seeding time config → added `seeding_time_minutes`
+- ✅ Scheduled jobs not running on schedule → implemented Bull repeatable jobs with cron scheduling
+- ✅ MaxListenersExceededWarning → increased maxListeners to 20 on both Redis client and Bull queue
+- ✅ Cron expressions not user-friendly → added human-readable descriptions and visual schedule builder
+- ✅ Scheduled jobs triggered by timer not appearing in system logs → Job records now created automatically for timer-triggered jobs
+- ✅ Scheduled jobs triggered by timer not updating lastRun timestamp → Job queue now updates lastRun when processing timer-triggered jobs
+- ✅ Daily cron patterns at non-midnight hours not recognized → Fixed `getIntervalFromCron` to parse any daily time (e.g., "0 4 * * *")
+- ✅ "Every 24 hours" interval validation error → Auto-converts 24+ hour intervals to daily schedule (0 0 * * *)
+
+## Tech Stack
+
+- Bull repeatable jobs
+- PostgreSQL (scheduled_jobs table)
+- Bull/Redis infrastructure