mirror of
https://github.com/kikootwo/ReadMeABook.git
synced 2026-06-02 20:30:10 +00:00
Add e-book sidecar integration and improve request handling
Introduces optional e-book sidecar downloads from Anna's Archive, including admin UI, settings API, FlareSolverr integration, and documentation. Enhances request creation logic to prevent duplicate downloads by checking for 'downloaded' and 'available' statuses, updates UI to reflect processing state, and adds SABnzbd support to download and cleanup flows. Also updates ranking algorithm documentation and improves cache invalidation for recent requests.
This commit is contained in:
kikootwo
@@ -36,6 +36,11 @@
|
||||
- **Database caching, real-time matching** → [integrations/audible.md](integrations/audible.md)
|
||||
- **Book covers API for login page** → [frontend/pages/login.md](frontend/pages/login.md)
|
||||
|
||||
## E-book Sidecar
|
||||
- **Optional e-book downloads from Anna's Archive** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
- **ASIN-based matching, format selection** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
- **Non-blocking, atomic failures** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
|
||||
## Automation Pipeline
|
||||
- **Full pipeline overview** → [phase3/README.md](phase3/README.md)
|
||||
- **Search via Prowlarr (torrents + NZBs)** → [phase3/prowlarr.md](phase3/prowlarr.md)
|
||||
@@ -76,6 +81,8 @@
|
||||
**"How do torrent downloads work?"** → [phase3/qbittorrent.md](phase3/qbittorrent.md), [backend/services/jobs.md](backend/services/jobs.md)
|
||||
**"How do Usenet/NZB downloads work?"** → [phase3/sabnzbd.md](phase3/sabnzbd.md), [backend/services/jobs.md](backend/services/jobs.md)
|
||||
**"How does Plex matching work?"** → [integrations/plex.md](integrations/plex.md)
|
||||
**"How does e-book sidecar work?"** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md)
|
||||
**"How do I enable e-book downloads?"** → [integrations/ebook-sidecar.md](integrations/ebook-sidecar.md), [settings-pages.md](settings-pages.md)
|
||||
**"How do scheduled jobs work?"** → [backend/services/scheduler.md](backend/services/scheduler.md)
|
||||
**"How do I configure external services?"** → [setup-wizard.md](setup-wizard.md), [settings-pages.md](settings-pages.md)
|
||||
**"What's the database schema?"** → [backend/database.md](backend/database.md)
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
# E-book Sidecar
|
||||
|
||||
**Status:** ✅ Implemented | Optional e-book downloads from Anna's Archive
|
||||
|
||||
## Overview
|
||||
Automatically downloads e-books from Anna's Archive to accompany audiobooks, placing them in the same folder.
|
||||
|
||||
## Key Details
|
||||
- **When:** Runs during file organization (after audiobook copied, after cover art)
|
||||
- **Matching:** ASIN-based search (exact match)
|
||||
- **Non-blocking:** Failures don't affect audiobook download
|
||||
- **Atomic:** Either succeeds or fails gracefully
|
||||
- **Location:** E-book placed in same directory as audiobook
|
||||
- **Filename:** `[Title] - [Author].[format]` (sanitized)
|
||||
|
||||
## Configuration
|
||||
|
||||
**Admin Settings → E-book Sidecar tab**
|
||||
|
||||
| Key | Default | Options | Description |
|
||||
|-----|---------|---------|-------------|
|
||||
| `ebook_sidecar_enabled` | `false` | `true/false` | Enable feature |
|
||||
| `ebook_sidecar_preferred_format` | `epub` | `epub, pdf, mobi, azw3, any` | Preferred format |
|
||||
| `ebook_sidecar_base_url` | `https://annas-archive.li` | URL | Base URL (mirror resilience) |
|
||||
| `ebook_sidecar_flaresolverr_url` | `` (empty) | URL | FlareSolverr proxy URL (optional) |
|
||||
|
||||
**Stored in:** `Configuration` table (database)
|
||||
|
||||
## FlareSolverr Integration
|
||||
|
||||
Anna's Archive uses Cloudflare protection which may block direct scraping requests. FlareSolverr solves this by using a headless browser to bypass the protection.
|
||||
|
||||
### What is FlareSolverr?
|
||||
- Proxy server using headless Chrome/Chromium
|
||||
- Automatically solves Cloudflare challenges
|
||||
- Returns HTML content after challenge is solved
|
||||
- Open source: https://github.com/FlareSolverr/FlareSolverr
|
||||
|
||||
### When to Use FlareSolverr
|
||||
- **Required:** When e-book downloads consistently fail with no search results
|
||||
- **Optional:** If direct requests work (depends on Cloudflare's current state)
|
||||
- **Recommended:** For reliable, consistent downloads
|
||||
|
||||
### Setup
|
||||
1. Run FlareSolverr via Docker:
|
||||
```bash
|
||||
docker run -d --name flaresolverr -p 8191:8191 ghcr.io/flaresolverr/flaresolverr:latest
|
||||
```
|
||||
2. In Admin Settings → E-book Sidecar, enter: `http://localhost:8191`
|
||||
3. Click "Test Connection" to verify
|
||||
|
||||
### How It Works
|
||||
1. Requests are routed through FlareSolverr
|
||||
2. FlareSolverr loads the page in headless Chrome
|
||||
3. If Cloudflare challenge appears, it waits for solution
|
||||
4. HTML is returned after page loads
|
||||
5. Falls back to direct requests if FlareSolverr fails
|
||||
|
||||
### Performance Impact
|
||||
- **First request:** ~5-10 seconds (browser startup)
|
||||
- **Subsequent requests:** ~2-5 seconds per page
|
||||
- **Total time:** ~15-30 seconds per e-book (vs ~5-15 without)
|
||||
|
||||
## How It Works
|
||||
|
||||
### Flow
|
||||
1. **Trigger:** File organization completes audiobook copy
|
||||
2. **Check:** `ebook_sidecar_enabled === 'true'`
|
||||
3. **Search:** Try ASIN first (if available), then fall back to title + author
|
||||
4. **Extract MD5:** First search result → MD5 hash
|
||||
5. **Get Download Links:** Find "no waitlist" slow download links
|
||||
6. **Extract URL:** Parse slow download page for actual file server URL
|
||||
7. **Download:** Stream file to audiobook directory
|
||||
8. **Rename:** Sanitize filename based on metadata
|
||||
|
||||
### Scraping Strategy
|
||||
|
||||
**Method 1: ASIN Search (exact match)**
|
||||
```
|
||||
Search: https://annas-archive.li/search?ext=epub&q="asin:B09TWSRMCB"
|
||||
↓
|
||||
MD5 Page: https://annas-archive.li/md5/[md5]
|
||||
↓ (Filter: "slow partner server" links)
|
||||
Slow Download: https://annas-archive.li/slow_download/[md5]/0/5
|
||||
↓ (Parse for actual download URL)
|
||||
File Server: http://[server-ip]:port/path/to/file.epub
|
||||
```
|
||||
|
||||
**Method 2: Title + Author Search (fallback)**
|
||||
```
|
||||
Search: https://annas-archive.li/search?q=The+Housemaid+Freida+McFadden
|
||||
&ext=epub
|
||||
&content=book_nonfiction&content=book_fiction&content=book_unknown
|
||||
&lang=en
|
||||
↓
|
||||
(Same flow as ASIN search from MD5 page onwards)
|
||||
```
|
||||
|
||||
### Matching Priority
|
||||
1. **ASIN** (exact match - most accurate, if available)
|
||||
2. **Title + Author** (fuzzy match with book/language filters)
|
||||
|
||||
### Retry Logic
|
||||
- **Max attempts:** 5 slow download links
|
||||
- **Timeout:** 60 seconds per download
|
||||
- **Delays:** 1.5 seconds between requests
|
||||
- **Retries:** 3x for 5xx errors with exponential backoff
|
||||
|
||||
## Format Support
|
||||
|
||||
| Format | Extension | Recommended | Notes |
|
||||
|--------|-----------|-------------|-------|
|
||||
| EPUB | `.epub` | ✅ Yes | Most compatible with e-readers |
|
||||
| PDF | `.pdf` | ⚠️ Sometimes | Best for fixed-layout books |
|
||||
| MOBI | `.mobi` | ⚠️ Legacy | Kindle (older devices) |
|
||||
| AZW3 | `.azw3` | ⚠️ Sometimes | Kindle (newer devices) |
|
||||
| Any | `[first available]` | ❌ No | Downloads first match |
|
||||
|
||||
**Recommendation:** Use EPUB for maximum compatibility.
|
||||
|
||||
## File Naming
|
||||
|
||||
**Pattern:** `[Title] - [Author].[format]`
|
||||
|
||||
**Sanitization:**
|
||||
- Remove invalid chars: `<>:"/\|?*`
|
||||
- Collapse multiple spaces
|
||||
- Trim leading/trailing spaces and dots
|
||||
- Limit to 100 characters
|
||||
|
||||
**Examples:**
|
||||
- `The Housemaid - Freida McFadden.epub`
|
||||
- `Project Hail Mary - Andy Weir.pdf`
|
||||
|
||||
## Error Handling
|
||||
|
||||
**Graceful Failures (non-blocking):**
|
||||
- No ASIN available → Skip silently (log info)
|
||||
- No search results → Log warning, continue audiobook
|
||||
- No download links → Log warning, continue audiobook
|
||||
- All downloads fail → Log error, continue audiobook
|
||||
- Download timeout → Log error, continue audiobook
|
||||
|
||||
**Never Blocks Audiobook:**
|
||||
- All e-book errors are non-fatal
|
||||
- Audiobook organization completes regardless
|
||||
- Errors logged to job events (visible in admin)
|
||||
|
||||
## Logging
|
||||
|
||||
**Success (with FlareSolverr):**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Using FlareSolverr at http://localhost:8191
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
Found via ASIN: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found MD5: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found 8 download link(s)
|
||||
Attempting download link 1/5...
|
||||
Downloading from: 93.123.118.12
|
||||
E-book downloaded: The Housemaid - Freida McFadden.epub
|
||||
```
|
||||
|
||||
**Success (ASIN match, direct):**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
Found via ASIN: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found MD5: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found 8 download link(s)
|
||||
Attempting download link 1/5...
|
||||
Downloading from: 93.123.118.12
|
||||
E-book downloaded: The Housemaid - Freida McFadden.epub
|
||||
```
|
||||
|
||||
**Success (Title fallback):**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
No results for ASIN, falling back to title + author search...
|
||||
Searching by title + author: "The Housemaid" by Freida McFadden...
|
||||
Found via title search: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found MD5: 3b6f9c0f1665c4ba6e3214d43c37e1de
|
||||
Found 8 download link(s)
|
||||
E-book downloaded: The Housemaid - Freida McFadden.epub
|
||||
```
|
||||
|
||||
**Failure:**
|
||||
```
|
||||
E-book sidecar enabled, searching for e-book...
|
||||
Searching by ASIN: B09TWSRMCB (format: epub)...
|
||||
No results for ASIN, falling back to title + author search...
|
||||
Searching by title + author: "The Housemaid" by Freida McFadden...
|
||||
No search results found for title: "The Housemaid" by Freida McFadden
|
||||
E-book download failed: No search results found (tried ASIN and title+author)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### E-book Not Downloaded
|
||||
|
||||
**Cause:** No matching e-book in Anna's Archive (tried ASIN and title+author)
|
||||
**Solution:** Not all audiobooks have e-book equivalents, this is expected
|
||||
|
||||
**Cause:** ASIN mismatch (Anna's Archive has different ASIN)
|
||||
**Solution:** Feature now automatically falls back to title + author search
|
||||
|
||||
**Cause:** All download links failed
|
||||
**Solution:** Check job logs for errors, may be temporary server issues
|
||||
|
||||
### Wrong Format Downloaded
|
||||
|
||||
**Cause:** Preferred format not available
|
||||
**Solution:** Anna's Archive doesn't have that format, falls back to available format
|
||||
|
||||
### Download Timeout
|
||||
|
||||
**Cause:** Slow file server or large file
|
||||
**Solution:** Automatic retry with next download link
|
||||
|
||||
### Feature Not Working
|
||||
|
||||
**Cause:** Feature disabled
|
||||
**Solution:** Admin Settings → E-book Sidecar → Enable toggle
|
||||
|
||||
### Cloudflare Blocking
|
||||
|
||||
**Cause:** Anna's Archive has Cloudflare protection enabled
|
||||
**Solution:** Configure FlareSolverr (see FlareSolverr Integration section)
|
||||
|
||||
**Symptoms:**
|
||||
- No search results found
|
||||
- Requests timing out
|
||||
- Errors about Cloudflare challenge
|
||||
|
||||
### FlareSolverr Not Working
|
||||
|
||||
**Cause:** FlareSolverr not running or unreachable
|
||||
**Solution:**
|
||||
1. Verify FlareSolverr is running: `docker ps | grep flaresolverr`
|
||||
2. Check URL is correct (usually `http://localhost:8191`)
|
||||
3. Test connection in Admin Settings
|
||||
|
||||
**Cause:** FlareSolverr timing out
|
||||
**Solution:** FlareSolverr may need more time; check container logs for errors
|
||||
|
||||
## Security & Legal
|
||||
|
||||
**Important Notes:**
|
||||
- Anna's Archive is a shadow library
|
||||
- Use at your own discretion and responsibility
|
||||
- Ensure compliance with local laws and regulations
|
||||
- Feature is optional and disabled by default
|
||||
- No API key required (web scraping)
|
||||
|
||||
**Privacy:**
|
||||
- User-Agent: `ReadMeABook/1.0 (Audiobook Automation)`
|
||||
- No tracking or analytics
|
||||
- Distributed (each user scrapes for themselves)
|
||||
|
||||
## Technical Implementation
|
||||
|
||||
**Files:**
|
||||
- Service: `src/lib/services/ebook-scraper.ts`
|
||||
- Integration: `src/lib/utils/file-organizer.ts` (line 265+)
|
||||
- Settings API: `src/app/api/admin/settings/ebook/route.ts`
|
||||
- FlareSolverr Test API: `src/app/api/admin/settings/ebook/test-flaresolverr/route.ts`
|
||||
- UI: `src/app/admin/settings/page.tsx` (ebook tab)
|
||||
|
||||
**Dependencies:**
|
||||
- axios (HTTP requests)
|
||||
- cheerio (HTML parsing)
|
||||
- fs/promises (file operations)
|
||||
|
||||
**Caching:**
|
||||
- MD5 lookups cached in-memory (prevents re-scraping same ASIN)
|
||||
- Cache cleared on service restart
|
||||
|
||||
## Performance
|
||||
|
||||
**Impact:**
|
||||
- **Network:** 3-5 requests per e-book (search, MD5, slow download pages)
|
||||
- **Time:** ~5-15 seconds per e-book (depends on file server)
|
||||
- **Storage:** E-books typically 1-50 MB
|
||||
- **CPU:** Minimal (streaming download)
|
||||
|
||||
## Limitations
|
||||
|
||||
1. **Match Accuracy:** Title + author search may return wrong book if title is common
|
||||
2. **Format Availability:** Depends on Anna's Archive catalog
|
||||
3. **Download Speed:** Depends on file server load
|
||||
4. **Language:** Title search filters for English books only
|
||||
5. **Success Rate:** ~70-90% (ASIN has higher accuracy, title fallback is less precise)
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- ISBN-13 fallback matching (between ASIN and title search)
|
||||
- Format preference priority list (try EPUB, then PDF, then MOBI)
|
||||
- Per-request override (API endpoint)
|
||||
- Statistics tracking (success rate, formats, match method)
|
||||
- Rate limit monitoring
|
||||
- Relevance scoring for title search results
|
||||
|
||||
## Related
|
||||
- [File Organization](../phase3/file-organization.md) - Where e-book download happens
|
||||
- [Settings Pages](../settings-pages.md) - Configuration UI
|
||||
- [Configuration Service](../backend/services/config.md) - Settings storage
|
||||
@@ -30,11 +30,16 @@ Evaluates and scores torrents to automatically select best audiobook download.
|
||||
- Example: "We Are Legion (We Are Bob)" tries both full title and "We Are Legion"
|
||||
- Handles torrents that include subtitle AND those that omit it
|
||||
- Complete title match requirements (both must be true):
|
||||
- No significant words BEFORE matched title (prevents "This Inevitable Ruin Dungeon Crawler Carl, Book 7")
|
||||
- Followed by metadata markers: " by", " [", " -", " (", " {", " :", ","
|
||||
- **Acceptable prefix** (any of these):
|
||||
- No significant words before title (clean match)
|
||||
- Title preceded by metadata separator (` - `, `: `, `—`) — handles "Author - Series - 01 - Title"
|
||||
- Author name appears in prefix — handles "Author Name - Title"
|
||||
- **Acceptable suffix**: Followed by metadata markers: " by", " [", " -", " (", " {", " :", "," or end of string
|
||||
- Complete match → 35 pts
|
||||
- Title has prefix/suffix words OR continues with more words → fuzzy similarity (partial credit)
|
||||
- Prevents series confusion: "The Housemaid" vs "The Housemaid's Secret", "Dungeon Crawler Carl" vs "Book 7"
|
||||
- Unstructured prefix (words without separators) → fuzzy similarity (partial credit)
|
||||
- Prevents: "This Inevitable Ruin Dungeon Crawler Carl" matching "Dungeon Crawler Carl"
|
||||
- Suffix continues with non-metadata → fuzzy similarity (partial credit)
|
||||
- Prevents: "The Housemaid's Secret" matching "The Housemaid"
|
||||
- No substring match → fuzzy similarity (best score from full or required title)
|
||||
|
||||
**Stage 3: Author Matching (0-15 pts)**
|
||||
|
||||
@@ -102,6 +102,9 @@ export function RecentRequestsTable({ requests }: RecentRequestsTableProps) {
|
||||
await mutate('/api/admin/requests/recent');
|
||||
await mutate('/api/admin/metrics');
|
||||
|
||||
// Invalidate audiobook caches to update request status on home/search pages
|
||||
await mutate((key) => typeof key === 'string' && key.includes('/api/audiobooks'));
|
||||
|
||||
// Close dialog
|
||||
setShowDeleteConfirm(false);
|
||||
setSelectedRequest(null);
|
||||
|
||||
@@ -82,6 +82,12 @@ interface Settings {
|
||||
mediaDir: string;
|
||||
metadataTaggingEnabled: boolean;
|
||||
};
|
||||
ebook: {
|
||||
enabled: boolean;
|
||||
preferredFormat: string;
|
||||
baseUrl: string;
|
||||
flaresolverrUrl: string;
|
||||
};
|
||||
}
|
||||
|
||||
interface PendingUser {
|
||||
@@ -127,7 +133,7 @@ export default function AdminSettings() {
|
||||
const [message, setMessage] = useState<{ type: 'success' | 'error'; text: string } | null>(
|
||||
null
|
||||
);
|
||||
const [activeTab, setActiveTab] = useState<'library' | 'auth' | 'prowlarr' | 'download' | 'paths' | 'account' | 'bookdate'>('library');
|
||||
const [activeTab, setActiveTab] = useState<'library' | 'auth' | 'prowlarr' | 'download' | 'paths' | 'ebook' | 'account' | 'bookdate'>('library');
|
||||
|
||||
// Password change form state
|
||||
const [passwordForm, setPasswordForm] = useState({
|
||||
@@ -147,6 +153,14 @@ export default function AdminSettings() {
|
||||
const [testingBookdate, setTestingBookdate] = useState(false);
|
||||
const [clearingBookdateSwipes, setClearingBookdateSwipes] = useState(false);
|
||||
|
||||
// FlareSolverr testing state
|
||||
const [testingFlaresolverr, setTestingFlaresolverr] = useState(false);
|
||||
const [flaresolverrTestResult, setFlaresolverrTestResult] = useState<{
|
||||
success: boolean;
|
||||
message: string;
|
||||
responseTime?: number;
|
||||
} | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
fetchSettings();
|
||||
fetchCurrentUser();
|
||||
@@ -460,6 +474,73 @@ export default function AdminSettings() {
|
||||
}
|
||||
};
|
||||
|
||||
const handleSaveEbookSettings = async () => {
|
||||
if (!settings) return;
|
||||
|
||||
setSaving(true);
|
||||
setMessage(null);
|
||||
|
||||
try {
|
||||
const response = await fetchWithAuth('/api/admin/settings/ebook', {
|
||||
method: 'PUT',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
enabled: settings.ebook?.enabled || false,
|
||||
format: settings.ebook?.preferredFormat || 'epub',
|
||||
baseUrl: settings.ebook?.baseUrl || 'https://annas-archive.li',
|
||||
flaresolverrUrl: settings.ebook?.flaresolverrUrl || '',
|
||||
}),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error('Failed to save e-book settings');
|
||||
}
|
||||
|
||||
setMessage({ type: 'success', text: 'E-book sidecar settings saved successfully!' });
|
||||
// Update original settings to reflect the saved state
|
||||
setOriginalSettings(JSON.parse(JSON.stringify(settings)));
|
||||
setTimeout(() => setMessage(null), 3000);
|
||||
} catch (error) {
|
||||
setMessage({
|
||||
type: 'error',
|
||||
text: error instanceof Error ? error.message : 'Failed to save e-book settings',
|
||||
});
|
||||
} finally {
|
||||
setSaving(false);
|
||||
}
|
||||
};
|
||||
|
||||
const testFlaresolverrConnection = async () => {
|
||||
if (!settings?.ebook?.flaresolverrUrl) {
|
||||
setFlaresolverrTestResult({
|
||||
success: false,
|
||||
message: 'Please enter a FlareSolverr URL first',
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
setTestingFlaresolverr(true);
|
||||
setFlaresolverrTestResult(null);
|
||||
|
||||
try {
|
||||
const response = await fetchWithAuth('/api/admin/settings/ebook/test-flaresolverr', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ url: settings.ebook.flaresolverrUrl }),
|
||||
});
|
||||
|
||||
const result = await response.json();
|
||||
setFlaresolverrTestResult(result);
|
||||
} catch (error) {
|
||||
setFlaresolverrTestResult({
|
||||
success: false,
|
||||
message: error instanceof Error ? error.message : 'Test failed',
|
||||
});
|
||||
} finally {
|
||||
setTestingFlaresolverr(false);
|
||||
}
|
||||
};
|
||||
|
||||
const testPlexConnection = async () => {
|
||||
if (!settings) return;
|
||||
|
||||
@@ -924,6 +1005,7 @@ export default function AdminSettings() {
|
||||
{ id: 'prowlarr', label: 'Indexers', icon: '🔍' },
|
||||
{ id: 'download', label: 'Download Client', icon: '⬇️' },
|
||||
{ id: 'paths', label: 'Paths', icon: '📁' },
|
||||
{ id: 'ebook', label: 'E-book Sidecar', icon: '📖' },
|
||||
{ id: 'bookdate', label: 'BookDate', icon: '📚' },
|
||||
...(isLocalAdmin ? [{ id: 'account', label: 'Account', icon: '🔒' }] : []),
|
||||
];
|
||||
@@ -1915,6 +1997,201 @@ export default function AdminSettings() {
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* E-book Sidecar Tab */}
|
||||
{activeTab === 'ebook' && (
|
||||
<div className="space-y-6 max-w-2xl">
|
||||
<div>
|
||||
<h2 className="text-xl font-semibold text-gray-900 dark:text-gray-100 mb-4">
|
||||
E-book Sidecar
|
||||
</h2>
|
||||
<p className="text-gray-600 dark:text-gray-400 mb-6">
|
||||
Automatically download e-books from Anna's Archive to accompany your audiobooks.
|
||||
E-books are placed in the same folder as the audiobook files.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Enable Toggle */}
|
||||
<div className="bg-gray-50 dark:bg-gray-800 rounded-lg p-4 border border-gray-200 dark:border-gray-700">
|
||||
<div className="flex items-start gap-4">
|
||||
<input
|
||||
type="checkbox"
|
||||
id="ebook-enabled"
|
||||
checked={settings.ebook?.enabled || false}
|
||||
onChange={(e) => {
|
||||
setSettings({
|
||||
...settings,
|
||||
ebook: { ...settings.ebook, enabled: e.target.checked },
|
||||
});
|
||||
}}
|
||||
className="mt-1 h-5 w-5 rounded border-gray-300 text-blue-600 focus:ring-blue-500"
|
||||
/>
|
||||
<div className="flex-1">
|
||||
<label
|
||||
htmlFor="ebook-enabled"
|
||||
className="block text-sm font-medium text-gray-900 dark:text-gray-100 cursor-pointer"
|
||||
>
|
||||
Enable e-book sidecar downloads
|
||||
</label>
|
||||
<p className="text-sm text-gray-600 dark:text-gray-400 mt-1">
|
||||
When enabled, the system will search for e-books matching your audiobook's ASIN
|
||||
and download them to the same folder.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Format Selection */}
|
||||
{settings.ebook?.enabled && (
|
||||
<div>
|
||||
<label className="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
|
||||
Preferred Format
|
||||
</label>
|
||||
<select
|
||||
value={settings.ebook?.preferredFormat || 'epub'}
|
||||
onChange={(e) => {
|
||||
setSettings({
|
||||
...settings,
|
||||
ebook: { ...settings.ebook, preferredFormat: e.target.value },
|
||||
});
|
||||
}}
|
||||
className="w-full px-3 py-2 border border-gray-300 dark:border-gray-600 rounded-lg
|
||||
bg-white dark:bg-gray-800 text-gray-900 dark:text-gray-100
|
||||
focus:ring-2 focus:ring-blue-500 focus:border-transparent"
|
||||
>
|
||||
<option value="epub">EPUB</option>
|
||||
<option value="pdf">PDF</option>
|
||||
<option value="mobi">MOBI</option>
|
||||
<option value="azw3">AZW3</option>
|
||||
<option value="any">Any format</option>
|
||||
</select>
|
||||
<p className="text-sm text-gray-500 dark:text-gray-400 mt-1">
|
||||
EPUB is recommended for most e-readers. "Any format" will download the first available format.
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Base URL (Advanced) */}
|
||||
{settings.ebook?.enabled && (
|
||||
<div>
|
||||
<label className="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
|
||||
Base URL (Advanced)
|
||||
</label>
|
||||
<Input
|
||||
type="text"
|
||||
value={settings.ebook?.baseUrl || 'https://annas-archive.li'}
|
||||
onChange={(e) => {
|
||||
setSettings({
|
||||
...settings,
|
||||
ebook: { ...settings.ebook, baseUrl: e.target.value },
|
||||
});
|
||||
}}
|
||||
placeholder="https://annas-archive.li"
|
||||
className="font-mono"
|
||||
/>
|
||||
<p className="text-sm text-gray-500 dark:text-gray-400 mt-1">
|
||||
Change this if the primary Anna's Archive mirror is unavailable.
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* FlareSolverr (Optional - for Cloudflare bypass) */}
|
||||
{settings.ebook?.enabled && (
|
||||
<div className="space-y-4">
|
||||
<div>
|
||||
<label className="block text-sm font-medium text-gray-700 dark:text-gray-300 mb-2">
|
||||
FlareSolverr URL (Optional)
|
||||
</label>
|
||||
<div className="flex gap-2">
|
||||
<Input
|
||||
type="text"
|
||||
value={settings.ebook?.flaresolverrUrl || ''}
|
||||
onChange={(e) => {
|
||||
setSettings({
|
||||
...settings,
|
||||
ebook: { ...settings.ebook, flaresolverrUrl: e.target.value },
|
||||
});
|
||||
setFlaresolverrTestResult(null);
|
||||
}}
|
||||
placeholder="http://localhost:8191"
|
||||
className="font-mono flex-1"
|
||||
/>
|
||||
<Button
|
||||
onClick={testFlaresolverrConnection}
|
||||
loading={testingFlaresolverr}
|
||||
variant="secondary"
|
||||
className="whitespace-nowrap"
|
||||
>
|
||||
Test Connection
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-sm text-gray-500 dark:text-gray-400 mt-1">
|
||||
FlareSolverr helps bypass Cloudflare protection on Anna's Archive.
|
||||
Leave empty if not needed.
|
||||
</p>
|
||||
{flaresolverrTestResult && (
|
||||
<div
|
||||
className={`mt-2 p-3 rounded-lg text-sm ${
|
||||
flaresolverrTestResult.success
|
||||
? 'bg-green-50 dark:bg-green-900/20 text-green-800 dark:text-green-200 border border-green-200 dark:border-green-800'
|
||||
: 'bg-red-50 dark:bg-red-900/20 text-red-800 dark:text-red-200 border border-red-200 dark:border-red-800'
|
||||
}`}
|
||||
>
|
||||
{flaresolverrTestResult.success ? '✓ ' : '✗ '}
|
||||
{flaresolverrTestResult.message}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
{!settings.ebook?.flaresolverrUrl && (
|
||||
<div className="bg-amber-50 dark:bg-amber-900/20 border border-amber-200 dark:border-amber-800 rounded-lg p-3">
|
||||
<p className="text-sm text-amber-800 dark:text-amber-200">
|
||||
<strong>Note:</strong> Without FlareSolverr, e-book downloads may fail if Anna's Archive
|
||||
has Cloudflare protection enabled. Success rates are typically lower without it.
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Info Box */}
|
||||
<div className="bg-blue-50 dark:bg-blue-900/20 border border-blue-200 dark:border-blue-800 rounded-lg p-4">
|
||||
<h3 className="text-sm font-semibold text-blue-900 dark:text-blue-100 mb-2">
|
||||
How it works
|
||||
</h3>
|
||||
<ul className="space-y-2 text-sm text-blue-800 dark:text-blue-200">
|
||||
<li>• Searches Anna's Archive in two ways:</li>
|
||||
<li className="ml-4">1. First tries ASIN (exact match - most accurate)</li>
|
||||
<li className="ml-4">2. Falls back to title + author (with book/language filters)</li>
|
||||
<li>• Downloads matching e-book in your preferred format</li>
|
||||
<li>• Places e-book file in the same folder as the audiobook</li>
|
||||
<li>• If no match is found or download fails, audiobook download continues normally</li>
|
||||
<li>• Completely optional and non-blocking</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
{/* Warning Box */}
|
||||
<div className="bg-yellow-50 dark:bg-yellow-900/20 border border-yellow-200 dark:border-yellow-800 rounded-lg p-4">
|
||||
<h3 className="text-sm font-semibold text-yellow-900 dark:text-yellow-100 mb-2">
|
||||
⚠️ Important Note
|
||||
</h3>
|
||||
<p className="text-sm text-yellow-800 dark:text-yellow-200">
|
||||
Anna's Archive is a shadow library. Use of this feature is at your own discretion and responsibility.
|
||||
Ensure compliance with your local laws and regulations.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* Save Button */}
|
||||
<div className="border-t border-gray-200 dark:border-gray-700 pt-6">
|
||||
<Button
|
||||
onClick={handleSaveEbookSettings}
|
||||
loading={saving}
|
||||
className="w-full bg-blue-600 hover:bg-blue-700"
|
||||
>
|
||||
Save E-book Sidecar Settings
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* BookDate Tab */}
|
||||
{activeTab === 'bookdate' && (
|
||||
<div className="space-y-6 max-w-2xl">
|
||||
@@ -2738,8 +3015,8 @@ export default function AdminSettings() {
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Footer - Hide for Account tab */}
|
||||
{activeTab !== 'account' && activeTab !== 'bookdate' && (
|
||||
{/* Footer - Hide for Account, BookDate, and E-book tabs (they have their own save buttons) */}
|
||||
{activeTab !== 'account' && activeTab !== 'bookdate' && activeTab !== 'ebook' && (
|
||||
<div className="bg-gray-50 dark:bg-gray-900 px-8 py-4 border-t border-gray-200 dark:border-gray-700">
|
||||
<div className="flex justify-end gap-4">
|
||||
<Button variant="outline" onClick={() => window.location.reload()}>
|
||||
|
||||
@@ -7,6 +7,8 @@ import { NextRequest, NextResponse } from 'next/server';
|
||||
import { requireAuth, requireAdmin, AuthenticatedRequest } from '@/lib/middleware/auth';
|
||||
import { prisma } from '@/lib/db';
|
||||
import { getQBittorrentService } from '@/lib/integrations/qbittorrent.service';
|
||||
import { getSABnzbdService } from '@/lib/integrations/sabnzbd.service';
|
||||
import { getConfigService } from '@/lib/services/config.service';
|
||||
|
||||
export async function GET(request: NextRequest) {
|
||||
return requireAuth(request, async (req: AuthenticatedRequest) => {
|
||||
@@ -48,6 +50,7 @@ export async function GET(request: NextRequest) {
|
||||
downloadStatus: true,
|
||||
torrentName: true,
|
||||
torrentHash: true,
|
||||
nzbId: true,
|
||||
startedAt: true,
|
||||
createdAt: true,
|
||||
},
|
||||
@@ -59,48 +62,41 @@ export async function GET(request: NextRequest) {
|
||||
take: 20,
|
||||
});
|
||||
|
||||
// Get qBittorrent service
|
||||
let qbService;
|
||||
try {
|
||||
qbService = await getQBittorrentService();
|
||||
} catch (error) {
|
||||
console.error('[Admin] Failed to initialize qBittorrent service:', error);
|
||||
// Return downloads without speed/eta if qBittorrent is unavailable
|
||||
const formatted = activeDownloads.map((download) => ({
|
||||
requestId: download.id,
|
||||
title: download.audiobook.title,
|
||||
author: download.audiobook.author,
|
||||
status: download.status,
|
||||
progress: download.progress,
|
||||
speed: 0,
|
||||
eta: null,
|
||||
torrentName: download.downloadHistory[0]?.torrentName || null,
|
||||
downloadStatus: download.downloadHistory[0]?.downloadStatus || null,
|
||||
user: download.user.plexUsername,
|
||||
startedAt: download.downloadHistory[0]?.startedAt || download.downloadHistory[0]?.createdAt || download.updatedAt,
|
||||
}));
|
||||
return NextResponse.json({ downloads: formatted });
|
||||
}
|
||||
// Get configured download client type
|
||||
const configService = getConfigService();
|
||||
const clientType = (await configService.get('download_client_type')) || 'qbittorrent';
|
||||
|
||||
// Format response with speed and ETA from qBittorrent
|
||||
// Format response with speed and ETA from download client
|
||||
const formatted = await Promise.all(
|
||||
activeDownloads.map(async (download) => {
|
||||
let speed = 0;
|
||||
let eta: number | null = null;
|
||||
|
||||
try {
|
||||
if (clientType === 'qbittorrent') {
|
||||
// Get torrent hash from download history
|
||||
const torrentHash = download.downloadHistory[0]?.torrentHash;
|
||||
|
||||
// Fetch torrent info from qBittorrent if we have a hash
|
||||
if (torrentHash) {
|
||||
try {
|
||||
const qbService = await getQBittorrentService();
|
||||
const torrentInfo = await qbService.getTorrent(torrentHash);
|
||||
speed = torrentInfo.dlspeed;
|
||||
eta = torrentInfo.eta > 0 ? torrentInfo.eta : null;
|
||||
} catch (error) {
|
||||
// Torrent not found or other error - use defaults
|
||||
console.error(`[Admin] Failed to get torrent info for ${torrentHash}:`, error);
|
||||
}
|
||||
} else if (clientType === 'sabnzbd') {
|
||||
// Get NZB ID from download history
|
||||
const nzbId = download.downloadHistory[0]?.nzbId;
|
||||
if (nzbId) {
|
||||
const sabnzbdService = await getSABnzbdService();
|
||||
const nzbInfo = await sabnzbdService.getNZB(nzbId);
|
||||
if (nzbInfo) {
|
||||
speed = nzbInfo.downloadSpeed;
|
||||
eta = nzbInfo.timeLeft > 0 ? nzbInfo.timeLeft : null;
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
// Download client unavailable or download not found - use defaults
|
||||
console.error(`[Admin] Failed to get download info:`, error);
|
||||
}
|
||||
|
||||
return {
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
/**
|
||||
* Component: E-book Sidecar Settings API
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*/
|
||||
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
import { requireAuth, requireAdmin, AuthenticatedRequest } from '@/lib/middleware/auth';
|
||||
|
||||
export async function PUT(request: NextRequest) {
|
||||
return requireAuth(request, async (req: AuthenticatedRequest) => {
|
||||
return requireAdmin(req, async () => {
|
||||
try {
|
||||
// Parse request body
|
||||
const { enabled, format, baseUrl, flaresolverrUrl } = await request.json();
|
||||
|
||||
// Validate format
|
||||
const validFormats = ['epub', 'pdf', 'mobi', 'azw3', 'any'];
|
||||
if (format && !validFormats.includes(format)) {
|
||||
return NextResponse.json(
|
||||
{ error: `Invalid format. Must be one of: ${validFormats.join(', ')}` },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
// Validate baseUrl (basic check)
|
||||
if (baseUrl && !baseUrl.startsWith('http')) {
|
||||
return NextResponse.json(
|
||||
{ error: 'Base URL must start with http:// or https://' },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
// Validate flaresolverrUrl if provided
|
||||
if (flaresolverrUrl && !flaresolverrUrl.startsWith('http')) {
|
||||
return NextResponse.json(
|
||||
{ error: 'FlareSolverr URL must start with http:// or https://' },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
// Save configuration
|
||||
const { getConfigService } = await import('@/lib/services/config.service');
|
||||
const configService = getConfigService();
|
||||
|
||||
const configs = [
|
||||
{
|
||||
key: 'ebook_sidecar_enabled',
|
||||
value: enabled ? 'true' : 'false',
|
||||
category: 'ebook',
|
||||
description: 'Enable e-book sidecar downloads from Annas Archive',
|
||||
},
|
||||
{
|
||||
key: 'ebook_sidecar_preferred_format',
|
||||
value: format || 'epub',
|
||||
category: 'ebook',
|
||||
description: 'Preferred e-book format',
|
||||
},
|
||||
{
|
||||
key: 'ebook_sidecar_base_url',
|
||||
value: baseUrl || 'https://annas-archive.li',
|
||||
category: 'ebook',
|
||||
description: 'Base URL for Annas Archive',
|
||||
},
|
||||
{
|
||||
key: 'ebook_sidecar_flaresolverr_url',
|
||||
value: flaresolverrUrl || '',
|
||||
category: 'ebook',
|
||||
description: 'FlareSolverr URL for bypassing Cloudflare protection',
|
||||
},
|
||||
];
|
||||
|
||||
await configService.setMany(configs);
|
||||
|
||||
return NextResponse.json({ success: true });
|
||||
} catch (error) {
|
||||
console.error('Failed to save e-book settings:', error);
|
||||
return NextResponse.json(
|
||||
{ error: 'Failed to save settings' },
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
/**
|
||||
* Component: FlareSolverr Connection Test API
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*/
|
||||
|
||||
import { NextRequest, NextResponse } from 'next/server';
|
||||
import { requireAuth, requireAdmin, AuthenticatedRequest } from '@/lib/middleware/auth';
|
||||
import { testFlareSolverrConnection } from '@/lib/services/ebook-scraper';
|
||||
|
||||
export async function POST(request: NextRequest) {
|
||||
return requireAuth(request, async (req: AuthenticatedRequest) => {
|
||||
return requireAdmin(req, async () => {
|
||||
try {
|
||||
const { url } = await request.json();
|
||||
|
||||
if (!url) {
|
||||
return NextResponse.json(
|
||||
{ error: 'FlareSolverr URL is required' },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
if (!url.startsWith('http')) {
|
||||
return NextResponse.json(
|
||||
{ error: 'URL must start with http:// or https://' },
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
const result = await testFlareSolverrConnection(url);
|
||||
|
||||
return NextResponse.json(result);
|
||||
} catch (error) {
|
||||
console.error('FlareSolverr test failed:', error);
|
||||
return NextResponse.json(
|
||||
{
|
||||
success: false,
|
||||
message: error instanceof Error ? error.message : 'Unknown error',
|
||||
},
|
||||
{ status: 500 }
|
||||
);
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -82,6 +82,12 @@ export async function GET(request: NextRequest) {
|
||||
mediaDir: configMap.get('media_dir') || '/media/audiobooks',
|
||||
metadataTaggingEnabled: configMap.get('metadata_tagging_enabled') === 'true',
|
||||
},
|
||||
ebook: {
|
||||
enabled: configMap.get('ebook_sidecar_enabled') === 'true',
|
||||
preferredFormat: configMap.get('ebook_sidecar_preferred_format') || 'epub',
|
||||
baseUrl: configMap.get('ebook_sidecar_base_url') || 'https://annas-archive.li',
|
||||
flaresolverrUrl: configMap.get('ebook_sidecar_flaresolverr_url') || '',
|
||||
},
|
||||
general: {
|
||||
appName: configMap.get('app_name') || 'ReadMeABook',
|
||||
allowRegistrations: configMap.get('allow_registrations') === 'true',
|
||||
|
||||
@@ -24,6 +24,8 @@ export async function POST(request: NextRequest) {
|
||||
localPath,
|
||||
} = await request.json();
|
||||
|
||||
console.log('[TestDownloadClient] Received request:', { type, url, hasUsername: !!username, hasPassword: !!password });
|
||||
|
||||
if (!type || !url) {
|
||||
return NextResponse.json(
|
||||
{ success: false, error: 'Type and URL are required' },
|
||||
@@ -59,6 +61,7 @@ export async function POST(request: NextRequest) {
|
||||
let version: string | undefined;
|
||||
|
||||
if (type === 'qbittorrent') {
|
||||
console.log('[TestDownloadClient] Testing qBittorrent connection');
|
||||
if (!username || !actualPassword) {
|
||||
return NextResponse.json(
|
||||
{ success: false, error: 'Username and password are required for qBittorrent' },
|
||||
@@ -74,6 +77,7 @@ export async function POST(request: NextRequest) {
|
||||
disableSSLVerify || false
|
||||
);
|
||||
} else if (type === 'sabnzbd') {
|
||||
console.log('[TestDownloadClient] Testing SABnzbd connection');
|
||||
if (!actualPassword) {
|
||||
return NextResponse.json(
|
||||
{ success: false, error: 'API key (password) is required for SABnzbd' },
|
||||
|
||||
@@ -57,7 +57,40 @@ export async function POST(request: NextRequest) {
|
||||
const body = await req.json();
|
||||
const { audiobook, torrent } = RequestWithTorrentSchema.parse(body);
|
||||
|
||||
// Check if audiobook is already available in Plex library
|
||||
// First check: Is there an existing request in 'downloaded' or 'available' status?
|
||||
// This catches the gap where files are organized but Plex hasn't scanned yet
|
||||
const existingActiveRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
audiobook: {
|
||||
audibleAsin: audiobook.asin,
|
||||
},
|
||||
status: { in: ['downloaded', 'available'] },
|
||||
deletedAt: null,
|
||||
},
|
||||
include: {
|
||||
user: { select: { plexUsername: true } },
|
||||
},
|
||||
});
|
||||
|
||||
if (existingActiveRequest) {
|
||||
const status = existingActiveRequest.status;
|
||||
const isOwnRequest = existingActiveRequest.userId === req.user.id;
|
||||
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: status === 'available' ? 'AlreadyAvailable' : 'BeingProcessed',
|
||||
message: status === 'available'
|
||||
? 'This audiobook is already available in your Plex library'
|
||||
: 'This audiobook is being processed and will be available soon',
|
||||
requestStatus: status,
|
||||
isOwnRequest,
|
||||
requestedBy: existingActiveRequest.user?.plexUsername,
|
||||
},
|
||||
{ status: 409 }
|
||||
);
|
||||
}
|
||||
|
||||
// Second check: Is audiobook already in Plex library? (fallback for non-requested books)
|
||||
const plexMatch = await findPlexMatch({
|
||||
asin: audiobook.asin,
|
||||
title: audiobook.title,
|
||||
|
||||
@@ -194,11 +194,39 @@ export async function PATCH(
|
||||
|
||||
const downloadHistory = requestWithData.downloadHistory[0];
|
||||
|
||||
// Get download path from qBittorrent
|
||||
// Get download path from the appropriate download client
|
||||
let downloadPath: string;
|
||||
|
||||
if (downloadHistory.torrentHash) {
|
||||
// qBittorrent - get path from torrent info
|
||||
const { getQBittorrentService } = await import('@/lib/integrations/qbittorrent.service');
|
||||
const qbt = await getQBittorrentService();
|
||||
const torrent = await qbt.getTorrent(downloadHistory.downloadClientId!);
|
||||
const downloadPath = `${torrent.save_path}/${torrent.name}`;
|
||||
const torrent = await qbt.getTorrent(downloadHistory.torrentHash);
|
||||
downloadPath = `${torrent.save_path}/${torrent.name}`;
|
||||
} else if (downloadHistory.nzbId) {
|
||||
// SABnzbd - get path from NZB info
|
||||
const { getSABnzbdService } = await import('@/lib/integrations/sabnzbd.service');
|
||||
const sabnzbd = await getSABnzbdService();
|
||||
const nzbInfo = await sabnzbd.getNZB(downloadHistory.nzbId);
|
||||
if (!nzbInfo || !nzbInfo.downloadPath) {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: 'ValidationError',
|
||||
message: 'Download path not available from SABnzbd',
|
||||
},
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
downloadPath = nzbInfo.downloadPath;
|
||||
} else {
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: 'ValidationError',
|
||||
message: 'No download client ID found in history',
|
||||
},
|
||||
{ status: 400 }
|
||||
);
|
||||
}
|
||||
|
||||
await jobQueue.addOrganizeJob(
|
||||
id,
|
||||
|
||||
@@ -41,7 +41,40 @@ export async function POST(request: NextRequest) {
|
||||
const body = await req.json();
|
||||
const { audiobook } = CreateRequestSchema.parse(body);
|
||||
|
||||
// Check if audiobook is already available in Plex library
|
||||
// First check: Is there an existing request in 'downloaded' or 'available' status?
|
||||
// This catches the gap where files are organized but Plex hasn't scanned yet
|
||||
const existingActiveRequest = await prisma.request.findFirst({
|
||||
where: {
|
||||
audiobook: {
|
||||
audibleAsin: audiobook.asin,
|
||||
},
|
||||
status: { in: ['downloaded', 'available'] },
|
||||
deletedAt: null,
|
||||
},
|
||||
include: {
|
||||
user: { select: { plexUsername: true } },
|
||||
},
|
||||
});
|
||||
|
||||
if (existingActiveRequest) {
|
||||
const status = existingActiveRequest.status;
|
||||
const isOwnRequest = existingActiveRequest.userId === req.user.id;
|
||||
|
||||
return NextResponse.json(
|
||||
{
|
||||
error: status === 'available' ? 'AlreadyAvailable' : 'BeingProcessed',
|
||||
message: status === 'available'
|
||||
? 'This audiobook is already available in your Plex library'
|
||||
: 'This audiobook is being processed and will be available soon',
|
||||
requestStatus: status,
|
||||
isOwnRequest,
|
||||
requestedBy: existingActiveRequest.user?.plexUsername,
|
||||
},
|
||||
{ status: 409 }
|
||||
);
|
||||
}
|
||||
|
||||
// Second check: Is audiobook already in Plex library? (fallback for non-requested books)
|
||||
const plexMatch = await findPlexMatch({
|
||||
asin: audiobook.asin,
|
||||
title: audiobook.title,
|
||||
|
||||
@@ -110,6 +110,17 @@ export function AudiobookCard({
|
||||
<span>Available</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Processing Badge - show when status is 'downloaded' */}
|
||||
{audiobook.requestStatus === 'downloaded' && (
|
||||
<div className="absolute top-2 right-2 bg-orange-500 text-white text-xs font-semibold px-2 py-1 rounded-md shadow-lg flex items-center gap-1">
|
||||
<svg className="w-3 h-3 animate-spin" fill="none" viewBox="0 0 24 24">
|
||||
<circle className="opacity-25" cx="12" cy="12" r="10" stroke="currentColor" strokeWidth="4"></circle>
|
||||
<path className="opacity-75" fill="currentColor" d="M4 12a8 8 0 018-8V0C5.373 0 0 5.373 0 12h4zm2 5.291A7.962 7.962 0 014 12H0c0 3.042 1.135 5.824 3 7.938l3-2.647z"></path>
|
||||
</svg>
|
||||
<span>Processing</span>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Content */}
|
||||
@@ -162,12 +173,17 @@ export function AudiobookCard({
|
||||
}
|
||||
|
||||
// Check if book is requested and in progress (non-re-requestable statuses)
|
||||
const inProgressStatuses = ['pending', 'awaiting_search', 'searching', 'downloading', 'processing', 'awaiting_import'];
|
||||
const inProgressStatuses = ['pending', 'awaiting_search', 'searching', 'downloading', 'processing', 'downloaded', 'awaiting_import'];
|
||||
if (audiobook.isRequested && audiobook.requestStatus && inProgressStatuses.includes(audiobook.requestStatus)) {
|
||||
// Show who requested it
|
||||
const buttonText = audiobook.requestedByUsername
|
||||
// Special text for 'downloaded' status (waiting for Plex scan)
|
||||
let buttonText;
|
||||
if (audiobook.requestStatus === 'downloaded') {
|
||||
buttonText = 'Processing...';
|
||||
} else {
|
||||
buttonText = audiobook.requestedByUsername
|
||||
? `Requested by ${audiobook.requestedByUsername}`
|
||||
: 'Requested';
|
||||
}
|
||||
|
||||
return (
|
||||
<Button
|
||||
|
||||
@@ -364,6 +364,7 @@ export function AudiobookDetailsModal({
|
||||
'searching',
|
||||
'downloading',
|
||||
'processing',
|
||||
'downloaded',
|
||||
'awaiting_import',
|
||||
];
|
||||
if (
|
||||
@@ -371,10 +372,15 @@ export function AudiobookDetailsModal({
|
||||
requestStatus &&
|
||||
inProgressStatuses.includes(requestStatus)
|
||||
) {
|
||||
// Show who requested it
|
||||
const buttonText = requestedByUsername
|
||||
// Special text for 'downloaded' status (waiting for Plex scan)
|
||||
let buttonText;
|
||||
if (requestStatus === 'downloaded') {
|
||||
buttonText = 'Processing...';
|
||||
} else {
|
||||
buttonText = requestedByUsername
|
||||
? `Requested by ${requestedByUsername}`
|
||||
: 'Already Requested';
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex-1">
|
||||
|
||||
@@ -105,6 +105,13 @@ export function useCreateRequest() {
|
||||
const data = await response.json();
|
||||
|
||||
if (!response.ok) {
|
||||
// Handle specific error types with custom messages
|
||||
if (data.error === 'BeingProcessed') {
|
||||
throw new Error('This audiobook is being processed. It will be available in your library soon.');
|
||||
}
|
||||
if (data.error === 'AlreadyAvailable') {
|
||||
throw new Error('This audiobook is already in your Plex library.');
|
||||
}
|
||||
throw new Error(data.message || 'Failed to create request');
|
||||
}
|
||||
|
||||
@@ -362,6 +369,13 @@ export function useRequestWithTorrent() {
|
||||
const data = await response.json();
|
||||
|
||||
if (!response.ok) {
|
||||
// Handle specific error types with custom messages
|
||||
if (data.error === 'BeingProcessed') {
|
||||
throw new Error('This audiobook is being processed. It will be available in your library soon.');
|
||||
}
|
||||
if (data.error === 'AlreadyAvailable') {
|
||||
throw new Error('This audiobook is already in your Plex library.');
|
||||
}
|
||||
throw new Error(data.message || 'Failed to create request and download torrent');
|
||||
}
|
||||
|
||||
|
||||
@@ -82,7 +82,22 @@ export async function processCleanupSeededTorrents(payload: CleanupSeededTorrent
|
||||
try {
|
||||
const downloadHistory = request.downloadHistory[0];
|
||||
|
||||
if (!downloadHistory || !downloadHistory.downloadClientId || !downloadHistory.indexerName) {
|
||||
if (!downloadHistory || !downloadHistory.indexerName) {
|
||||
continue;
|
||||
}
|
||||
|
||||
// Skip SABnzbd downloads - Usenet doesn't have seeding concept
|
||||
if (downloadHistory.nzbId && !downloadHistory.torrentHash) {
|
||||
// For soft-deleted SABnzbd requests, hard delete immediately (no seeding needed)
|
||||
if (request.deletedAt) {
|
||||
await prisma.request.delete({ where: { id: request.id } });
|
||||
await logger?.info(`Hard-deleted orphaned SABnzbd request ${request.id}`);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
// Only process torrent downloads
|
||||
if (!downloadHistory.torrentHash) {
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -111,7 +126,7 @@ export async function processCleanupSeededTorrents(payload: CleanupSeededTorrent
|
||||
|
||||
let torrent;
|
||||
try {
|
||||
torrent = await qbt.getTorrent(downloadHistory.downloadClientId);
|
||||
torrent = await qbt.getTorrent(downloadHistory.torrentHash);
|
||||
} catch (error) {
|
||||
// Torrent might already be deleted, skip
|
||||
continue;
|
||||
@@ -130,7 +145,7 @@ export async function processCleanupSeededTorrents(payload: CleanupSeededTorrent
|
||||
await logger?.info(`Torrent ${torrent.name} (${indexerName}) has met seeding requirement (${Math.floor(actualSeedingTime / 60)}/${seedingConfig.seedingTimeMinutes} minutes)`);
|
||||
|
||||
// Delete torrent and files from qBittorrent
|
||||
await qbt.deleteTorrent(downloadHistory.downloadClientId, true); // true = delete files
|
||||
await qbt.deleteTorrent(downloadHistory.torrentHash, true); // true = delete files
|
||||
|
||||
// If this is a soft-deleted request (orphaned download), hard delete it now
|
||||
if (request.deletedAt) {
|
||||
|
||||
@@ -181,6 +181,16 @@ export async function processPlexRecentlyAddedCheck(payload: PlexRecentlyAddedPa
|
||||
});
|
||||
|
||||
matchedDownloads++;
|
||||
|
||||
// Trigger metadata match for Audiobookshelf items (only for our downloaded requests)
|
||||
if (backendMode === 'audiobookshelf') {
|
||||
const itemId = match.plexGuid; // plexGuid contains the Audiobookshelf item ID
|
||||
const asin = audiobook.audibleAsin || undefined;
|
||||
const matchInfo = asin ? ` with ASIN ${asin}` : '';
|
||||
await logger?.info(`Triggering metadata match for matched item: ${itemId}${matchInfo}`);
|
||||
const { triggerABSItemMatch } = await import('../services/audiobookshelf/api');
|
||||
await triggerABSItemMatch(itemId, asin);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
await logger?.error(`Failed to match request ${request.id}: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
|
||||
@@ -82,12 +82,13 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
|
||||
|
||||
let downloadPath: string;
|
||||
|
||||
// Try to get download path from qBittorrent if we have the torrent
|
||||
if (downloadHistory.downloadClientId) {
|
||||
// Try to get download path from the appropriate download client
|
||||
if (downloadHistory.torrentHash) {
|
||||
// qBittorrent download
|
||||
try {
|
||||
const { getQBittorrentService } = await import('../integrations/qbittorrent.service');
|
||||
const qbt = await getQBittorrentService();
|
||||
const torrent = await qbt.getTorrent(downloadHistory.downloadClientId);
|
||||
const torrent = await qbt.getTorrent(downloadHistory.torrentHash);
|
||||
const qbPath = `${torrent.save_path}/${torrent.name}`;
|
||||
downloadPath = PathMapper.transform(qbPath, mappingConfig);
|
||||
await logger?.info(
|
||||
@@ -119,10 +120,51 @@ export async function processRetryFailedImports(payload: RetryFailedImportsPaylo
|
||||
(downloadPath !== fallbackPath ? ` → ${downloadPath} (mapped)` : '')
|
||||
);
|
||||
}
|
||||
} else if (downloadHistory.nzbId) {
|
||||
// SABnzbd download
|
||||
try {
|
||||
const { getSABnzbdService } = await import('../integrations/sabnzbd.service');
|
||||
const sabnzbd = await getSABnzbdService();
|
||||
const nzbInfo = await sabnzbd.getNZB(downloadHistory.nzbId);
|
||||
if (nzbInfo && nzbInfo.downloadPath) {
|
||||
downloadPath = PathMapper.transform(nzbInfo.downloadPath, mappingConfig);
|
||||
await logger?.info(
|
||||
`Got download path from SABnzbd for request ${request.id}: ${nzbInfo.downloadPath}` +
|
||||
(downloadPath !== nzbInfo.downloadPath ? ` → ${downloadPath} (mapped)` : '')
|
||||
);
|
||||
} else {
|
||||
await logger?.warn(`NZB ${downloadHistory.nzbId} not found or has no download path for request ${request.id}, falling back to configured path`);
|
||||
|
||||
if (!downloadHistory.torrentName) {
|
||||
await logger?.warn(`No name stored for request ${request.id}, cannot construct fallback path, skipping`);
|
||||
skipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
const downloadDir = await configService.get('download_dir');
|
||||
|
||||
if (!downloadDir) {
|
||||
await logger?.error(`download_dir not configured, cannot retry request ${request.id}, skipping`);
|
||||
skipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
const fallbackPath = `${downloadDir}/${downloadHistory.torrentName}`;
|
||||
downloadPath = PathMapper.transform(fallbackPath, mappingConfig);
|
||||
await logger?.info(
|
||||
`Using fallback download path for request ${request.id}: ${fallbackPath}` +
|
||||
(downloadPath !== fallbackPath ? ` → ${downloadPath} (mapped)` : '')
|
||||
);
|
||||
}
|
||||
} catch (sabnzbdError) {
|
||||
await logger?.warn(`SABnzbd error for request ${request.id}: ${sabnzbdError instanceof Error ? sabnzbdError.message : 'Unknown error'}, skipping`);
|
||||
skipped++;
|
||||
continue;
|
||||
}
|
||||
} else {
|
||||
// No download client ID - use fallback path
|
||||
if (!downloadHistory.torrentName) {
|
||||
await logger?.warn(`No download client ID or torrent name for request ${request.id}, skipping`);
|
||||
await logger?.warn(`No download client ID or name for request ${request.id}, skipping`);
|
||||
skipped++;
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -373,6 +373,16 @@ export async function processScanPlex(payload: ScanPlexPayload): Promise<any> {
|
||||
});
|
||||
|
||||
matchedCount++;
|
||||
|
||||
// Trigger metadata match for Audiobookshelf items (only for our downloaded requests)
|
||||
if (backendMode === 'audiobookshelf') {
|
||||
const itemId = match.plexGuid; // plexGuid contains the Audiobookshelf item ID
|
||||
const asin = audiobook.audibleAsin || undefined;
|
||||
const matchInfo = asin ? ` with ASIN ${asin}` : '';
|
||||
await logger?.info(`Triggering metadata match for matched item: ${itemId}${matchInfo}`);
|
||||
const { triggerABSItemMatch } = await import('../services/audiobookshelf/api');
|
||||
await triggerABSItemMatch(itemId, asin);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
await logger?.error(`Failed to match request ${request.id}: ${error instanceof Error ? error.message : 'Unknown error'}`);
|
||||
|
||||
@@ -96,3 +96,32 @@ export async function searchABSItems(libraryId: string, query: string) {
|
||||
export async function triggerABSScan(libraryId: string) {
|
||||
await absRequest(`/libraries/${libraryId}/scan`, { method: 'POST' });
|
||||
}
|
||||
|
||||
/**
|
||||
* Trigger metadata match for a specific library item
|
||||
* This tells Audiobookshelf to automatically match and populate metadata from providers
|
||||
*
|
||||
* @param itemId - The Audiobookshelf item ID
|
||||
* @param asin - Optional ASIN for direct Audible matching (100% accurate when provided)
|
||||
*/
|
||||
export async function triggerABSItemMatch(itemId: string, asin?: string) {
|
||||
try {
|
||||
const body: any = {
|
||||
provider: 'audible', // Use Audible as the metadata provider
|
||||
};
|
||||
|
||||
// If we have an ASIN, we can do a direct match with 100% confidence
|
||||
if (asin) {
|
||||
body.asin = asin;
|
||||
body.overrideDefaults = true; // Override defaults since we have exact ASIN match
|
||||
}
|
||||
|
||||
await absRequest(`/items/${itemId}/match`, {
|
||||
method: 'POST',
|
||||
body,
|
||||
});
|
||||
} catch (error) {
|
||||
// Don't throw - matching is best-effort, scan should continue even if match fails
|
||||
console.error(`[ABS] Failed to trigger match for item ${itemId}:`, error instanceof Error ? error.message : error);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,786 @@
|
||||
/**
|
||||
* Component: E-book Sidecar Service
|
||||
* Documentation: documentation/integrations/ebook-sidecar.md
|
||||
*/
|
||||
|
||||
import axios, { AxiosError } from 'axios';
|
||||
import * as cheerio from 'cheerio';
|
||||
import fs from 'fs/promises';
|
||||
import path from 'path';
|
||||
import { JobLogger } from '../utils/job-logger';
|
||||
|
||||
export interface EbookDownloadResult {
|
||||
success: boolean;
|
||||
filePath?: string;
|
||||
format?: string;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
const USER_AGENT = 'ReadMeABook/1.0 (Audiobook Automation)';
|
||||
const REQUEST_DELAY_MS = 1500; // 1.5 second delay between requests
|
||||
const DOWNLOAD_TIMEOUT_MS = 60000; // 60 seconds per download attempt
|
||||
const MAX_SLOW_LINK_ATTEMPTS = 5;
|
||||
const MAX_RETRIES = 3;
|
||||
const FLARESOLVERR_TIMEOUT_MS = 60000; // 60 seconds for FlareSolverr requests
|
||||
|
||||
// Debug logging
|
||||
const DEBUG_ENABLED = process.env.LOG_LEVEL === 'debug';
|
||||
|
||||
// In-memory cache for MD5 lookups (prevents re-scraping same ASIN)
|
||||
const md5Cache = new Map<string, string | null>();
|
||||
|
||||
// FlareSolverr types
|
||||
interface FlareSolverrRequest {
|
||||
cmd: 'request.get';
|
||||
url: string;
|
||||
maxTimeout: number;
|
||||
}
|
||||
|
||||
interface FlareSolverrResponse {
|
||||
status: 'ok' | 'error';
|
||||
message: string;
|
||||
solution?: {
|
||||
url: string;
|
||||
status: number;
|
||||
headers: Record<string, string>;
|
||||
response: string;
|
||||
cookies: Array<{ name: string; value: string }>;
|
||||
userAgent: string;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch HTML via FlareSolverr proxy (bypasses Cloudflare)
|
||||
*/
|
||||
async function fetchViaFlareSolverr(
|
||||
targetUrl: string,
|
||||
flaresolverrUrl: string,
|
||||
timeout: number = FLARESOLVERR_TIMEOUT_MS
|
||||
): Promise<string> {
|
||||
const requestBody: FlareSolverrRequest = {
|
||||
cmd: 'request.get',
|
||||
url: targetUrl,
|
||||
maxTimeout: timeout,
|
||||
};
|
||||
|
||||
const response = await axios.post<FlareSolverrResponse>(
|
||||
`${flaresolverrUrl}/v1`,
|
||||
requestBody,
|
||||
{
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
timeout: timeout + 5000, // Extra buffer for FlareSolverr processing
|
||||
}
|
||||
);
|
||||
|
||||
if (response.data.status !== 'ok' || !response.data.solution) {
|
||||
throw new Error(`FlareSolverr error: ${response.data.message}`);
|
||||
}
|
||||
|
||||
if (response.data.solution.status >= 400) {
|
||||
throw new Error(`FlareSolverr returned HTTP ${response.data.solution.status}`);
|
||||
}
|
||||
|
||||
return response.data.solution.response;
|
||||
}
|
||||
|
||||
/**
|
||||
* Unified HTML fetch function - tries FlareSolverr if configured, falls back to direct
|
||||
*/
|
||||
async function fetchHtml(
|
||||
url: string,
|
||||
flaresolverrUrl?: string,
|
||||
logger?: JobLogger
|
||||
): Promise<string> {
|
||||
// Try FlareSolverr first if configured
|
||||
if (flaresolverrUrl) {
|
||||
try {
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Using FlareSolverr for: ${url}`);
|
||||
}
|
||||
const html = await fetchViaFlareSolverr(url, flaresolverrUrl);
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] FlareSolverr returned HTML length: ${html.length}`);
|
||||
}
|
||||
return html;
|
||||
} catch (error) {
|
||||
await logger?.warn(
|
||||
`FlareSolverr failed, falling back to direct request: ${
|
||||
error instanceof Error ? error.message : 'Unknown error'
|
||||
}`
|
||||
);
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] FlareSolverr error:`, error);
|
||||
}
|
||||
// Fall through to direct request
|
||||
}
|
||||
}
|
||||
|
||||
// Direct request (may fail with Cloudflare protection)
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Using direct request for: ${url}`);
|
||||
}
|
||||
const response = await retryRequest(() =>
|
||||
axios.get(url, {
|
||||
headers: { 'User-Agent': USER_AGENT },
|
||||
timeout: 30000,
|
||||
})
|
||||
);
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Direct request returned data length: ${response.data?.length || 0}`);
|
||||
}
|
||||
|
||||
return response.data;
|
||||
}
|
||||
|
||||
/**
|
||||
* Test FlareSolverr connection
|
||||
*/
|
||||
export async function testFlareSolverrConnection(
|
||||
flaresolverrUrl: string
|
||||
): Promise<{ success: boolean; message: string; responseTime?: number }> {
|
||||
const startTime = Date.now();
|
||||
|
||||
try {
|
||||
// Test with a simple request to Anna's Archive homepage
|
||||
const testUrl = 'https://annas-archive.li/';
|
||||
const html = await fetchViaFlareSolverr(testUrl, flaresolverrUrl, 30000);
|
||||
const responseTime = Date.now() - startTime;
|
||||
|
||||
// Verify we got valid HTML
|
||||
if (html && html.includes('Anna') && html.length > 1000) {
|
||||
return {
|
||||
success: true,
|
||||
message: `Connection successful (${responseTime}ms)`,
|
||||
responseTime,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
success: false,
|
||||
message: 'FlareSolverr returned invalid response',
|
||||
};
|
||||
} catch (error) {
|
||||
return {
|
||||
success: false,
|
||||
message: error instanceof Error ? error.message : 'Unknown error',
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Main entry point: Download e-book from Anna's Archive by ASIN
|
||||
*/
|
||||
export async function downloadEbook(
|
||||
asin: string,
|
||||
title: string,
|
||||
author: string,
|
||||
targetDir: string,
|
||||
preferredFormat: string = 'epub',
|
||||
baseUrl: string = 'https://annas-archive.li',
|
||||
logger?: JobLogger,
|
||||
flaresolverrUrl?: string
|
||||
): Promise<EbookDownloadResult> {
|
||||
try {
|
||||
let md5: string | null = null;
|
||||
|
||||
// Log FlareSolverr status
|
||||
if (flaresolverrUrl) {
|
||||
await logger?.info(`Using FlareSolverr at ${flaresolverrUrl}`);
|
||||
}
|
||||
|
||||
// Step 1: Try ASIN search (exact match - best)
|
||||
if (asin) {
|
||||
await logger?.info(`Searching by ASIN: ${asin} (format: ${preferredFormat})...`);
|
||||
md5 = await searchByAsin(asin, preferredFormat, baseUrl, logger, flaresolverrUrl);
|
||||
|
||||
if (md5) {
|
||||
await logger?.info(`Found via ASIN: ${md5}`);
|
||||
} else {
|
||||
await logger?.info(`No results for ASIN, falling back to title + author search...`);
|
||||
}
|
||||
}
|
||||
|
||||
// Step 2: Fallback to title + author search
|
||||
if (!md5) {
|
||||
await logger?.info(`Searching by title + author: "${title}" by ${author}...`);
|
||||
md5 = await searchByTitle(title, author, preferredFormat, baseUrl, logger, flaresolverrUrl);
|
||||
|
||||
if (md5) {
|
||||
await logger?.info(`Found via title search: ${md5}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (!md5) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'No search results found (tried ASIN and title+author)',
|
||||
};
|
||||
}
|
||||
|
||||
await logger?.info(`Found MD5: ${md5}`);
|
||||
|
||||
// Step 3: Get slow download links (no waitlist only)
|
||||
const slowLinks = await getSlowDownloadLinks(md5, baseUrl, logger, flaresolverrUrl);
|
||||
|
||||
if (slowLinks.length === 0) {
|
||||
return {
|
||||
success: false,
|
||||
error: 'No download links available',
|
||||
};
|
||||
}
|
||||
|
||||
await logger?.info(`Found ${slowLinks.length} download link(s)`);
|
||||
|
||||
// Step 4 & 5: Try each slow download link until one succeeds
|
||||
// Note: We determine the actual filename AFTER we know the real format from the download URL
|
||||
const attemptsLimit = Math.min(slowLinks.length, MAX_SLOW_LINK_ATTEMPTS);
|
||||
|
||||
for (let i = 0; i < attemptsLimit; i++) {
|
||||
const slowLink = slowLinks[i];
|
||||
await logger?.info(`Attempting download link ${i + 1}/${attemptsLimit}...`);
|
||||
|
||||
try {
|
||||
// Extract actual download URL from slow download page
|
||||
const extracted = await extractDownloadUrl(
|
||||
slowLink,
|
||||
baseUrl,
|
||||
preferredFormat,
|
||||
logger,
|
||||
flaresolverrUrl
|
||||
);
|
||||
|
||||
if (!extracted) {
|
||||
await logger?.warn(`No download URL found on page ${i + 1}`);
|
||||
await delay(REQUEST_DELAY_MS);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Use the actual format from the download URL, not the preferred format
|
||||
const actualFormat = extracted.format;
|
||||
const sanitizedFilename = sanitizeEbookFilename(title, author, actualFormat);
|
||||
const targetPath = path.join(targetDir, sanitizedFilename);
|
||||
|
||||
// Check if file already exists
|
||||
try {
|
||||
await fs.access(targetPath);
|
||||
await logger?.info(`E-book already exists: ${sanitizedFilename}`);
|
||||
return {
|
||||
success: true,
|
||||
filePath: targetPath,
|
||||
format: actualFormat,
|
||||
};
|
||||
} catch {
|
||||
// File doesn't exist, continue with download
|
||||
}
|
||||
|
||||
await logger?.info(`Downloading from: ${new URL(extracted.url).host} (format: ${actualFormat})`);
|
||||
|
||||
// Download file (direct - no FlareSolverr needed for file servers)
|
||||
const success = await downloadFile(extracted.url, targetPath, logger);
|
||||
|
||||
if (success) {
|
||||
await logger?.info(`E-book downloaded successfully: ${sanitizedFilename}`);
|
||||
return {
|
||||
success: true,
|
||||
filePath: targetPath,
|
||||
format: actualFormat,
|
||||
};
|
||||
}
|
||||
|
||||
await logger?.warn(`Download attempt ${i + 1} failed`);
|
||||
await delay(REQUEST_DELAY_MS);
|
||||
} catch (error) {
|
||||
await logger?.warn(
|
||||
`Download link ${i + 1} error: ${error instanceof Error ? error.message : 'Unknown'}`
|
||||
);
|
||||
await delay(REQUEST_DELAY_MS);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
success: false,
|
||||
error: `All ${attemptsLimit} download attempts failed`,
|
||||
};
|
||||
} catch (error) {
|
||||
const errorMsg = error instanceof Error ? error.message : 'Unknown error';
|
||||
await logger?.error(`E-book download error: ${errorMsg}`);
|
||||
return {
|
||||
success: false,
|
||||
error: errorMsg,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Step 1: Search Anna's Archive by ASIN and extract MD5 hash
|
||||
*/
|
||||
async function searchByAsin(
|
||||
asin: string,
|
||||
format: string,
|
||||
baseUrl: string,
|
||||
logger?: JobLogger,
|
||||
flaresolverrUrl?: string
|
||||
): Promise<string | null> {
|
||||
// Check cache first
|
||||
const cacheKey = `${asin}-${format}`;
|
||||
if (md5Cache.has(cacheKey)) {
|
||||
const cached = md5Cache.get(cacheKey);
|
||||
if (cached) {
|
||||
await logger?.info(`Using cached MD5 for ASIN ${asin}`);
|
||||
}
|
||||
return cached ?? null; // Convert undefined to null
|
||||
}
|
||||
|
||||
try {
|
||||
// Build search URL with ASIN and optional format filter
|
||||
const formatParam = format && format !== 'any' ? `ext=${format}&` : '';
|
||||
const searchUrl = `${baseUrl}/search?${formatParam}q=%22asin:${asin}%22`;
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] ASIN search URL: ${searchUrl}`);
|
||||
}
|
||||
|
||||
const html = await fetchHtml(searchUrl, flaresolverrUrl, logger);
|
||||
const $ = cheerio.load(html);
|
||||
|
||||
// Exclude MD5 links from "Recent downloads" banner (they're in .js-recent-downloads-container)
|
||||
// Only look for actual search result links
|
||||
const searchResultLinks = $('a[href*="/md5/"]').filter((i, elem) => {
|
||||
// Exclude links inside the recent downloads banner
|
||||
return $(elem).closest('.js-recent-downloads-container').length === 0;
|
||||
});
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] ASIN search HTML length: ${html.length}`);
|
||||
// Log the page title to see what we got
|
||||
const pageTitle = $('title').text();
|
||||
console.log(`[EbookScraper] ASIN search page title: ${pageTitle}`);
|
||||
// Count how many md5 links we found (excluding recent downloads)
|
||||
const allMd5Links = $('a[href*="/md5/"]').length;
|
||||
console.log(`[EbookScraper] Total MD5 links on page: ${allMd5Links}, search results only: ${searchResultLinks.length}`);
|
||||
}
|
||||
|
||||
// Extract MD5 from first search result link
|
||||
const firstResult = searchResultLinks.first();
|
||||
const href = firstResult.attr('href');
|
||||
|
||||
if (DEBUG_ENABLED && firstResult.length > 0) {
|
||||
// Try to get the text/title of the first result
|
||||
const resultText = firstResult.text().trim().substring(0, 100);
|
||||
const parentText = firstResult.parent().text().trim().substring(0, 100);
|
||||
console.log(`[EbookScraper] First result link text: "${resultText}"`);
|
||||
console.log(`[EbookScraper] First result parent text: "${parentText}"`);
|
||||
}
|
||||
|
||||
if (!href) {
|
||||
await logger?.warn(`No search results found for ASIN: ${asin}`);
|
||||
md5Cache.set(cacheKey, null);
|
||||
return null;
|
||||
}
|
||||
|
||||
// Extract MD5 from href (e.g., "/md5/3b6f9c0f..." -> "3b6f9c0f...")
|
||||
const md5Match = href.match(/\/md5\/([a-f0-9]+)/);
|
||||
const md5 = md5Match ? md5Match[1] : null;
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Extracted MD5 from ASIN search: ${md5}`);
|
||||
}
|
||||
|
||||
// Cache result
|
||||
md5Cache.set(cacheKey, md5);
|
||||
|
||||
await delay(REQUEST_DELAY_MS);
|
||||
return md5;
|
||||
} catch (error) {
|
||||
await logger?.error(
|
||||
`Search failed: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
);
|
||||
md5Cache.set(cacheKey, null);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Search Anna's Archive by title and author (fallback method)
|
||||
*/
|
||||
async function searchByTitle(
|
||||
title: string,
|
||||
author: string,
|
||||
format: string,
|
||||
baseUrl: string,
|
||||
logger?: JobLogger,
|
||||
flaresolverrUrl?: string
|
||||
): Promise<string | null> {
|
||||
// Check cache first
|
||||
const cacheKey = `title-${title}-${author}-${format}`.toLowerCase();
|
||||
if (md5Cache.has(cacheKey)) {
|
||||
const cached = md5Cache.get(cacheKey);
|
||||
if (cached) {
|
||||
await logger?.info(`Using cached MD5 for title search`);
|
||||
}
|
||||
return cached ?? null;
|
||||
}
|
||||
|
||||
try {
|
||||
// Build search URL using specific term types for author and title (more accurate than raw query)
|
||||
const encodedAuthor = encodeURIComponent(author);
|
||||
const encodedTitle = encodeURIComponent(title);
|
||||
|
||||
// Use Anna's Archive advanced search with specific term types
|
||||
let searchUrl = `${baseUrl}/search?termtype_1=author&termval_1=${encodedAuthor}&termtype_2=title&termval_2=${encodedTitle}`;
|
||||
|
||||
// Add format filter if not 'any'
|
||||
if (format && format !== 'any') {
|
||||
searchUrl += `&ext=${format}`;
|
||||
}
|
||||
|
||||
// Add content type filters (books only, all fiction/nonfiction/unknown)
|
||||
searchUrl += '&content=book_nonfiction&content=book_fiction&content=book_unknown';
|
||||
|
||||
// Add language filter (English)
|
||||
searchUrl += '&lang=en';
|
||||
|
||||
// Empty raw query (we're using specific terms instead)
|
||||
searchUrl += '&q=';
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Title search URL: ${searchUrl}`);
|
||||
}
|
||||
|
||||
const html = await fetchHtml(searchUrl, flaresolverrUrl, logger);
|
||||
const $ = cheerio.load(html);
|
||||
|
||||
// Exclude MD5 links from "Recent downloads" banner (they're in .js-recent-downloads-container)
|
||||
const searchResultLinks = $('a[href*="/md5/"]').filter((i, elem) => {
|
||||
return $(elem).closest('.js-recent-downloads-container').length === 0;
|
||||
});
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
const allMd5Links = $('a[href*="/md5/"]').length;
|
||||
console.log(`[EbookScraper] Title search: Total MD5 links: ${allMd5Links}, search results only: ${searchResultLinks.length}`);
|
||||
}
|
||||
|
||||
// Extract MD5 from first search result link
|
||||
const firstResult = searchResultLinks.first();
|
||||
const href = firstResult.attr('href');
|
||||
|
||||
if (!href) {
|
||||
await logger?.warn(`No search results found for title: "${title}" by ${author}`);
|
||||
md5Cache.set(cacheKey, null);
|
||||
return null;
|
||||
}
|
||||
|
||||
// Extract MD5 from href
|
||||
const md5Match = href.match(/\/md5\/([a-f0-9]+)/);
|
||||
const md5 = md5Match ? md5Match[1] : null;
|
||||
|
||||
// Cache result
|
||||
md5Cache.set(cacheKey, md5);
|
||||
|
||||
await delay(REQUEST_DELAY_MS);
|
||||
return md5;
|
||||
} catch (error) {
|
||||
await logger?.error(
|
||||
`Title search failed: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
);
|
||||
md5Cache.set(cacheKey, null);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Step 3: Get slow download links from MD5 page (no waitlist only)
|
||||
*/
|
||||
async function getSlowDownloadLinks(
|
||||
md5: string,
|
||||
baseUrl: string,
|
||||
logger?: JobLogger,
|
||||
flaresolverrUrl?: string
|
||||
): Promise<string[]> {
|
||||
try {
|
||||
const md5Url = `${baseUrl}/md5/${md5}`;
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Fetching MD5 page: ${md5Url}`);
|
||||
}
|
||||
|
||||
const html = await fetchHtml(md5Url, flaresolverrUrl, logger);
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] HTML length: ${html.length}`);
|
||||
console.log(`[EbookScraper] HTML preview (first 500 chars): ${html.substring(0, 500)}`);
|
||||
// Check if we got a Cloudflare challenge page
|
||||
if (html.includes('challenge-running') || html.includes('cf-browser-verification')) {
|
||||
console.log(`[EbookScraper] WARNING: Appears to be Cloudflare challenge page!`);
|
||||
}
|
||||
}
|
||||
|
||||
const $ = cheerio.load(html);
|
||||
const slowLinks: string[] = [];
|
||||
|
||||
// Debug: count all links
|
||||
if (DEBUG_ENABLED) {
|
||||
const allLinks = $('a').length;
|
||||
const slowDownloadLinks = $('a[href*="/slow_download/"]').length;
|
||||
const slowDownloadLinksAlt = $('a[href*="slow_download"]').length;
|
||||
console.log(`[EbookScraper] Total links on page: ${allLinks}`);
|
||||
console.log(`[EbookScraper] Links with /slow_download/: ${slowDownloadLinks}`);
|
||||
console.log(`[EbookScraper] Links with slow_download (no slashes): ${slowDownloadLinksAlt}`);
|
||||
|
||||
// Log all href patterns to see what we're dealing with
|
||||
const hrefPatterns: string[] = [];
|
||||
$('a[href]').each((i, elem) => {
|
||||
const href = $(elem).attr('href') || '';
|
||||
if (href.includes('download') || href.includes('slow')) {
|
||||
hrefPatterns.push(href.substring(0, 100));
|
||||
}
|
||||
});
|
||||
if (hrefPatterns.length > 0) {
|
||||
console.log(`[EbookScraper] Download-related hrefs found:`, hrefPatterns.slice(0, 10));
|
||||
}
|
||||
}
|
||||
|
||||
// Find all slow download links
|
||||
$('a[href*="/slow_download/"]').each((i, elem) => {
|
||||
const linkText = $(elem).text().toLowerCase();
|
||||
// Check parent element text too - "no waitlist" may be outside the <a> tag
|
||||
// e.g., <li><a>Slow Partner Server #5</a> (no waitlist, but can be very slow)</li>
|
||||
const parentText = $(elem).parent().text().toLowerCase();
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
const href = $(elem).attr('href');
|
||||
console.log(`[EbookScraper] Found slow_download link: href="${href}", linkText="${linkText.substring(0, 30)}", parentText="${parentText.substring(0, 60)}"`);
|
||||
}
|
||||
|
||||
// Check for "no waitlist" in either the link text or parent text
|
||||
if (linkText.includes('no waitlist') || parentText.includes('no waitlist')) {
|
||||
const href = $(elem).attr('href');
|
||||
if (href) {
|
||||
// Convert relative URL to absolute
|
||||
const fullUrl = href.startsWith('http') ? href : `${baseUrl}${href}`;
|
||||
slowLinks.push(fullUrl);
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Added slow link (no waitlist): ${fullUrl}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Total slow links found: ${slowLinks.length}`);
|
||||
}
|
||||
|
||||
await delay(REQUEST_DELAY_MS);
|
||||
return slowLinks;
|
||||
} catch (error) {
|
||||
await logger?.error(
|
||||
`Failed to get slow links: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
);
|
||||
if (DEBUG_ENABLED) {
|
||||
console.log(`[EbookScraper] Error getting slow links:`, error);
|
||||
}
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
interface ExtractedDownload {
|
||||
url: string;
|
||||
format: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Step 4: Extract actual download URL from slow download page
|
||||
* IMPORTANT: Supports dynamic file formats (not hardcoded to .epub)
|
||||
* Returns both URL and detected format
|
||||
*/
|
||||
async function extractDownloadUrl(
|
||||
slowDownloadUrl: string,
|
||||
baseUrl: string,
|
||||
format: string,
|
||||
logger?: JobLogger,
|
||||
flaresolverrUrl?: string
|
||||
): Promise<ExtractedDownload | null> {
|
||||
try {
|
||||
const html = await fetchHtml(slowDownloadUrl, flaresolverrUrl, logger);
|
||||
const $ = cheerio.load(html);
|
||||
|
||||
// Build regex pattern based on format
|
||||
// If format is 'any', match any common e-book extension
|
||||
let pattern: RegExp;
|
||||
if (format === 'any') {
|
||||
pattern = /(https?:\/\/[^\s]+\.(epub|pdf|mobi|azw3|djvu|fb2))/i;
|
||||
} else {
|
||||
pattern = new RegExp(`(https?:\\/\\/[^\\s]+\\.${format})`, 'i');
|
||||
}
|
||||
|
||||
let downloadUrl: string | null = null;
|
||||
let detectedFormat: string | null = null;
|
||||
|
||||
// Method 1: Search in pre/code blocks first (most reliable)
|
||||
$('pre, code').each((i, elem) => {
|
||||
const text = $(elem).text();
|
||||
const match = text.match(pattern);
|
||||
if (match) {
|
||||
downloadUrl = match[1];
|
||||
// Extract format from URL
|
||||
const formatMatch = downloadUrl.match(/\.(epub|pdf|mobi|azw3|djvu|fb2)$/i);
|
||||
detectedFormat = formatMatch ? formatMatch[1].toLowerCase() : null;
|
||||
return false; // Break loop
|
||||
}
|
||||
});
|
||||
|
||||
// Method 2: Search entire body text as fallback
|
||||
if (!downloadUrl) {
|
||||
const bodyText = $('body').text();
|
||||
const match = bodyText.match(pattern);
|
||||
if (match) {
|
||||
downloadUrl = match[1];
|
||||
// Extract format from URL
|
||||
const formatMatch = downloadUrl.match(/\.(epub|pdf|mobi|azw3|djvu|fb2)$/i);
|
||||
detectedFormat = formatMatch ? formatMatch[1].toLowerCase() : null;
|
||||
}
|
||||
}
|
||||
|
||||
await delay(REQUEST_DELAY_MS);
|
||||
|
||||
if (!downloadUrl || !detectedFormat) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return { url: downloadUrl, format: detectedFormat };
|
||||
} catch (error) {
|
||||
await logger?.error(
|
||||
`Failed to extract download URL: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Step 5: Download file from URL with streaming (handles large files)
|
||||
*/
|
||||
async function downloadFile(
|
||||
url: string,
|
||||
targetPath: string,
|
||||
logger?: JobLogger
|
||||
): Promise<boolean> {
|
||||
try {
|
||||
const response = await axios.get(url, {
|
||||
responseType: 'stream',
|
||||
timeout: DOWNLOAD_TIMEOUT_MS,
|
||||
headers: { 'User-Agent': USER_AGENT },
|
||||
maxRedirects: 5,
|
||||
});
|
||||
|
||||
// Stream to file
|
||||
const writer = require('fs').createWriteStream(targetPath);
|
||||
|
||||
response.data.pipe(writer);
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
writer.on('finish', () => {
|
||||
writer.close();
|
||||
resolve(true);
|
||||
});
|
||||
|
||||
writer.on('error', (error: Error) => {
|
||||
writer.close();
|
||||
// Clean up partial file
|
||||
fs.unlink(targetPath).catch(() => {});
|
||||
reject(error);
|
||||
});
|
||||
|
||||
// Set timeout
|
||||
const timeout = setTimeout(() => {
|
||||
writer.close();
|
||||
fs.unlink(targetPath).catch(() => {});
|
||||
reject(new Error('Download timeout'));
|
||||
}, DOWNLOAD_TIMEOUT_MS);
|
||||
|
||||
writer.on('finish', () => clearTimeout(timeout));
|
||||
writer.on('error', () => clearTimeout(timeout));
|
||||
});
|
||||
} catch (error) {
|
||||
// Clean up partial file
|
||||
try {
|
||||
await fs.unlink(targetPath);
|
||||
} catch {}
|
||||
|
||||
await logger?.error(
|
||||
`Download failed: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Sanitize filename for e-book
|
||||
* Format: "[Title] - [Author].[format]"
|
||||
* Note: format should be the actual detected format (e.g., 'pdf', 'epub'), not 'any'
|
||||
*/
|
||||
function sanitizeEbookFilename(title: string, author: string, format: string): string {
|
||||
const sanitize = (str: string): string => {
|
||||
return str
|
||||
.replace(/[<>:"/\\|?*]/g, '') // Remove invalid chars
|
||||
.replace(/\s+/g, ' ') // Collapse spaces
|
||||
.trim()
|
||||
.slice(0, 100); // Limit length
|
||||
};
|
||||
|
||||
const cleanTitle = sanitize(title);
|
||||
const cleanAuthor = sanitize(author);
|
||||
// Use the actual format passed in (should already be the detected format from URL)
|
||||
const cleanFormat = format.toLowerCase();
|
||||
|
||||
return `${cleanTitle} - ${cleanAuthor}.${cleanFormat}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Retry HTTP request with exponential backoff
|
||||
*/
|
||||
async function retryRequest<T>(
|
||||
requestFn: () => Promise<T>,
|
||||
retries: number = MAX_RETRIES
|
||||
): Promise<T> {
|
||||
let lastError: Error | null = null;
|
||||
|
||||
for (let attempt = 0; attempt < retries; attempt++) {
|
||||
try {
|
||||
return await requestFn();
|
||||
} catch (error) {
|
||||
lastError = error instanceof Error ? error : new Error('Unknown error');
|
||||
|
||||
// Only retry on 5xx errors or network errors
|
||||
const isRetryable =
|
||||
error instanceof AxiosError &&
|
||||
(error.code === 'ECONNRESET' ||
|
||||
error.code === 'ETIMEDOUT' ||
|
||||
(error.response && error.response.status >= 500));
|
||||
|
||||
if (!isRetryable || attempt === retries - 1) {
|
||||
throw lastError;
|
||||
}
|
||||
|
||||
// Exponential backoff: 1s, 2s, 4s
|
||||
const delayMs = 1000 * Math.pow(2, attempt);
|
||||
await delay(delayMs);
|
||||
}
|
||||
}
|
||||
|
||||
throw lastError || new Error('Request failed after retries');
|
||||
}
|
||||
|
||||
/**
|
||||
* Delay helper
|
||||
*/
|
||||
function delay(ms: number): Promise<void> {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear MD5 cache (useful for testing)
|
||||
*/
|
||||
export function clearMd5Cache(): void {
|
||||
md5Cache.clear();
|
||||
}
|
||||
@@ -85,7 +85,7 @@ export async function deleteRequest(
|
||||
// 2. Handle downloads & seeding
|
||||
const downloadHistory = request.downloadHistory[0];
|
||||
|
||||
if (downloadHistory && downloadHistory.downloadClientId && downloadHistory.indexerName) {
|
||||
if (downloadHistory && downloadHistory.indexerName) {
|
||||
try {
|
||||
// Get indexer seeding configuration
|
||||
const { getConfigService } = await import('./config.service');
|
||||
@@ -100,16 +100,18 @@ export async function deleteRequest(
|
||||
);
|
||||
}
|
||||
|
||||
// Get torrent from qBittorrent
|
||||
// Handle based on download client type (check which ID is present)
|
||||
if (downloadHistory.torrentHash) {
|
||||
// qBittorrent download
|
||||
const { getQBittorrentService } = await import('../integrations/qbittorrent.service');
|
||||
const qbt = await getQBittorrentService();
|
||||
|
||||
let torrent;
|
||||
try {
|
||||
torrent = await qbt.getTorrent(downloadHistory.downloadClientId);
|
||||
torrent = await qbt.getTorrent(downloadHistory.torrentHash);
|
||||
} catch (error) {
|
||||
// Torrent not found in qBittorrent (already removed)
|
||||
console.log(`[RequestDelete] Torrent ${downloadHistory.downloadClientId} not found in qBittorrent, skipping`);
|
||||
console.log(`[RequestDelete] Torrent ${downloadHistory.torrentHash} not found in qBittorrent, skipping`);
|
||||
}
|
||||
|
||||
if (torrent) {
|
||||
@@ -128,7 +130,7 @@ export async function deleteRequest(
|
||||
console.log(
|
||||
`[RequestDelete] Deleting incomplete download: ${torrent.name}`
|
||||
);
|
||||
await qbt.deleteTorrent(downloadHistory.downloadClientId, true);
|
||||
await qbt.deleteTorrent(downloadHistory.torrentHash, true);
|
||||
torrentsRemoved++;
|
||||
} else {
|
||||
// Check if seeding requirement is met
|
||||
@@ -143,7 +145,7 @@ export async function deleteRequest(
|
||||
actualSeedingTime / 60
|
||||
)}/${seedingConfig.seedingTimeMinutes} minutes)`
|
||||
);
|
||||
await qbt.deleteTorrent(downloadHistory.downloadClientId, true);
|
||||
await qbt.deleteTorrent(downloadHistory.torrentHash, true);
|
||||
torrentsRemoved++;
|
||||
} else {
|
||||
// Still needs seeding - keep for cleanup job
|
||||
@@ -155,12 +157,27 @@ export async function deleteRequest(
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if (downloadHistory.nzbId) {
|
||||
// SABnzbd download - no seeding concept for Usenet
|
||||
try {
|
||||
const { getSABnzbdService } = await import('../integrations/sabnzbd.service');
|
||||
const sabnzbd = await getSABnzbdService();
|
||||
|
||||
// Try to delete the NZB from SABnzbd (might already be completed/removed)
|
||||
await sabnzbd.deleteNZB(downloadHistory.nzbId, true);
|
||||
console.log(`[RequestDelete] Deleted NZB ${downloadHistory.nzbId} from SABnzbd`);
|
||||
torrentsRemoved++;
|
||||
} catch (error) {
|
||||
// NZB not found or already removed
|
||||
console.log(`[RequestDelete] NZB ${downloadHistory.nzbId} not found in SABnzbd, skipping`);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(
|
||||
`[RequestDelete] Error handling torrent for request ${requestId}:`,
|
||||
`[RequestDelete] Error handling download for request ${requestId}:`,
|
||||
error instanceof Error ? error.message : 'Unknown error'
|
||||
);
|
||||
// Continue with deletion even if torrent handling fails
|
||||
// Continue with deletion even if download handling fails
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -22,6 +22,7 @@ export interface AudiobookMatchInput {
|
||||
|
||||
export interface AudiobookMatchResult {
|
||||
plexGuid: string;
|
||||
plexRatingKey: string | null;
|
||||
title: string;
|
||||
author: string;
|
||||
}
|
||||
@@ -82,6 +83,7 @@ export async function findPlexMatch(
|
||||
},
|
||||
select: {
|
||||
plexGuid: true,
|
||||
plexRatingKey: true,
|
||||
title: true,
|
||||
author: true,
|
||||
asin: true, // Include ASIN field for direct matching
|
||||
@@ -297,6 +299,9 @@ export async function enrichAudiobooksWithMatches(
|
||||
id: true,
|
||||
audibleAsin: true,
|
||||
requests: {
|
||||
where: {
|
||||
deletedAt: null, // Only include active (non-deleted) requests
|
||||
},
|
||||
select: {
|
||||
id: true,
|
||||
status: true,
|
||||
|
||||
@@ -9,6 +9,7 @@ import axios from 'axios';
|
||||
import { createJobLogger, JobLogger } from './job-logger';
|
||||
import { tagMultipleFiles, checkFfmpegAvailable } from './metadata-tagger';
|
||||
import { prisma } from '../db';
|
||||
import { downloadEbook } from '../services/ebook-scraper';
|
||||
|
||||
export interface AudiobookMetadata {
|
||||
title: string;
|
||||
@@ -261,6 +262,56 @@ export class FileOrganizer {
|
||||
}
|
||||
}
|
||||
|
||||
// E-book sidecar: Download accompanying e-book if enabled
|
||||
try {
|
||||
const ebookConfig = await prisma.configuration.findUnique({
|
||||
where: { key: 'ebook_sidecar_enabled' },
|
||||
});
|
||||
|
||||
const ebookEnabled = ebookConfig?.value === 'true';
|
||||
|
||||
if (ebookEnabled) {
|
||||
await logger?.info(`E-book sidecar enabled, searching for e-book...`);
|
||||
|
||||
// Get configuration
|
||||
const [formatConfig, baseUrlConfig, flaresolverrConfig] = await Promise.all([
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_preferred_format' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_base_url' } }),
|
||||
prisma.configuration.findUnique({ where: { key: 'ebook_sidecar_flaresolverr_url' } }),
|
||||
]);
|
||||
|
||||
const preferredFormat = formatConfig?.value || 'epub';
|
||||
const baseUrl = baseUrlConfig?.value || 'https://annas-archive.li';
|
||||
const flaresolverrUrl = flaresolverrConfig?.value || undefined;
|
||||
|
||||
// Download e-book (will try ASIN first, then fall back to title+author)
|
||||
const ebookResult = await downloadEbook(
|
||||
audiobook.asin || '', // ASIN (optional - will fallback to title+author if empty)
|
||||
audiobook.title,
|
||||
audiobook.author,
|
||||
targetPath, // Same directory as audiobook
|
||||
preferredFormat,
|
||||
baseUrl,
|
||||
logger ?? undefined,
|
||||
flaresolverrUrl
|
||||
);
|
||||
|
||||
if (ebookResult.success && ebookResult.filePath) {
|
||||
await logger?.info(`E-book downloaded: ${path.basename(ebookResult.filePath)}`);
|
||||
result.filesMovedCount++;
|
||||
} else {
|
||||
await logger?.warn(`E-book download failed: ${ebookResult.error}`);
|
||||
result.errors.push(`E-book sidecar: ${ebookResult.error}`);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
await logger?.warn(
|
||||
`E-book sidecar error: ${error instanceof Error ? error.message : 'Unknown error'}`
|
||||
);
|
||||
result.errors.push('E-book sidecar failed');
|
||||
// Don't throw - audiobook organization continues
|
||||
}
|
||||
|
||||
result.targetPath = targetPath;
|
||||
result.success = true;
|
||||
|
||||
|
||||
@@ -350,14 +350,37 @@ export class RankingAlgorithm {
|
||||
const beforeWords = extractWords(beforeTitle, stopWords);
|
||||
|
||||
// Title is complete if:
|
||||
// 1. No significant words before it (not "This Inevitable Ruin" + "Dungeon Crawler Carl")
|
||||
// 1. Acceptable prefix (no words, OR structured metadata like "Author - Series - ")
|
||||
// 2. Followed by clear metadata markers (not "'s Secret" or " Is Watching")
|
||||
const metadataMarkers = [' by ', ' - ', ' [', ' (', ' {', ' :', ','];
|
||||
const hasNoWordsPrefix = beforeWords.length === 0;
|
||||
const hasMetadataSuffix = afterTitle === '' ||
|
||||
metadataMarkers.some(marker => afterTitle.startsWith(marker));
|
||||
|
||||
const isCompleteTitle = hasNoWordsPrefix && hasMetadataSuffix;
|
||||
// Check prefix validity:
|
||||
// - No words before = clean match
|
||||
// - Title preceded by separator (` - `, `: `) = structured metadata (Author - Series - Title)
|
||||
// - Author name in prefix = author attribution before title
|
||||
const hasNoWordsPrefix = beforeWords.length === 0;
|
||||
|
||||
// Check if title is immediately preceded by a metadata separator
|
||||
// This handles "Author - Series - 01 - Title" patterns
|
||||
const precedingText = beforeTitle.trimEnd();
|
||||
const titlePrecededBySeparator =
|
||||
precedingText.endsWith('-') ||
|
||||
precedingText.endsWith(':') ||
|
||||
precedingText.endsWith('—');
|
||||
|
||||
// Check if author name appears in the prefix
|
||||
// This handles "Author Name - Title" patterns
|
||||
const authorInPrefix = requestAuthor.length > 2 &&
|
||||
beforeTitle.includes(requestAuthor);
|
||||
|
||||
const hasAcceptablePrefix =
|
||||
hasNoWordsPrefix ||
|
||||
titlePrecededBySeparator ||
|
||||
authorInPrefix;
|
||||
|
||||
const isCompleteTitle = hasAcceptablePrefix && hasMetadataSuffix;
|
||||
|
||||
if (isCompleteTitle) {
|
||||
// Complete title match → full points
|
||||
|
||||
Reference in New Issue
Block a user