From be9080d0a5c9c32b9fd7581f402647c2157959e4 Mon Sep 17 00:00:00 2001 From: Dmitry Kokorin Date: Wed, 23 Sep 2026 00:05:38 +0300 Subject: [PATCH] AGENTS.md: sec05 lessons (user oracle table, RU-title non-calques, mandatory rutitles diff, libgen mislinked-file trap, collection TOC verify) + sources 29-30 (fantlab, predanie.ru/LJ) + negatives gate v2 --- AGENTS.md | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 57 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 3ce846a..41c5e88 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,8 +7,9 @@ find the **Russian edition(s)**: correct Russian title, publisher, year, transla libgen download page (if exists), and РГБ (RSL) bibliographic record (if exists). We work **section by section**. Sections **01–04 are COMPLETE** (01: 38✅/2🔶/15❌; 02: 23✅/5🔶/29❌; -03: 38✅/30❌; 04: 12✅/1🔶/26❌). Sections 05–07 not started; 08–12 pending scope decision. -Sections 1–3, 5–7 are skeletons for later. +03: 38✅/30❌; 04: 12✅/1🔶/26❌). **Section 05 COMPLETE (2026-09-22): 17✅/12❌** — research + +RU downloads; 7 rescues came from user tips (37% of the initial ❌ list was wrong — see Lessons). +Section 06–07 pending; 08–12 scope decision pending. Sections 1–3, 5–7 are skeletons for later. ## Repository layout @@ -413,6 +414,52 @@ sections/0N-/ # one .md per BOOK (chapter-level list items are merged int -260-1, ред. Клименко А., сост. И. Надь) = «Сборник статей по античной мифологии» — a compilation, NOT *Essays on a Science of Mythology* (item 03 stays ❌). +### 29. fantlab.ru — structured work pages for RU translations (added 2026-09-22, sec05) +- `fantlab.ru/work` — server-side HTML, curl-able (CRW for the publications block). Blocks: + «Перевод на русский» (translator + year + N изд.), «Издания» (period/language/translator filters), + «Входит в» (anthologies — e.g. «Бригантина 72-73»). Find work id via SearXNG + `site:fantlab.ru ""`. +- Value: **Soviet-era / adventure / ethnographic / sci-fi books** that RSL and libgen miss + (found: van der Post «Потерянный мир Калахари» 1973, пер. Л. Деревянкина — sec05 #18; + Bjerrre «Затерянный мир Калахари» Мысль 1964 = homonym trap). Use for pre-1991 items. + +### 30. predanie.ru + livejournal — older-edition pages & community oracles (added 2026-09-22, sec05) +- `predanie.ru/book/-/` — curl-able book pages for older RU editions (Ладомир etc.), + GOST-like specs. Find via SearXNG `site:predanie.ru ""`. +- **livejournal.com** reviews («Книжная полка культуролога» etc.) — CRW-rendered; often cite + BOTH RU title variants of one book (Benedict «Модели культуры»/«Образцы культуры»). Good + oracle for title-variant discovery; not a bibliographic source. +- **jungnewyork.com** — EN publisher/institute pages (per-book: TOC, praise, EN metadata); + NOT a RU source (book_mi.shtml = Adams EN page). + +## Lessons from sec05 (2026-09-22) — what made the 7 rescues possible / what to do next time + +1. **User oracle table = highest-ROI step.** Initial ❌ list (19) was 37% wrong; user checked the + RU-title-guess table against OZON/labirint memory and rescued 7. **Convention: at section end, + present the ❌ table (author, EN title, guessed RU author/title, channels probed) to the user** + before finalizing MISSING.md. +2. **RU titles are NOT calques** — the actual titles found: «Очерки сравнительного РЕЛИГИОВЕДЕНИЯ» + (not «религии»), «Аспекты мифа» (translated from FRENCH *Aspects du mythe*, not from EN *Myth and + Reality*), «Модели культуры» (not «Узоры/Образцы культуры»), «Потерянный мир Калахари» (the + «Затерянный» variant is a DIFFERENT book — Bjerrre). Always probe 2–3 RU title variants; + for authors with FR/DE originals (Eliade, van Gennep) also try the FR/DE-based RU title. +3. **rutitles diff was SKIPPED for sec05 and would have caught 3/7 rescues** («Аспекты мифа», + «Культурный комплекс», «На пороге инициации» were all in data/ru-titles.jsonl from the cogito + crawl). **Mandatory: `tools/rutitles.py both ""` for every item before ❌** (added to + the negatives gate below). +4. **flib authorall for prolific RU authors** (Элиаде a/2943 = 49 books) caught «Тайные общества» + (sec05 #13) that libgen title search missed. Always run flib author sweep for authors with + >5 RU books. +5. **libgen mislinked-file trap**: edition 141685581 «На пороге инициации» carried an ePubLibre + Spanish Lermontov file. Before downloading, check the file `locator`/title via + `json.php?object=f&addkeys=*&ids=`; for recent books prefer flib (cleaner metadata). +6. **Collection-inclusion questions** (#27/#28): download the smallest fb2, parse the TOC + (`
`), grep for the EN book title + RU calque. «Символ и ритуал» (Наука + 1983) CONTAINS «Ритуальный процесс» (full text, core section) but NOT «Лес символов» + (citations only). +7. **flib pdf download**: `/b//pdf` may return an HTML stub; `/b//download` → 302 → + direct `static.flibusta.is:443/b.usr/.` URL — curl that directly. + ## Catalog caching convention Processed/scraper-unfriendly catalogs are cached under `data/catalogs/` (committed, with source @@ -516,8 +563,14 @@ not downloadable). Legend for ISBN columns: **✅** = ISBN found; **⚠** = publ with a one-line totals row (DL EN / DL RU file counts). Cross-check the DL columns against the section's `downloads/-*/MANIFEST.md` so the two never disagree. -Negatives gate (before ❌): author sweep (RSL + libgen + cogito + flibusta) + one title-level -SearXNG + livelib author page + one shop/retail check. Log every RU query in MISSING.md. +Negatives gate (before ❌) — **v2 (2026-09-22, after sec05 rescues)**: +author sweep (RSL + libgen + cogito + flibusta **authorall** for prolific RU authors) + +**`tools/rutitles.py both ""` diff (MANDATORY)** + 2–3 RU title variants +(calque + SearXNG/web_search `"" перевод/на русском` + FR/DE-original variant for +Eliade/van Gennep-type authors) + livelib author page + one shop/retail check (cogito person +page for identity) + **fantlab.ru for pre-1991/adventure/ethnographic** + predanie.ru for older +editions + **user oracle table at section end** (see Lessons). Log every RU query in MISSING.md. +Before downloading from libgen: verify file locator/title (mislinked-file trap). Special cases: - **Jung's CW volumes:** translated and widely available — don't deep-search; note the RU