AGENTS.md: sec05 lessons (user oracle table, RU-title non-calques, mandatory rutitles diff, libgen mislinked-file trap, collection TOC verify) + sources 29-30 (fantlab, predanie.ru/LJ) + negatives gate v2

This commit is contained in:
Dmitry Kokorin 2026-09-23 00:05:38 +03:00
parent 9b492e21c2
commit be9080d0a5

View file

@ -7,8 +7,9 @@ find the **Russian edition(s)**: correct Russian title, publisher, year, transla
libgen download page (if exists), and РГБ (RSL) bibliographic record (if exists).
We work **section by section**. Sections **01–04 are COMPLETE** (01: 38✅/2🔶/15❌; 02: 23✅/5🔶/29❌;
03: 38✅/30❌; 04: 12✅/1🔶/26❌). Sections 05–07 not started; 08–12 pending scope decision.
Sections 1–3, 5–7 are skeletons for later.
03: 38✅/30❌; 04: 12✅/1🔶/26❌). **Section 05 COMPLETE (2026-09-22): 17✅/12❌** — research +
RU downloads; 7 rescues came from user tips (37% of the initial ❌ list was wrong — see Lessons).
Section 06–07 pending; 08–12 scope decision pending. Sections 1–3, 5–7 are skeletons for later.
## Repository layout
@ -413,6 +414,52 @@ sections/0N-<name>/ # one .md per BOOK (chapter-level list items are merged int
-260-1, ред. Клименко А., сост. И. Надь) = «Сборник статей по античной мифологии» — a
compilation, NOT *Essays on a Science of Mythology* (item 03 stays ❌).
### 29. fantlab.ru — structured work pages for RU translations (added 2026-09-22, sec05)
- `fantlab.ru/work<id>` — server-side HTML, curl-able (CRW for the publications block). Blocks:
«Перевод на русский» (translator + year + N изд.), «Издания» (period/language/translator filters),
«Входит в» (anthologies — e.g. «Бригантина 72-73»). Find work id via SearXNG
`site:fantlab.ru "<RU title>"`.
- Value: **Soviet-era / adventure / ethnographic / sci-fi books** that RSL and libgen miss
(found: van der Post «Потерянный мир Калахари» 1973, пер. Л. Деревянкина — sec05 #18;
Bjerrre «Затерянный мир Калахари» Мысль 1964 = homonym trap). Use for pre-1991 items.
### 30. predanie.ru + livejournal — older-edition pages & community oracles (added 2026-09-22, sec05)
- `predanie.ru/book/<id>-<slug>/` — curl-able book pages for older RU editions (Ладомир etc.),
GOST-like specs. Find via SearXNG `site:predanie.ru "<RU title>"`.
- **livejournal.com** reviews («Книжная полка культуролога» etc.) — CRW-rendered; often cite
BOTH RU title variants of one book (Benedict «Модели культуры»/«Образцы культуры»). Good
oracle for title-variant discovery; not a bibliographic source.
- **jungnewyork.com** — EN publisher/institute pages (per-book: TOC, praise, EN metadata);
NOT a RU source (book_mi.shtml = Adams EN page).
## Lessons from sec05 (2026-09-22) — what made the 7 rescues possible / what to do next time
1. **User oracle table = highest-ROI step.** Initial ❌ list (19) was 37% wrong; user checked the
RU-title-guess table against OZON/labirint memory and rescued 7. **Convention: at section end,
present the ❌ table (author, EN title, guessed RU author/title, channels probed) to the user**
before finalizing MISSING.md.
2. **RU titles are NOT calques** — the actual titles found: «Очерки сравнительного РЕЛИГИОВЕДЕНИЯ»
(not «религии»), «Аспекты мифа» (translated from FRENCH *Aspects du mythe*, not from EN *Myth and
Reality*), «Модели культуры» (not «Узоры/Образцы культуры»), «Потерянный мир Калахари» (the
«Затерянный» variant is a DIFFERENT book — Bjerrre). Always probe 2–3 RU title variants;
for authors with FR/DE originals (Eliade, van Gennep) also try the FR/DE-based RU title.
3. **rutitles diff was SKIPPED for sec05 and would have caught 3/7 rescues** («Аспекты мифа»,
«Культурный комплекс», «На пороге инициации» were all in data/ru-titles.jsonl from the cogito
crawl). **Mandatory: `tools/rutitles.py both "<EN title>"` for every item before ❌** (added to
the negatives gate below).
4. **flib authorall for prolific RU authors** (Элиаде a/2943 = 49 books) caught «Тайные общества»
(sec05 #13) that libgen title search missed. Always run flib author sweep for authors with
>5 RU books.
5. **libgen mislinked-file trap**: edition 141685581 «На пороге инициации» carried an ePubLibre
Spanish Lermontov file. Before downloading, check the file `locator`/title via
`json.php?object=f&addkeys=*&ids=<fid>`; for recent books prefer flib (cleaner metadata).
6. **Collection-inclusion questions** (#27/#28): download the smallest fb2, parse the TOC
(`<section name="title">`), grep for the EN book title + RU calque. «Символ и ритуал» (Наука
1983) CONTAINS «Ритуальный процесс» (full text, core section) but NOT «Лес символов»
(citations only).
7. **flib pdf download**: `/b/<id>/pdf` may return an HTML stub; `/b/<id>/download` → 302 →
direct `static.flibusta.is:443/b.usr/<File>.<ext>` URL — curl that directly.
## Catalog caching convention
Processed/scraper-unfriendly catalogs are cached under `data/catalogs/` (committed, with source
@ -516,8 +563,14 @@ not downloadable). Legend for ISBN columns: **✅** = ISBN found; **⚠** = publ
with a one-line totals row (DL EN / DL RU file counts). Cross-check the DL columns against the
section's `downloads/<nn>-*/MANIFEST.md` so the two never disagree.
Negatives gate (before ❌): author sweep (RSL + libgen + cogito + flibusta) + one title-level
SearXNG + livelib author page + one shop/retail check. Log every RU query in MISSING.md.
Negatives gate (before ❌) — **v2 (2026-09-22, after sec05 rescues)**:
author sweep (RSL + libgen + cogito + flibusta **authorall** for prolific RU authors) +
**`tools/rutitles.py both "<EN title>"` diff (MANDATORY)** + 2–3 RU title variants
(calque + SearXNG/web_search `"<EN title>" перевод/на русском` + FR/DE-original variant for
Eliade/van Gennep-type authors) + livelib author page + one shop/retail check (cogito person
page for identity) + **fantlab.ru for pre-1991/adventure/ethnographic** + predanie.ru for older
editions + **user oracle table at section end** (see Lessons). Log every RU query in MISSING.md.
Before downloading from libgen: verify file locator/title (mislinked-file trap).
Special cases:
- **Jung's CW volumes:** translated and widely available — don't deep-search; note the RU