jung/data
Dmitry Kokorin 5441dddbb8 pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref
- tools/rutitles.py: EN->RU token translation (data/ru-dict.tsv, 540+ pairs, Kogito/Castalia
  conventions) + Jaccard diff against data/ru-titles.jsonl (692 titles: Kogito 518, Litres 24,
  OPP 55, flibusta a/5272+57639+118921+122883+193690+193689). Matching is lemmatized
  (pymorphy3) — case endings handled: 'великой матери' -> 'Великая мать' 1.25.
  'The Great Mother' -> 'Великая мать' 1.25 top hit; 'The Symbolic Quest' -> correct negative.
- tools/sweep.py: batch driver for Phases 1-2 (rsl+lg+alib+flib+cogito+SearXNG-OZON-snippet
  parse+rutitles diff per item; --nlr optional). Query log data/queries.log (JSONL).
  Fixed: cogito nav-menu leak (parse bx_product_item only), libgen robot-block (lg.py curl fallback).
- tools/oa.py: OpenAlex + Crossref (Phase 0 identity/ISBN, no key; found Margaret Wilkinson,
  Karen Evers-Fahey, Symbolic Quest Princeton ISBN).
- data/sweeps/01-fundamentals/input.tsv: 16 ❌ items loaded; background sweep running.
- AGENTS.md: source 10b (oa.py), flibusta .su = reduced mirror (dropped from pipeline),
  pipeline v3 section (rutitles/oa/sweep/pymorphy3 note: pymorphy2 broken on py3.12).
- data/ru-titles.jsonl committed as the RU market universe asset.
2026-09-18 10:56:36 +03:00
..
catalogs sec01: round 2 closed — 0 new rescues; cache processed catalogs in data/catalogs/ 2026-09-18 09:28:33 +03:00
sweeps/01-fundamentals pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
castalia-catalog-slugs.txt Phase 2b sweep results: castalia.ru catalog saved (data/), all remaining section-04 items re-verified ❌ via alib author sweep; status 12 ✅ / 1 🔶 / 26 ❌ 2026-09-09 00:52:08 +03:00
isbn-sec01-validation.tsv sec01 phase 3: RU ISBN validation (23 ISBNs) + sec04 flibusta sweep 2026-09-17 09:40:36 +03:00
isbn-sec04.tsv ISBN validation (58 ISBNs): tools/isbnval.py + data/isbn-validation.tsv; SUMMARY Table 4 + fixed 2 bad-check-digit typos and 2 publisher-prefix mismatches; AGENTS.md source 21 2026-09-16 00:01:13 +03:00
isbn-validation.tsv ISBN validation (58 ISBNs): tools/isbnval.py + data/isbn-validation.tsv; SUMMARY Table 4 + fixed 2 bad-check-digit typos and 2 publisher-prefix mismatches; AGENTS.md source 21 2026-09-16 00:01:13 +03:00
ol-sec01-queries.tsv repo-structure: authors/ is canonical cross-section dossier store; section AUTHORS-EN.md now thin indexes 2026-09-16 10:48:43 +03:00
ol-sec01-requery.tsv repo-structure: authors/ is canonical cross-section dossier store; section AUTHORS-EN.md now thin indexes 2026-09-16 10:48:43 +03:00
ol-sec01.tsv repo-structure: authors/ is canonical cross-section dossier store; section AUTHORS-EN.md now thin indexes 2026-09-16 10:48:43 +03:00
queries.log pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
ru-dict.tsv pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
ru-titles.jsonl pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
wd-sec01.txt repo-structure: authors/ is canonical cross-section dossier store; section AUTHORS-EN.md now thin indexes 2026-09-16 10:48:43 +03:00