jung/tools
Dmitry Kokorin 5441dddbb8 pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref
- tools/rutitles.py: EN->RU token translation (data/ru-dict.tsv, 540+ pairs, Kogito/Castalia
  conventions) + Jaccard diff against data/ru-titles.jsonl (692 titles: Kogito 518, Litres 24,
  OPP 55, flibusta a/5272+57639+118921+122883+193690+193689). Matching is lemmatized
  (pymorphy3) — case endings handled: 'великой матери' -> 'Великая мать' 1.25.
  'The Great Mother' -> 'Великая мать' 1.25 top hit; 'The Symbolic Quest' -> correct negative.
- tools/sweep.py: batch driver for Phases 1-2 (rsl+lg+alib+flib+cogito+SearXNG-OZON-snippet
  parse+rutitles diff per item; --nlr optional). Query log data/queries.log (JSONL).
  Fixed: cogito nav-menu leak (parse bx_product_item only), libgen robot-block (lg.py curl fallback).
- tools/oa.py: OpenAlex + Crossref (Phase 0 identity/ISBN, no key; found Margaret Wilkinson,
  Karen Evers-Fahey, Symbolic Quest Princeton ISBN).
- data/sweeps/01-fundamentals/input.tsv: 16 ❌ items loaded; background sweep running.
- AGENTS.md: source 10b (oa.py), flibusta .su = reduced mirror (dropped from pipeline),
  pipeline v3 section (rutitles/oa/sweep/pymorphy3 note: pymorphy2 broken on py3.12).
- data/ru-titles.jsonl committed as the RU market universe asset.
2026-09-18 10:56:36 +03:00
..
alib.py alib.ru source (tools/alib.py, CP1251 stem-search marketplace): Turner handbook = Дипак 2015 ISBN 978-5-98580-069-2 (RSL 7609331); Furth 2nd ed 2014; Neumann market round 4 (КДУ/Питер/Маниф/Касталия editions, Angelika Lev bio 2021); notes #16/#26/#29 updated 2026-09-09 00:19:52 +03:00
cogito.sh sec04: Jung CW9/1 — ✅ «Архетипы и коллективное бессознательное» АСТ 2019/2020, libgen 141693619 2026-09-08 23:06:59 +03:00
crw.py #38 FOUND: «Символизм цвета. Из архивов Эраноса 1972» (Клуб Касталия, ISBN 978-5-90811-054-9) — same Ottmann editor as Spring 2005; via castalia.ru sitemap sweep (Phase 2b) 2026-09-09 00:46:19 +03:00
flib.py sec01: phase 4 downloads complete (50 files, 204MB) + item 12 EN preview + flib.py pagination 2026-09-17 10:41:43 +03:00
ia.py sec01 phase 2 (round 1): 17/55 items resolved 2026-09-17 00:09:03 +03:00
isbncheck.py ISBN validation (58 ISBNs): tools/isbnval.py + data/isbn-validation.tsv; SUMMARY Table 4 + fixed 2 bad-check-digit typos and 2 publisher-prefix mismatches; AGENTS.md source 21 2026-09-16 00:01:13 +03:00
isbnval.py ISBN validation (58 ISBNs): tools/isbnval.py + data/isbn-validation.tsv; SUMMARY Table 4 + fixed 2 bad-check-digit typos and 2 publisher-prefix mismatches; AGENTS.md source 21 2026-09-16 00:01:13 +03:00
lg.py pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
lgdl.py lgdl.py: stall detection + fresh-start wipe + speed guard; MANIFEST queue status 2026-09-15 13:23:18 +03:00
mgu.py tools/mgu.py: MGU Scientific Library search (GET-discovery, fields/methods/pagination, ISB dead, SER goldmine); AGENTS.md source 22 + Google low-volume warning 2026-09-16 09:59:32 +03:00
nlr.py NLR sweep round 7: all section-04 surnames checked (tools/nlr.py, CRW+Primo); Cunningham 2018 identified; Dora Kalff confirmed absent from NLR (correct spelling Калфф); Schaverien 'Умирающий пациент' in NLR holdings 2026-09-09 12:38:57 +03:00
oa.py pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
ol.py EN-side research pillars: tools/ol.py (Open Library) + tools/wd.py (Wikidata), Google Books ISBN pattern, archive.org verification, HSE shop-directory + NikBook/Skifiya/beta2alpha sources, workflow v2 (full names first, reverse publisher sweep, community lists, content verification gate); full EN author names resolved for all 39 items (AUTHORS-EN.md); CONTEXT round 3 (Neumann cogito finds, 1959 disambiguation) 2026-09-09 00:10:25 +03:00
rsl.py AGENTS/CONTEXT: new sources (cogito person pages, livelib author pages), verified RU editions (Franz, Jaffé/Man&HisSymbols, Lüscher, Kandinsky, Kast, Schaverien) 2026-09-08 22:28:09 +03:00
rutitles.py pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
sweep.py pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 2026-09-18 10:56:36 +03:00
sx.sh Init: research structure, tools (lg.py, rsl.py, sx.sh), AGENTS.md, CONTEXT.md, section 04 skeleton (39 book notes) 2026-09-08 22:10:07 +03:00
wd.py EN-side research pillars: tools/ol.py (Open Library) + tools/wd.py (Wikidata), Google Books ISBN pattern, archive.org verification, HSE shop-directory + NikBook/Skifiya/beta2alpha sources, workflow v2 (full names first, reverse publisher sweep, community lists, content verification gate); full EN author names resolved for all 39 items (AUTHORS-EN.md); CONTEXT round 3 (Neumann cogito finds, 1959 disambiguation) 2026-09-09 00:10:25 +03:00