dima/jung
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
144 commits 1 branch 0 tags 9 GiB
Commit graph

43 commits

Author SHA1 Message Date
Dmitry Kokorin
4e776ff8fc sec02: Not-downloadable cleanup (user rule: gap resolved = remove line) + #38 rescue: 
- removed 5 stale lines (22/26/28 EN rescues 09-21, 22 RU rescue 09-25, 38 mislabel:
  it named book #39 which is downloaded; #38 = chapter 'Dreams' in Handbook of
  Jungian Psychology)
- #38 EN rescue: whole handbook downloaded (lg f/93292966, 409 pp, ch. 11 p.244
  verified via full-text OCR), MANIFEST per-item + card Downloads + SUMMARY DL EN
- AGENTS.md: Not-downloadable semantics (true gaps only; rescues remove the line,
  history in git; partial rescue narrows the line to the remaining language)
- sec04 FINAL TALLY: #01 CW9/1 = 4 pdf + 1 cross-ref sec02
 2026-09-26 00:01:57 +03:00
Dmitry Kokorin
a086faa2a4 AGENTS.md: xref trigger points clarified (once done / start of each section / end-of-section stale check) 2026-09-25 23:41:06 +03:00
Dmitry Kokorin
7d3794560e cross-section dedup system (user tasks 2+3, 2026-09-25): 
- tools/xref.py: master book list for ALL 12 sections (RAW 07-12 scraped: sections/07/RAW.md
  + data/isap-raw/), cross-ref matcher (CW gate + distinctive tokens + years + Cyrillic
  transliteration), stale MANIFEST 'Not downloadable' checker; data/MASTER-LIST.md generated
  (17 book-dupes in 01-06 found, e.g. sec01#51=sec06#14 Solomon)
- file-level dedup: 17 md5-identical files removed from later sections (kept in first),
  MANIFESTs/cards -> cross-ref rows, totals updated (sec01 102 / 02 37 / 03 125 / 04 38 /
  05 38 / 06 37), dedup rule in AGENTS.md + MANIFEST Notes
- sec05 MANIFEST phantom line '03 Aion' removed (Aion not in sec05 list; files in sec01)
- sec03 #44: 1968 Bodley file was #41's book (content 'English Fairy Tales') — removed,
  #44 keeps ABC-CLIO 2002
 2026-09-25 23:25:05 +03:00
Dmitry Kokorin
b7ddfac17e sec06: key-vs-f_id discovery — 12 EN files recovered (6 rescues) + precheck v2 
- USER FINDING: libgen edition files map {key: {f_id, md5}} — KEY and f_id VALUE
  are DIFFERENT file records; we were downloading/prechecking by the KEY (wrong file).
- Recovered by f_id: 09-t1 (Eliade UCP 1978), 13 Sungod (Cornell 2010), 14 Solomon
  (Karnac 2007), 17 God-Image (Inner Light 1992), 19 Hornung (UNC 1982), 20 Psyche in
  Scripture (1995), 23 Dionysos (Bollingen LXV/2 Princeton) + backups 12/16/24/25/26/27.
  All content-verified. '24 systemic mislinks' = our bug, not libgen.
- lg.py: objects[]=f,e,s,a,p,w + topics[] (file-level search; found Solomon md5) +
  md5 in output. lgdl.py: bymd5 subcommand.
- precheck v2: rev-value match = OK; locator words >=2 shared = OK; words present
  0 shared = BAD; hash/ISBN/empty locator = no signal (SUSPECT); digit-runs masked
  before tokenization (hex 'aaae' trap).
- MANIFEST/SUMMARY/cards/AGENTS.md updated. 34 files, 608 MB (28 backup still downloading).
 2026-09-25 10:04:01 +03:00
Dmitry Kokorin
61f9287e4a sec06: downloads FINAL — 22 files 413MB (RU 13 + EN 8 + DE 1) + SUMMARY + MANIFEST 
- RU: flib 8 books + libgen 4 (Otto/Kerényi/Schimmel×2), all content-verified
- EN: libgen batch 24/26 mislinked (precheck + content-verify caught); 9 files
  from archive.org open user-uploads (HRI t2, Otto, Corbin, MacCulloch, Schimmel,
  Scholem, Suzuki, Zimmer, Perera via lg)
- cross-refs 01/02/03/05/06/11 NOT re-downloaded (files in sec01-03)
- RU paid (castalia PDF 1190₽): 08/10/13/17/18/21 — not downloaded (user rule)
- sec03#20 RU gap noted in sec03 MANIFEST (МИФ 2024, no free file)
- SUMMARY.md at-a-glance 28 rows; card download notes appended
 2026-09-25 08:56:52 +03:00
Dmitry Kokorin
57b88fe39e AGENTS.md: Workflow v3 — Phase 1.5 (batch catalog sweep до карточек): кросс-свип по обработанным секциям (PRIORITY 1), rutitles по всем item, каталоги издательств, EN-свип batch; каноническое RU имя (rulabel) — первый вариант всех запросов 2026-09-24 23:09:25 +03:00
Dmitry Kokorin
f08930fd7d AGENTS.md: Castalia catalog rule (user instruction) — обязательный этап поиска для каждого item; negatives gate v3; rutitles DB 1977; sec06 статус в шапке 2026-09-24 23:04:36 +03:00
Dmitry Kokorin
109662ffe4 sec06: user oracle — 7 спасений (10,13,17,18,20,21,27✅) + каталог Касталии (1285) в data/catalogs + sec03#65 исправлен 
- #10 Переписка. Юнг и Уайт — Клуб Касталия 978-5-521-16231-4 (+PDF на castalia.ru)
- #13 Путь Ра через Дуат — Касталия 2022, пер. И. Ветринский, 978-5-521-18582-5
- #17 Бог и бессознательное (Эдингер) — Клуб Касталия 2016, 386с.
- #18 Нисхождение к богине (Перера) — Касталия 2024, 168с., 978-5-521-24012-8
- #20 Глубинная психология в священном писании (Клюгер) — ИНФРА-М 2000, 96с.
- #21 Сатана в Ветхом Завете (Клюгер) — Касталия 2018, 194с.
- #27 Введение в дзэн-буддизм (Судзуки) — RU подтверждён (livelib+jhana.ru), изд. TBD
- Бонус: sec03 #65 Клюгер «Гильгамеш» Касталия 2018 (была ❌) → ✅ (39✅/29❌)
- Клюгер: досье переписано (dates 1907-1987 Q72049066; RU name = «Клюгер»!)
- castalia.ru /collection/all = полный каталог (1285) → data/catalogs + rutitles DB
- Итог sec06: 19 ✅ / 1 🔶 / 8 ❌
 2026-09-24 23:01:36 +03:00
Dmitry Kokorin
7b24100450 sec06: 28 карточек (11✅/2🔶/15❌) + rulabel.py (ru labels как канал канонической кириллицы) 
- Кросс-рефы: 01→sec01#05, 02→sec02#06, 03→sec01#32, 05→sec03#09, 06→sec01#15, 11→sec03#20
- Новые ✅: Eliade HRI («История веры и религиозных идей» Акад.проект 2008-09),
  Otto «Священное» СПбГУ 2008, Kerényi «Дионис» ВРС 2007, MacCulloch Э/Эксмо 2018,
  Schimmel «Мир исламского мистицизма» Садра 2012, Scholem 4 изд., von Franz CW6
  («Видения Николая из Флюэ…» Акад.проект 2025, rescue через rutitles diff)
- tools/rulabel.py: Wikidata ru label + aliases = каноническая кириллица
  (VIAF/ISNI/GND/BnF с этой машины недоступны: 403/mёртвые API). Ловушки:
  Циммер не Зиммер, Корбен не Корбин, Шолем не Шулем.
- AGENTS.md: источник 12 дополнен (rulabel.py + статус authority-APIs)
 2026-09-24 11:26:20 +03:00
Dmitry Kokorin
bc1f91d9be authors: systemic identity audit — Wiki anchors + 9 date/kinship corrections 
User caught: Emma Jung dossier said '1877-1965, Jung's eldest daughter'
(actually 1882-1955, his WIFE, Q124210). Systemic response:
- New MANDATORY field **Wiki anchor:** (QID + URL) in dossier canon (AGENTS.md)
- tools/verify_dossiers.py: resumable Wikidata audit (bulk wbgetentities for
  anchored, search+CANDIDATE review for the rest, 429 backoff, TSV report)
- 82 dossiers now anchored; 76 marked no-confirmed-QID (WIKI-CHECK notes)
- Date/kinship corrections found by the audit:
  * Emma Jung: wife not daughter, 1882-1955 (Q124210)
  * Verena Kast: b. 1943 not 1951, father Walter Kast not 'Hans Kast' (Q2515260)
  * James Hillman: 1926 not 1941 (Q934774)
  * Jolande Jacobi: 1890-1973 not 1902-1980 (Q88022)
  * Maria Czaplicka: d. 1921 not 1939 (Q532804)
  * Henri Ellenberger: 1905-1993 not 1900-1987 (Q115111)
  * Edward F. Edinger: 1922-1998 not 1905-2001 (Q5819984, en.wiki)
  * Hans Dieckmann: d. 2007 not 2005 (Q59526701)
  * Heinrich Zimmer: 1890-1943, anchored to the Indologist Q215922
    (not the Celtist Q76954)
- Homonym traps documented in TSV notes (John Hill=botanist, Karl
  Koch=hacker, Barbara Meier=model, Andrew Samuels=soccer, etc.)
 2026-09-24 10:29:54 +03:00
Dmitry Kokorin
2a2699d61a sec05: EN downloads complete (30 files) + SUMMARY at-a-glance + lgdl queue infra 
- 30/31 EN candidates downloaded via libgen, all content-verified
  (magic+pages+first-page text); 08b 37-pp fragment discarded
- EN gaps: 04/05/15/16/23 not found (rare academic), 03/14 ia-restricted
- MANIFEST.md: EN sections per item + not-downloadable extended
- 26 cards: Downloads tables w/ EN rows + EN NOT FOUND notes (6 cards)
- SUMMARY.md created (canon v1): at-a-glance table 29 items, EN->RU authors
- tools/lgdl.py: speed-probe + CDN-edge re-draw (per-request get.php
  302s to different storage hosts), connect timeout, stall wipe on first stall
- AGENTS.md: Lesson 9 (EN-queue infra: ads.php throttle, watchdog,
  unbuffered logs, dual-queue trap, AA 403, i2p verdict)
 2026-09-24 00:04:13 +03:00
Dmitry Kokorin
67fac97559 tools/lgdl.py: precheck <f_id> <edition_id> [pages] — mislinked-file gate (reverse-map editions + locator batch path + size sanity; exit 0/1/2); dl with 4th arg auto-prechecks and aborts on RED. Tested: caught 3 bad files (Одержимый магией / Laso / Captain Action comic); OK path verified 2026-09-23 00:49:32 +03:00
Dmitry Kokorin
b43e8f763c sec05: RU downloads — 17 files 85MB, all content-verified. libgen mislinked-batch incident (9/20 wrong files: EN papers/novels, RU fantasy, Spanish fiction) → re-sourced from flib; MANIFEST FINAL + card Downloads tables + AGENTS lessons 5/8 updated (content-verify gate mandatory) 2026-09-23 00:32:28 +03:00
Dmitry Kokorin
be9080d0a5 AGENTS.md: sec05 lessons (user oracle table, RU-title non-calques, mandatory rutitles diff, libgen mislinked-file trap, collection TOC verify) + sources 29-30 (fantlab, predanie.ru/LJ) + negatives gate v2 2026-09-23 00:05:38 +03:00
Dmitry Kokorin
d01e323e75 liart.py: OPAC-Global protocol probed (direct.exe FindView/FindSize, guest session); SearXNG record-URL fallback works; direct mode blocked on search labels+iddb (admin-only InfoDB) — data/liart-params.json stub for browser read 2026-09-22 08:42:06 +03:00
Dmitry Kokorin
92b9d589a2 AGENTS.md: section unification marked DONE (5/5, scripts + data finds + living-authors rule) 2026-09-22 00:00:25 +03:00
Dmitry Kokorin
c41ea3503a AGENTS.md: section file CANONS v1 (SUMMARY/INDEX/MANIFEST/MISSING/dossiers), migration order MANIFEST first 2026-09-21 23:26:58 +03:00
Dmitry Kokorin
f3fb432bd9 v3 migration: linter tools/check-md.py (0/219 issues); sec01 #47 RU table fixed; AGENTS.md: migration done, liart TODO + pending unification markers 2026-09-21 11:11:53 +03:00
Dmitry Kokorin
7189a219f3 AGENTS.md: book card CANON v3 (language-symmetric Editions—<lang> + Downloads + Catalog records, compact ❌ gate, placeholder bans); TODO liart OPAC tool; pending .md unification markers 2026-09-21 10:47:15 +03:00
Dmitry Kokorin
1ff76b4270 sec02: EN soft sweep — +8 files (Fordham 18, von Franz Dreams 19, Hall 22, Meier 26, Whitmont/Perera 28); sec03/04 gaps re-verified (0 rescues); AGENTS.md: EN sweep procedure documented 2026-09-21 10:02:56 +03:00
Dmitry Kokorin
40a3dbd10c full-repo consistency audit (2026-09-20): 32 stale ⬜ dossier rows refreshed from final cards; Dora Kalff dates fixed (1896-1981); sec03/02 SUMMARY DL cols reconciled with disk (7 rows); sec04 MANIFEST 38→39 files + Red Book file names; card 06 Acton: EN = ia borrow-only (0415148898), libgen ed 15863166 = 2p journal snippet (not book), 'Marjorie'→Mary; sec01 INDEX Samuels Anthony→Andrew; AGENTS.md status line 2026-09-20 19:42:40 +03:00
Dmitry Kokorin
139df77506 AGENTS.md: sources 23-28 (bookvoed, litres, royallib/coollib, ivran/liart OPACs, maap.pro articles, Три Квадрата) 2026-09-19 17:23:24 +03:00
Dmitry Kokorin
f59c49fbde AGENTS.md: source 3/9 corrections — cogito product pages curl-able w/ full specs; chitai-gorod product pages need CRW; Ozon blocked via CRW too (FAB); vse-svobodny cookie 2026-09-19 14:14:13 +03:00
Dmitry Kokorin
976732c15c tools: searxng fix — bing disabled (returns query-ignoring junk), brave/wikipedia/wikidata dead; google-only + explicit language param; AGENTS.md engine-status note 2026-09-19 13:14:57 +03:00
Dmitry Kokorin
ab8c539fcc sec02: COMPLETE — 23 ✅/5 🔶/29 ❌; downloads (31 files 261MB, incl. CarlJungCollectedWorks jackpot: all 12 Jung EN); SUMMARY/MISSING/MANIFEST; AGENTS.md: ia CW-set item + flibusta converter quirk 2026-09-19 13:04:04 +03:00
Dmitry Kokorin
371517aa06 sec02: Phase 3 triage — 23 ✅ / 5 🔶 / 29 ❌ final; MISSING.md; 12 dossiers updated; Hall CUD identity verified; bonus finds (Kaplan-Williams, R.A. Johnson) 2026-09-19 12:33:49 +03:00
Dmitry Kokorin
5441dddbb8 pipeline v3: RU-title candidates (rutitles+ru-dict+pymorphy3), batch sweep driver, OpenAlex/Crossref 
- tools/rutitles.py: EN->RU token translation (data/ru-dict.tsv, 540+ pairs, Kogito/Castalia
  conventions) + Jaccard diff against data/ru-titles.jsonl (692 titles: Kogito 518, Litres 24,
  OPP 55, flibusta a/5272+57639+118921+122883+193690+193689). Matching is lemmatized
  (pymorphy3) — case endings handled: 'великой матери' -> 'Великая мать' 1.25.
  'The Great Mother' -> 'Великая мать' 1.25 top hit; 'The Symbolic Quest' -> correct negative.
- tools/sweep.py: batch driver for Phases 1-2 (rsl+lg+alib+flib+cogito+SearXNG-OZON-snippet
  parse+rutitles diff per item; --nlr optional). Query log data/queries.log (JSONL).
  Fixed: cogito nav-menu leak (parse bx_product_item only), libgen robot-block (lg.py curl fallback).
- tools/oa.py: OpenAlex + Crossref (Phase 0 identity/ISBN, no key; found Margaret Wilkinson,
  Karen Evers-Fahey, Symbolic Quest Princeton ISBN).
- data/sweeps/01-fundamentals/input.tsv: 16 ❌ items loaded; background sweep running.
- AGENTS.md: source 10b (oa.py), flibusta .su = reduced mirror (dropped from pipeline),
  pipeline v3 section (rutitles/oa/sweep/pymorphy3 note: pymorphy2 broken on py3.12).
- data/ru-titles.jsonl committed as the RU market universe asset.
 2026-09-18 10:56:36 +03:00
Dmitry Kokorin
2157d058df sec01: round 2 closed — 0 new rescues; cache processed catalogs in data/catalogs/ 
- data/catalogs/: cogito Jungian series (518 titles, 30 pp.), litres 140409 (24 JSON-LD),
  OPP/MISP reading list (55 entries) — source URLs + dates in headers
- author-level check of 16 ❌ against cogito shop search (11 author queries) + full 518-title
  catalog diff: 0 matches; noise rejected (Shamdasani≠Casement, Jung 'Нераскрытая самость'
  ≠ McFarland-Solomon, Berdyaev essay)
- AGENTS.md: catalog caching convention
 2026-09-18 09:28:33 +03:00
Dmitry Kokorin
8e9167cc05 sec01: SUMMARY Table 3 (publication status at a glance: EN/RU ISBN + DL EN/DL RU) + AGENTS.md rule 
- Table 3 mirrors sec04 format: per-item EN/RU ISBN status + actual download counts
  (cross-checked with MANIFEST.md)
- AGENTS.md Phase 5: every completed section SUMMARY.md must include the at-a-glance
  table with legend (✅/⚠/❌/—) and DL EN/DL RU columns
 2026-09-17 21:44:53 +03:00
Dmitry Kokorin
32e505b2f9 sec01: phase 4 downloads complete (50 files, 204MB) + item 12 EN preview + flib.py pagination 
- 45 RU + 5 EN + 1 preview downloaded (flibusta.is OPDS, libgen, archive.org)
- 10 flib pdf stubs replaced via libgen (18,22,28,29,53) or fb2 twins
- flib 'fb2+zip' files unpacked to plain fb2
- item 12 (Evers-Fahey): RG=Request-PDF; PagePlace 23pp preview saved+verified
- evers-fahey dossier enriched (ofj.org bio)
- flib.py: authorall (rel=next pagination) + author <page>
- AGENTS.md: flib pagination/quirks + PagePlace preview tip
- INDEX: phase 4 done line; 8 trade-only items listed in MANIFEST
 2026-09-17 10:41:43 +03:00
Dmitry Kokorin
4420aef7c3 sec01 phase 2 (round 1): 17/55 items resolved 
- new ✅: 03 (Структура и динамика психического, Когито), 10 (Эго и архетип, 2 изд.),
  13 (Лекции по юнговской типологии, b/509587), 29 (Юнговская карта души, Стайн Мюррей 2010)
- ❌ with evidence: 02 (CW7), 05 (Aion — 'Эйон' 0), 16 (Shadow and Self), 21 (Animus and Anima),
  38 (Woman in the Mirror), 47 (Way of All Women)
- flibusta.is OPDS sweep (tools/flib.py): 30+ author feeds checked; bonus books logged
  (Edinger x10, Hillman x8, Harding x4, Henderson x2, Douglas Cambridge Handbook, Jung-E Grail Legend)
- tools/ia.py (archive.org EN search/metadata/files); AGENTS.md sources 10+13 updated
 2026-09-17 00:09:03 +03:00
Dmitry Kokorin
a2527a88ee repo-structure: authors/ is canonical cross-section dossier store; section AUTHORS-EN.md now thin indexes 
- 56 new author dossiers (all sec04 + sec01 authors), 69 total
- sec04/sec01 AUTHORS-EN.md rewritten as item→dossier link tables
- AGENTS.md conventions + Phase 0 updated
- item 54 identity: Ann Belford Miller (Alice Miller-Ulanov); homonym dossier miller-alice-englard (Alicja Englard, NOT the same person)
 2026-09-16 10:48:43 +03:00
Dmitry Kokorin
868fa9268a tools/mgu.py: MGU Scientific Library search (GET-discovery, fields/methods/pagination, ISB dead, SER goldmine); AGENTS.md source 22 + Google low-volume warning 2026-09-16 09:59:32 +03:00
Dmitry Kokorin
4feff7a0f7 ISBN validation (58 ISBNs): tools/isbnval.py + data/isbn-validation.tsv; SUMMARY Table 4 + fixed 2 bad-check-digit typos and 2 publisher-prefix mismatches; AGENTS.md source 21 2026-09-16 00:01:13 +03:00
Dmitry Kokorin
5ad38cb502 #28 +Медков С.Б. editions (2006 ISBN 5-902582-08-3 пер. Сиренко; 2020 ISBN 978-5-906891-48-8); Медков publisher profile (Jung line: also 'Психология переноса' 2025); livelib publisher-page pattern added to sources 2026-09-09 13:09:03 +03:00
Dmitry Kokorin
6316581afb sources 19-20 (NLR/РНБ Primo via CRW; RSL-Aleph direct WAU works / WTI+SYS empty); SWEEP-R7.md full surname results; Cunningham 2018; all 26 ❌ re-confirmed on independent NLR catalog 2026-09-09 12:40:01 +03:00
Dmitry Kokorin
645cb964b4 AGENTS: source #18 local CRW JS renderer (fastcrw, localhost:3000, tools/crw.py) + limits (Ozon/chitai-gorod API-blocked) 2026-09-09 00:55:17 +03:00
Dmitry Kokorin
59fb6d3570 alib.ru source (tools/alib.py, CP1251 stem-search marketplace): Turner handbook = Дипак 2015 ISBN 978-5-98580-069-2 (RSL 7609331); Furth 2nd ed 2014; Neumann market round 4 (КДУ/Питер/Маниф/Касталия editions, Angelika Lev bio 2021); notes #16/#26/#29 updated 2026-09-09 00:19:52 +03:00
Dmitry Kokorin
c0a3ac0f2f EN-side research pillars: tools/ol.py (Open Library) + tools/wd.py (Wikidata), Google Books ISBN pattern, archive.org verification, HSE shop-directory + NikBook/Skifiya/beta2alpha sources, workflow v2 (full names first, reverse publisher sweep, community lists, content verification gate); full EN author names resolved for all 39 items (AUTHORS-EN.md); CONTEXT round 3 (Neumann cogito finds, 1959 disambiguation) 2026-09-09 00:10:25 +03:00
Dmitry Kokorin
bdcade0f21 sec04: Schaverien #16 refined (psyjournal variant titles, flibusta/livelib negatives); flibusta.su added as source (param=ask); CONTEXT quirks updated 2026-09-08 23:24:35 +03:00
Dmitry Kokorin
d621928180 sec04: Pattis Zoja — found her RU book «Аборты. Утрата и обновление в поисках идентичности» (Т8/ЦГИ 2017, ISBN 978-5-9500699-9-4, per GBS SPb); added GBS SPb + retail sources to AGENTS.md 2026-09-08 23:16:10 +03:00
Dmitry Kokorin
a6c062344f AGENTS/CONTEXT: new sources (cogito person pages, livelib author pages), verified RU editions (Franz, Jaffé/Man&HisSymbols, Lüscher, Kandinsky, Kast, Schaverien) 2026-09-08 22:28:09 +03:00
Dmitry Kokorin
9b8cf58a4d Init: research structure, tools (lg.py, rsl.py, sx.sh), AGENTS.md, CONTEXT.md, section 04 skeleton (39 book notes) 2026-09-08 22:10:07 +03:00