ISBN validation (58 ISBNs): tools/isbnval.py + data/isbn-validation.tsv; SUMMARY Table 4 + fixed 2 bad-check-digit typos and 2 publisher-prefix mismatches; AGENTS.md source 21
This commit is contained in:
parent
38d17b6cc6
commit
61a02a4f43
6 changed files with 458 additions and 15 deletions
37
AGENTS.md
37
AGENTS.md
|
|
@ -49,7 +49,9 @@ sections/0N-<name>/ # one .md per BOOK (chapter-level list items are merged int
|
|||
- **Limitations:** no pagination (fixed first page) → always query by **full first+last name**
|
||||
for people, or 2–3 distinctive words. Loose matching (searches author AND title fields).
|
||||
- Other `type=` values on this mirror are **dead**: worldcat, googlebooks, isbndb, udc
|
||||
(all return "Nothing found" even for control queries). `type=isbn` only decodes ISBN structure.
|
||||
(all return "Nothing found" even for control queries). `type=isbn` decodes ISBN structure:
|
||||
checksum error + **registered publisher group name** (e.g. 5-519 → «Клуб Касталия»,
|
||||
5-98712 → «Петроглиф») — cheap, unlimited; used by `tools/isbnval.py` as the structure layer.
|
||||
|
||||
### 2. libgen.vg full-text search — PRIMARY for readable editions
|
||||
- Tool: `tools/lg.py "query"` (whole words only, res=100, prints pagination hint)
|
||||
|
|
@ -117,8 +119,9 @@ sections/0N-<name>/ # one .md per BOOK (chapter-level list items are merged int
|
|||
- No key, no rate limits observed (1s between calls is polite). Aggregates all editions of a
|
||||
work into one doc. **Best use: resolve full EN author names from initial-only list entries**
|
||||
(`title:"..."` → author_name) and get the EN works list + ISBNs to feed Google Books.
|
||||
- **NO RU editions indexed** (language=rus → 0 for known RU books) — RU hunting stays with
|
||||
RSL/libgen/flibusta/GBS SPb.
|
||||
- **RU coverage is sparse but REAL** (romanized titles, e.g. «Chelovek i ego simvoly»); the
|
||||
`search.json?isbn=` facet works for RU ISBNs too. Don't use a `language=rus` filter — it
|
||||
under-reports. RU hunting still mainly RSL/libgen/flibusta/GBS SPb/Google Books.
|
||||
|
||||
### 12. Wikidata — identity anchors (life dates + OFFICIAL RU name)
|
||||
- Tool: `tools/wd.py "Full Name"`
|
||||
|
|
@ -132,11 +135,13 @@ sections/0N-<name>/ # one .md per BOOK (chapter-level list items are merged int
|
|||
work (`/download/<item>/<file>`). OCR `_text.pdf` variants → pdftotext.
|
||||
- Used to verify the 4-essay match for Neumann «Сборник статей» (item in.ernet.dli.2015.139225).
|
||||
|
||||
### 14. Google Books — via fetch_content only (API is 429 without key)
|
||||
- `fetch_content https://books.google.com/books?vid=ISBN<digits>` → authoritative publisher
|
||||
metadata: full author/editor names, edition, series. Resolved Ottmann=Klaus, Goldstein=Ralph,
|
||||
Brutsche=Paul, Elder=George, Moon=Beverly, Killick=Katherine, Pennington/Staples, Rowland=Susan,
|
||||
Bolander=Karen, Acton=Mary via this or OL.
|
||||
### 14. Google Books — book page via curl OR fetch_content (API is 429 without key)
|
||||
- `books.google.com/books?vid=ISBN<13digits>`: **plain curl works** — 200 + `<title>"T - A -
|
||||
Google Книги"` if indexed, 404 if not. Good ISBN validator incl. RU (АСТ/Эксмо/БукСМарт hit;
|
||||
small-press 404). googleapis.com/books API stays 429 without a key.
|
||||
- `fetch_content` on the same URL → richer metadata: full author/editor names, edition, series.
|
||||
Resolved Ottmann=Klaus, Goldstein=Ralph, Brutsche=Paul, Elder=George, Moon=Beverly,
|
||||
Killick=Katherine, Pennington/Staples, Rowland=Susan, Bolander=Karen, Acton=Mary via this or OL.
|
||||
|
||||
### 15. Shop directory (HSE bookshelf34 pattern) — where RU psychoanalytic books are sold
|
||||
- HSE «Книжный шкаф» (hse.ru/ma/therapy/bookshelfNN) = a DIRECTORY of shops, not a book list:
|
||||
|
|
@ -192,6 +197,22 @@ sections/0N-<name>/ # one .md per BOOK (chapter-level list items are merged int
|
|||
(author) reliably works so far. libgen proxy (tools/rsl.py) still covers title/ISBN queries.
|
||||
- search.rsl.ru — same catalog, Yii app, no JSON API found (2026-07-09).
|
||||
|
||||
### 21. ISBN validation pipeline — `tools/isbnval.py` (2026-07-15)
|
||||
- Per ISBN: (1) libgen.vg biblioservice `type=isbn` — checksum + registered publisher group;
|
||||
(2) Open Library `search.json?isbn=` — title/author/year; (3) Google Books `?vid=ISBN` curl
|
||||
— title tag, 200/404; (4) NLR free-text for RU misses. Random pauses 2–5s, resumable.
|
||||
- Verdicts: VALID (found w/ matching title) / STRUCT-ONLY (real ISBN, prefix registered,
|
||||
not indexed — small-press RU) / BAD-STRUCT. Output TSV: `data/isbn-validation.tsv`.
|
||||
- Found in sec04: 2 bad-check-digit typos (5-89613-003-6, 5-263-00368-2), 2 publisher-prefix
|
||||
mismatches (Ленанд vs URSS; Академический проект vs Азбука-Аттикус), 2 co-publishing
|
||||
prefixes (Петроглиф for ЦГИ 2020; Пальмира for Касталия 2025).
|
||||
- **triumph.ru** (РГБ ISBN search, user-provided): `https://www.triumph.ru/html/serv/find-isbn.php?isbn=<13digits>`
|
||||
(relative to the poisk-isbn.html page!) → returns links to РГБ/МГУ/РНБ/БЕН РАН/ГПНТБ catalogs
|
||||
pre-filled with the ISBN. Itself no data; МГУ (nbmgu.ru) results are JS-rendered, БЕН РАН
|
||||
(Koha) is scrapable but small collection. NLR absence ≠ ISBN invalid.
|
||||
- **isbnsearch.org**: nice per-ISBN pages (author/publisher/year) but rate-walls after ~10
|
||||
requests ("Please Verify to Continue", no recovery in 7 min) — not for batches.
|
||||
|
||||
## Not usable / low value
|
||||
- libgen biblioservice worldcat/googlebooks/isbndb/udc (broken on this mirror)
|
||||
- imaton.com (МААП educational publisher, not translations)
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue