Thin records in, catalogued records out

01What it is

A catalogue record arrives thin: a title, an author, maybe an ISBN, and nothing a reader could search on. marc-flow takes that record plus whatever else you can point at — the publisher’s page, the PDF, a photograph of the title page — and returns field-level proposals: 650$a with this subject heading, 264$c with this year, each with a confidence score and the text it was read from.

Nothing is written silently. A processor proposes; it never decides.

The preview runs at preview.marc-flow.zauberto.nl. It is a public beta on real infrastructure — the SRU lookups go to the Library of Congress, one call a second, politely.

02Sign in, and claim a workspace

Registration is an email address and a verification link. There is no password: signing in is a one-time code by mail, or a passkey once you have registered one.

The sign-in form asks for two things — your email, and a workspace slug. A slug nobody has taken becomes yours, and you become its owner; one you are already a member of simply lets you in. A workspace is a real tenant, not a URL segment: jobs, records, artifacts and policy are scoped to it, and every request re-checks membership rather than trusting a token that was true an hour ago.

03Give the record something to read

Everything happens on Upload. A record is not one file — it is a small pile of things that are all about the same book, and it can carry several at once:

The interesting part is that these compose. Agents are not handlers for a source; they are typed edges in a small graph, and the runner keeps expanding a record’s artifacts until nothing new appears:

So handing the record only a publisher URL gets you: the page fetched, reduced to Markdown, read for a citation_isbn, that ISBN looked up in a catalogue, and the returned MARC record described — a chain nobody configured, and just what the nine edges add up to. It terminates on its own (an ISBN already looked up is not looked up twice), and is bounded at eight rounds anyway.

04Decide what applies without you

One dropdown, Acceptance mode, and it is the job’s standing decision for its whole life:

Auto-accept          every proposal applies
Manual review        none apply; you rule on each
Confidence >= 0.5    …0.75, 0.9 — at or above the bar applies

Two rules make this safe to use. Nothing is ever auto-rejected — below the bar stays pending, not refused. And a mode can never overturn a person: it only ever sees a round’s fresh proposals, and anything you already ruled on is carried across untouched. The reverse is allowed; you may revise a verdict, yours or your mode’s.

Start on Confidence >= 0.9 for a first run. It is the setting where the interesting disagreements are still in front of you.

05Watching it run

Submitting creates a job and drops you on its page. Progress is written as the work happens, one row per agent per record: queued → running → done, or not_found, or error.

queued and running are split on purpose. External catalogues are rate-limited — the SRU agent makes one call at a time, at most one a second, across every job in the process — and on a batch that wait is most of the elapsed time. A row sitting in queued is being polite, not stuck.

An agent reports not found only when nothing it ran produced anything. The web agent finding no ISBN on a page it just described is the ordinary case, not a failure.

06Reading the proposals

Open a record. Three things are on the page:

• the agent rail — what each agent read, in its own words, plus how many proposals it made;
• the proposals — field, new value, confidence, and the explanation;
• the MARC pane — the record as it is, and as it would be.

Each proposal names a field the way MARC21 does, and the notation goes further than 245$a: 008/06 is one byte of a control field, LDR/07-08 a span of the leader, 245 alone with no value is that field’s indicators. Coded values are half of what a cataloguer corrects.

A proposal also carries where it was read. The scan view splits them into “Read on this page” and “From the record”, tracing a suggestion back to the photograph of the colophon it came out of. That is the model’s claim about its own reasoning, not a proof — and the interface says so in those words.

Accept, reject, or undo, one at a time or with accept all ≥ N%. Undo puts a proposal back to pending and clears who decided it with it; that it was ever decided lives in the audit trail, which is the honest place for it.

07Score and handover

Every record is rated on four dimensions, before and after its accepted proposals, because the difference is the thing you actually want to see:

The weighting is a chosen model, not a MARC21 standard, and the package says so. Which tags are required is your workspace’s Policy screen — and a job carries the policy it was created under, so editing the profile never rewrites what a past job downloads.

Handover is where the profile stops describing and starts deciding: it marks a record finished with, and refuses unless there are no gaps, every dimension clears its target, and every proposal has a verdict. A refusal answers with the criteria, not with a sentence — being told “does not meet the profile” and left to find out which tag is barely better than not being told.

08Taking the record away

The enriched record is built on demand — binary MARC21, MARCXML or MARC-in-JSON, per record or for the whole job. It is not stored, because the accepted set changes every time somebody rules on a proposal, and a derived file needing invalidation is a second source of truth.

A record that was changed says so in 583 ($a enriched $c YYYYMMDD $x marc-flow), the field MARC21 keeps for a processing action. One with nothing accepted comes back unstamped rather than claiming work that wasn’t done.

Handover does not gate this. It states where a record stands; it is not a lock on the bytes.

09The same thing over HTTP

Everything the interface does is a documented REST call, under /api/docs. A job from an ISBN, with a webhook:

curl -X POST https://preview.marc-flow.zauberto.nl/api/workspace/acme/jobs \
  -H 'Authorization: Bearer mfk_…' \
  -H 'Content-Type: application/json' \
  -d '{
        "mode": "0.9",
        "records": [{"files": [{"source": "isbn", "isbn": "978-0-306-40615-7"}]}],
        "webhook": {"url": "https://example.com/hooks/marc-flow"}
      }'

The webhook fires once, on completed or on failed, and its body is the whole job — every record’s proposals and annotations, the same shape the status endpoint returns. Any 2xx counts as delivered; anything else is retried with exponential backoff up to five times, and the outcome is readable on the job itself. A webhook that never succeeds does not fail the job.

Requests are not signed. Treat the URL as the secret: make it unguessable and https, and if it matters, re-fetch the job by id rather than trusting the body.

Machines authenticate with a workspace API key (mfk_…), one per workspace, readable exactly once when it is rotated and refused by every settings route — a shared credential manages no policy.

10Try it

preview.marc-flow.zauberto.nl — claim a slug, paste one thin record, hand it the publisher’s URL, and set the mode to manual. The first thing worth reading is not the enriched file; it is the agent rail, saying what it found and where.