Recurring sequences and positions in the visual observations

Project Summary

Completed bounded descriptive first pass. The program’s raw flags (240 RELIABLE, 79 CONFIRMED_SENSITIVE, 92 EXPLORATORY_ONLY) are mechanical filter outputs. They stay frozen and unvalidated, and the formal “reliable” and “confirmed” wording is withdrawn. The 411 candidates are four overlapping ways of grouping the same forms, not 411 independent results.

Execution: complete. Scientific acceptance: none — the results are descriptive diagnostics under one grouping rule. Human-interface completion: depends on Jeremy being able to use this page from START HERE. Physical iPhone use: unverified.

What were we trying to find out?

Do the neutral v006 visual observations show neighbouring forms that recur in a particular left-to-right order, or forms that prefer particular places in a row, and do those patterns hold on leaves held out from the search?

What did we do?

• Six analyses were planned; two were run: how often each provisional visual group sits at the left or right end of a geometric row segment, and which ordered pairs of neighbouring groups occur (overlapping, tight and spaced pairs).
• Rows were rebuilt from the native boxes by linking mutual nearest neighbours, because older line routes often mixed text rows. Uncertain or ineligible forms block links; nothing was bridged.
• About 30% of leaves were held out and frozen before any statistics. 1,067 screening tests on the other leaves selected 411 candidates (at most 40 per pattern and grouping, chosen by screening strength), which were scored once on the held-out leaves.
• Sixteen explanatory pictures were rendered afterwards from the unchanged Yale TIFFs, and their file, decoded-pixel and exact-crop hashes were checked. A reviewer critiqued the frozen results, the author reconciled each point, and a final presentation critique led to the corrections listed below.

What did we find?

• A separately written routine applying the same frozen linking rule to the author’s loaded data reproduced every held-out pair count (0 mismatches). Row-end counts were not recounted.
• Among neighbours with a small gap (under 0.3 row-heights), some provisional groups follow one another more often than expected, but only modestly once shape size is accounted for: candidates #243, #251, #252, #254, #258 and #259, with size-conditioned O/E of about 1.2–1.6.
• Some ordered pairs are nearly absent: #262, #273, #275, #276 and #282. For example, #273 was seen 0 times where about 30.1 were expected after size conditioning. The extractor merging touching ink could still explain part of this.
• Some frequent groups sit less often than expected at row-segment ends (O/E about 0.19–0.73): #2, #3, #5, #8, #37, #39, #41, #42 and #45. The reviewer’s narrower list is #2, #3, #37, #39 and #42.
• Not established: particular groups concentrating at row ends, any reliable pattern across spaces, and all overlapping pairs.
• The candidates listed above passed every after-the-fact check applied to them. They involve about nine frequent groups and overlap heavily, so they are not independent findings. They are descriptive diagnostic candidates, not accepted units. No count of accepted findings is given.

What went wrong and was corrected

Leaf test selection bug

The original leaf-by-leaf sign test counted a leaf only if the pair occurred there or at least one was expected, so rare enrichments were counted as wins. Under the reviewer’s calibrated (and itself approximate) test, 128 of the 240 raw RELIABLE flags pass; 96 rest mostly on leaves with very low expected counts.

Size confounding

Small and tall shapes pair differently. Conditioning the expectation on size class shrinks the effects: #243 falls from O/E 3.79 to about 1.50, #250 disappears (221 vs 220.5) and spaced pair #375 fails. Only 68 of 158 tested family-pair RELIABLE flags pass.

Weak segmentation-clean check

The check that leaves out forms with open join/split alternatives had no minimum support: example 10 passed on 1 observed vs 0.095 expected. Results with fewer than 5 clean observations are treated as sensitive.

Multi-page and partial views

Photos showing two folios were assigned to their first folio number. As a result, five of the 32 held-out leaves (f069, f071, f089, f100, f102) are tied to search-set leaves through two-page photos; for example, page f089r is in the search-set photo f088v_and_f089r. Page f090r may appear both in a held-out photo (f089v_part_and_f090r) and as a separate search-set photo (f090r). Whether any text was counted on both sides is unresolved. No whole-leaf integrity is claimed for the pharmaceutical and astronomical sections. This corrects an earlier rough figure of seven.

Duplicate-text check is uninformative

A 4-gram check for duplicated text flagged nothing, but it had no positive control and even views expected to overlap scored near baseline. Its negative result is not proof that the views are independent.

Partition check not independent

The family-partition check reuses the same occurrences: the rare-pooled layer copies the v006 layer (67 of 68), and the v005 layer largely relabels it.

Row ends are not line starts

Row-segment ends include drawing boundaries (examples 2 and 3), false edges next to tall forms (example 12) and boxes merged across rows (example 13).

What failing a check means

Failing the calibrated leaf test means “cannot be tested leaf by leaf with this approximate test”, not “refuted”. Failing size conditioning means “not distinguishable from size pairing”. Size classes partly overlap with families, so this check may also remove some real family signal.

Corrections after the final presentation critique

• Source-check scope now includes the stage-1 samples and the full ledger cross-check, and states what those checks do not show.
• The multi-page holdout problem is restated as five affected held-out leaf groups, replacing an earlier rough figure.
• The recount is described as a separate routine on the author’s loaded data, covering pair counts only.
• Two after-the-fact diagnostics are labelled as z ≥ 2 screening scores.
• Meaning statements were narrowed to modest geometric preferences; the about-20 count and core-pattern tags were replaced by descriptive diagnostic-candidate wording.
• Captions for examples 6, 11, 12 and 14 were corrected; titles printed inside the pictures remain the author’s original, unvalidated post-test categories.
• Frozen results, the frozen reconciliation and all raw data are unchanged; the previous generator and report are preserved, and the reasoning is recorded in the author’s final reconciliation addendum (file names are under Technical Details).

What does it mean?

Under this geometric grouping, neighbours with a small gap show some modest ordering preferences and a few nearly absent orders, and some frequent groups sit at row-segment ends less often than expected. In the inspected pictures such small-gap neighbours lay inside one written group. That fits handwriting construction, the way the extractor cuts forms, ordinary spelling, abbreviation or generated text, and cannot tell them apart. No character, word, sound, language, cipher or reading direction is supported. Reversing the order cannot decide reading direction, because mirrored writing would show the same asymmetry.

Limitations

• Every later check reuses held-out data that had already been scored once, so they are diagnostics after the fact, not fresh confirmation. The holdout is now used; a new confirmation needs unused data and its own authorization.
• The holdout is within this study only; these pages were visible in earlier project studies.
• Only the 80 most frequent groups per layer were tested, uncertain forms were excluded, and the row rule has known false edges and merged boxes.
• Holm correction was applied only within each analysis group, so results near the threshold are fragile. Most held-out leaves are herbal, although the core tight results agree in herbal and non-herbal leaves.
• Source linkage was verified on samples, not for every form: stage 1 matched 12 sampled Yale TIFFs (file and decoded-pixel hashes) and 60 exact crops from search leaves, and cross-checked all 114,551 IDs, family labels, default flags and source hashes against the sealed v006 ledger; the 16 rendered pictures were separately hash-verified. These checks show the records point to the right pixels; they do not establish writing recall, segmentation accuracy or character identity. No comparison with an external transcription (EVA) was made.
• Not run: four of the six planned analyses (gap-group edges, recurring units and repetition, section profiles, locality) and the other deferred checks (top-of-block position, three-form sequences, uncertain-form sensitivity, external transcription). This is a bounded study, not all six analyses.

What needs attention?

Five picture cases invite your judgment (Review Needed). No answer is needed to close this study, and nothing follows automatically.

Strongest next test

A blinded crop audit. For the core tight pairs and the near-absent pairs, show 10 random examples beside 10 examples matched on size class but from different groups, and have a reviewer judge “one split form” or “two forms”. Any later test should correct the leaf grouping of multi-page views, link rows along the baseline, use a calibrated leaf test and a size-conditioned expectation, test the saved join/split alternatives, and use independent human segmentation as validation. Nothing follows automatically; it needs separate authorization.

Who did the work

The analysis and the critique were both written by Claude Opus from the same provider. The reviewer read the author’s assessment first, reused the author’s data-loading code, viewed seven of the sixteen pictures and did not open the native TIFFs. This presentation was written by a separate read-only Claude Opus session and run by Codex; it reran no statistics and changed no frozen result or raw label. A final presentation critique was written by the same reviewer session; it is not independent of the earlier critique. It could not open the rendered report (too large for its reading tool), checked the text through this generator, and viewed only display examples 11 and 12.

Explore All Results

Observed = count on held-out leaves. Expected = the count if groups were arranged independently within each page: for pairs, given how often each group appears as the left and the right member within that gap class; for row ends, within page and size class (page only for the size-class layer). O/E above 1 means more often than expected, below 1 less often, 0 never seen. Group IDs are provisional visual groupings, not characters. Tap a row’s details for diagnostics and source examples.

All 411 candidates All 1,067 screening tests All 16 pictures

All 411 frozen candidates (four overlapping layers)

The program’s raw flags (240 RELIABLE, 79 CONFIRMED_SENSITIVE, 92 EXPLORATORY_ONLY) are mechanical filter outputs. They stay frozen and unvalidated, and the formal “reliable” and “confirmed” wording is withdrawn. The 411 candidates are four overlapping ways of grouping the same forms, not 411 independent results. “Reading after review” repeats the reconciliation and critique.

All 1,067 exploratory screening tests, including non-signals

Every pattern scored on the search leaves, whether or not it became a candidate. Screening q is an occurrence-level screening score, not proof.

All 16 explanatory pictures

Chosen after the test by a fixed rule to explain results and counter-cases; not an accuracy sample. Tap a picture to enlarge it.

Review Needed

These five picture cases need human judgment. Each shows the exact Yale pixels with the example’s boxes. This page records no answers. No answer is needed to close this bounded study, ordinary uncertainty blocks nothing, and no answer would accept a character, word or language.

Case: example 6

Question: Is member 2 the same kind of form as the clearer members of family V6F000006, or only a faint partial stroke?

Possible answers:

• Same kind of form
• Faint partial stroke: the family is mixed
• Something else
• Unsure

Case: example 10

Question: Are the two marked forms really two separate forms on either side of a space? Only 1 clean observation supports this pair, against 0.095 expected.

Possible answers:

• Two separate forms across a space
• Not a real pair (one form, or the wrong neighbour)
• Unsure

Case: example 12

Question: Is the marked form really at the left end of its row segment, or does a neighbouring form sit immediately to its left?

Possible answers:

• True left end
• False edge: a neighbour sits to its left
• Unsure

Case: example 13

Question: Does member 1’s box hold one form in one row, or does it take in ink from the row below?

Possible answers:

• One form in one row
• Box merged across two rows
• Unsure

Case: example 11

Question: Author and reviewer disagree. Are the overlapping boxes two pieces of one pen form, or two separate adjacent forms whose boxes overlap because member 2’s box includes a long diagonal stroke reaching under member 1?

Possible answers:

• Two pieces of one pen form (author)
• Two separate forms (reviewer)
• Cannot tell which form owns the diagonal stroke
• Unsure