チェックが機能していないにもかかわらずパスしてしまう10の方法

Phronesis · essay Ways of Checking Rincón, D., with Claude · phronesis · 2026 · checking again is not checking A passing check is a claim, not a fact. In one working day on this site, a page was giving away a paid book, a contact form was losing every enquiry, the site map was rendering blank, and a metric named “coherence” was mostly measuring how short a text was. Every one of these had already passed a check. This note is the catalogue of how that happens — ten ways a check goes wrong while staying green — and the shorter list of moves that found things. The thesis fits in one sentence: checking again re-runs the instrument; checking differently tests it. The ledger The examples here are not hypothetical and not curated from a career — they are one audit day on this site, 18–19 July 2026. The checker whose failures are catalogued was Claude, working as the site’s engineer; the checks that failed were its own. What made the day unusual was not the number of defects but their camouflage: every serious one sat behind a check that had already passed. The passing check was the disguise. 1. The check that cannot fail A verification script printed twelve lines reading BAD and then concluded PASS. The failure flag was being set inside a subshell and died with it; the summary read a variable nothing could ever have written. The check was not lying — it was incapable of telling the truth. Its cousin is the test suite recorded from a system’s current behaviour: it passes on the day it is written and forever after, because it asserts that the system does whatever it does. The tell: a check that has never failed is unproven. Break the thing on purpose once and watch the check go red. If it cannot be made to fail, it is not an instrument; it is a decoration. 2. The wrong dialect A search for href="/courses" found nothing, because the framework emits href="/courses/". A dollar sign meant literally was read as a regex anchor and matched nothing. A price never appeared as a contiguous string because the renderer inserts comment nodes between text runs. An audit expecting min="0" met JSX writing min={0}. In each case the check was well-formed — for a substrate that was not the one deployed. Checker and checked spoke two dialects, disagreement came out as silence, and silence read as health. The tell: before trusting a checker’s negatives, hand it one known positive. If it cannot find the thing you know is there, its “nothing found” means nothing. 3. Looking where the light is The site map rendered blank for most of a day, and every audit passed while it did. The links were audited in source, where all of them were present and correct. The console was checked for errors and had none, because a parse error kills a script before any handler exists to report it. The HTML fallback held every link, hidden by CSS on desktop. Each check looked where checking was easy. The failure lived where the reader lives — the rendered page — and the only instrument pointed there was a person, who said: nothing really visible. The tell: check at the layer where the consequence happens. A source audit is a fact about source. Only a rendered check is a fact about the page. 4. The measure that measures something else The writing instrument on this site reported a “coherence” score — the algebraic connectivity of a sentence graph. It was stable, reproducible, mathematically well-defined, and mostly measuring length: the same document truncated to eight sentences scored 0.39; whole, 0.11. Identical prose, a fourfold swing. A measure can be perfectly reliable about the wrong thing. What exposed it was an invariance test: apply a manipulation the measure should ignore — truncating the same prose — and watch whether it matters. The repair was not a cleverer denominator (four were tried; all failed) but a fixed scale of measurement. The tell: reliability is not validity. Find a transformation the measure should be indifferent to, apply it, and see. 5. The sample the checker wrote To test that metric, four sample texts were first written by hand — a tight argument, a rumination, a disconnected list, a technical passage. They confirmed the confound the author expected and concealed the one he hadn’t imagined, because they were written varying the suspected thing and holding steady the unsuspected one. Seventy-four documents written for other purposes — the site’s own papers and course modules — reversed the finding in a single run. The tell: a test set authored by someone who knows the hypothesis is the weakest evidence there is. Use material that existed before the question did. 6. The design that erases its effect Three experimental designs in a row failed to find a hysteresis loop that was really there. The first held each measurement so long that the memory it hunted had already relaxed — quasi-static by construction, and hysteresis vanishes in the quasi-static limit by definition. The second compared the effect at its best against the control at its worst. The third took the ratio of two numbers that were both noise. The loop appeared on the fourth design — paired, rate-matched, absolute — and had been there all along. The tell: ask what would distinguish the effect from the artefact that mimics it, and design for the distinction. A design that only asks is there an effect? will find one, or erase one, by construction. 7. Silence read as success The contact form posted to a worker that had not existed for weeks. Every submission failed; every failure was swallowed by a catch block; the form looked fine and told no one. The suggestion box had the identical disease. A game’s scores went to endpoints never deployed, each request ending in catch(()=>{}). Nothing looked broken, because looking broken had been explicitly handled away. Silence is ambiguous between no failure and no signal, and each of these systems had been built to prefer the flattering reading. The tell: absence of an error is not presence of a success. Probe the path end to end and require the positive signal — the 201, the stored record, the reply — before believing it. 8. The document that drifted from its data A page argued that a steel bench and a pine bench sit about 9 °C apart at the skin. The figure was real, computed from a table of conductivities and densities that lived in the same file. Then the table was edited — better numbers, sound reasons, and every test still green, because the code was never wrong. Only the sentence was. A document quoting a figure it did not measure is an untested assertion, and it fails silently: nothing throws, nothing 404s, the page simply says a thing that is no longer true and keeps saying it. The fix was to write the sentences down as tests. Not one assertion per function — one per claim, named after the sentence it defends: steel and pine sit ~9 °C apart, a steel bench at 65 °C burns, cork moves 6 °C across a 75 °C swing while steel moves 69. Within seconds of being written the suite failed, and the failing assertion was the one that had been overclaimed: it said every plant material beat every petroleum one, and polyurethane foam had the best number on the board. The page had to be weakened from “biomaterials feel warmer” to “one of them matches the best synthetic there is.” That is a truer sentence, and it exists because an assertion refused the flattering one. Two more fell out of the same suite. A rule change made minerals admissible, and an assertion reading clean surfaces span less than concrete began comparing concrete against itself — the words had held still while the category beneath them changed meaning. And a superlative flipped: foam has the lowest effusivity was true, was asserted, and stopped being true the moment a new material was added. It carried a note saying what to rewrite when it flipped, so the flip arrived with its own instructions. The sharpest version of the move is to assert what the writing must not say. When two figures were close enough to call level rat