Skip to content

Layer C is where the framework becomes operational. Each criterion is a specific, falsifiable condition that must be met for a validity type to be satisfied. A validity type is satisfied only when all its criteria are met. Partial satisfaction is reported explicitly.

Construct validity — Is the theoretical entity coherent and well-defined?

Section titled “Construct validity — Is the theoretical entity coherent and well-defined?”
#CriterionOne-line pass conditionPage
C1FalsifiabilityA named result stated in advance would disconfirm the claimconstruct/A_falsifiability.md
C2Structural plausibilityComponents at predicted layers/positions with consistent weight-space signaturesconstruct/B_structural-plausibility.md
C3Task specificityCircuit does not score highly on unrelated tasks under same instrumentconstruct/C_task-specificity.md
C4MinimalityNo redundant members; removing any member degrades performanceconstruct/D_minimality.md
C5Convergent validityMultiple independent instruments nominate the same componentsconstruct/E_convergent-validity.md

Internal validity — Did the manipulation cause the effect?

Section titled “Internal validity — Did the manipulation cause the effect?”
#CriterionOne-line pass conditionPage
I1NecessityAblating the component reliably degrades the behavior across ≥2 methodsinternal/A_necessity.md
I2SufficiencyIsolating/restoring the component reproduces the behaviorinternal/B_sufficiency.md
I3SpecificityEffect is selective; control-axis IIA ≈ 0 while causal-axis IIA is highinternal/C_specificity.md
I4ConsistencyFinding holds across prompt samples, ablation methods, and random seedsinternal/D_consistency.md
I5Confound controlEffect not explained by collateral disruption to non-circuit componentsinternal/E_confound-control.md

External validity — Does the claim generalize?

Section titled “External validity — Does the claim generalize?”
#CriterionOne-line pass conditionPage
E1Intervention reachActivation delta at hook point is in predicted direction and non-trivialexternal/A_intervention-reach.md
E2Graded responseEffect scales monotonically with intervention strength; threshold and plateau visibleexternal/B_graded-response.md
E3SelectivityOn-task effect exceeds off-task effect at the same intervention strengthexternal/C_selectivity.md
E4Effect magnitudeAbsolute effect large enough to support the computational storyexternal/D_effect-magnitude.md
E5RobustnessClaim survives prompt paraphrase, cross-scale transfer, held-out generalizationexternal/E_robustness.md
E6Cross-architecture generalizationMechanism appears in at least one other model familyexternal/F_cross-architecture.md

Measurement validity — Is the instrument trustworthy?

Section titled “Measurement validity — Is the instrument trustworthy?”
#CriterionOne-line pass conditionPage
M1ReliabilityScores stable across prompt splits, seeds, and checkpointsmeasurement/A_reliability.md
M2InvarianceInstrument gives comparable results across model sizes and familiesmeasurement/B_invariance.md
M3Baseline separationScore exceeds random-vector AND untrained-model baselines by meaningful marginmeasurement/C_baseline-separation.md
M4SensitivityDetects real circuits at acceptable hit rates (AUROC ≥ 0.85) without excess false positivesmeasurement/D_sensitivity.md
M5CalibrationRaw scores interpretable relative to known reference pointsmeasurement/E_calibration.md
M6Construct coverageInstrument measures its nominal target, not a correlated proxymeasurement/F_construct-coverage.md

Interpretive validity — Does the verdict match the evidence?

Section titled “Interpretive validity — Does the verdict match the evidence?”
#CriterionOne-line pass conditionPage
V1Level declarationA specific description-mode tag is stated explicitly in the verdictinterpretive/A_level-declaration.md
V2Level–evidence matchEvidence collected is sufficient to license the declared mode taginterpretive/B_level-evidence-match.md
V3Narrative coherenceProse description is consistent with and entailed by the mode-tagged claiminterpretive/C_narrative-coherence.md
V4Alternative exclusionCompeting mechanism descriptions have been considered and addressedinterpretive/D_alternative-exclusion.md
V5Scope honestyVerdict does not silently generalize beyond the evidence scopeinterpretive/E_scope-honesty.md

Building a claim (bottom-up): Identify instruments run (Layer A). For each, locate the criteria it addresses from the mapping table in ../00_taxonomy/index.md. Check each criterion’s pass condition. Assemble the verdict from satisfied and unsatisfied criteria.

Auditing a claim (top-down): Start with the verdict tier. All criteria in the required validity types must be satisfied. Check each criterion page against reported evidence. Note gaps.

Minimum-reporting rule: Every published claim must report, for each satisfied criterion, which instrument satisfied it and what value was obtained.