Definitions

Every term used on this site, in one sentence each.

RQ2

The research question this site answers: does a failure taxonomy built in a simulator describe real accidents, or only the simulator?

Taxonomy

The set of categories, built in Phase 1, for how a shared-authority system fails.

Injector

The program that deliberately broke tasks in the simulator to produce the failures the taxonomy was built from.

Simulator

ALFRED, a household-robot task environment where the injector ran.

Pre-registration

The analysis plan and pass-fail thresholds, written down and time-stamped before any rating began.

Census

Rating every one of the 754 finding codes in the sample, rather than a subset.

NTSB

The National Transportation Safety Board, whose investigators wrote every finding used here.

Finding

One coded statement from an accident investigation about what went wrong.

Finding code

The five-part code NTSB assigns to a finding, such as Personnel issues-Psychological-Attention/monitoring-Monitoring-Pilot.

Modifier

The last part of a finding code, which usually names who or what was at fault.

Propagation

A verdict on one code applies to every finding that carries that code, so 754 judgments cover 3,176 findings.

Sample

The 1,008 accidents drawn from the 22,840 that qualified, split into four groups.

Arm

One of the four accident groups in the sample; the word comes from clinical trials.

T1

The multi-aircraft group: accidents involving two or more aircraft.

T2

The crew group: airline and charter flights with more than one pilot.

T3

The air-traffic-control group: instrument flights where a controller held separation.

C

The single-pilot comparison group: one pilot, one aircraft, daytime, clear weather, no controller.

Comparison group

A group where coordination is absent by construction, so the taxonomy should not fit it well.

Scope

The sealed rule that decides which findings enter the study at all.

Scope objection

A rater's note that a finding should not have been in scope; it is recorded but changes nothing.

Source tag

The label for which version of the sealed vocabulary a rating was made against.

Drift type

One of the five kinds of failure: WorldState, Perception, PolicyDecision, Constraint, HumanOverride.

WorldState

The world changed and nobody's picture of it kept up.

Perception

Something real was there and a party failed to see it or saw it wrongly.

PolicyDecision

A party with an adequate picture chose the wrong action.

Constraint

A rule, procedure, permission, or policy was the thing that failed.

HumanOverride

A human took, withheld, or should have taken control from an automated participant.

Layer

Where in the system a type lives; each type has exactly one layer, so it adds no categories.

Actor

Who the finding attributes the failure to: Human, AI, Automation, or Policy.

Human

A person in the operating loop, such as a pilot, a controller, or a mechanic.

AI

A learned component, trained from data rather than specified; almost absent from this period of aviation.

Automation

A non-learned system or piece of equipment behaving as built, including one that reached its designed limit.

Policy

An organization, regulator, or rulebook, such as the operator, the manufacturer, or the FAA.

No actor

The finding names a condition that no participant produced, such as weather or terrain.

Cell

One combination of a drift type and an actor; there are 20 admissible cells.

Reachable cell

A cell the injector can actually produce; there are 6.

Escaped cell

A cell the injector can never produce; there are 14.

Maps

The finding is an instance of exactly one drift type, with no stretching.

Maps partially

The finding spans two types, or fits only under a stretched reading.

Unmapped

No drift type describes the finding, and the rater writes a short label saying what it is about.

Residual

Everything that was unmapped, grouped by label; this is where the taxonomy's gaps show.

Residual label

A short phrase naming what an unmapped finding is about, such as equipment-limit-exceeded.

Dwell

The seconds between an item appearing and its verdict being saved; a quality signal.

Coverage

The share of findings that map to some drift type; on its own it cannot fail.

Escape

The share of mapped findings that land in cells the injector never produced.

Random baseline

The escape rate you would get by filing findings at random, which is 70%.

Actor-weighted baseline

The same random baseline, adjusted for how often each actor actually appears.

Ratio

Observed escape divided by the baseline; near 1 means no better than random.

Group difference

Coverage in the coordination groups minus coverage in the single-pilot group, in percentage points.

Delta

Another name for the group difference.

Confidence interval

The range a number could reasonably take given the sample; every gate reads the cautious end of it.

Lower bound

The cautious end of a range when the gate requires a number to be at least something.

Upper bound

The cautious end of a range when the gate requires a number to be at most something.

Effective sample size

How many findings actually distinguish the groups after shared codes cancel out; about 26 here.

Specific agreement

Of all the times either rater called a code unmapped, the share where both did.

Kappa

An agreement statistic that discounts chance; it collapses when one verdict dominates.

AC1

An agreement statistic that does not collapse under that skew, reported beside kappa.

Gate

A pass-fail test with a threshold fixed before rating began.

G5 escape

Escape divided by the random baseline must be below 0.60.

G4 agreement

The author and an independent rater must agree on unmapped codes at least 70% of the time, at the lower bound.

G3 ceiling

The single-pilot group must fit no more than 60% of the time, at the upper bound.

G1 difference

Coordination groups must fit at least 15 points better than single-pilot, at the lower bound; supporting evidence only.

Decision rule

The pre-registered claim is made only if G5, G4, and G3 all pass.

Rater

A person who assigns verdicts to codes.

Author

The rater who built the taxonomy; their verdicts alone cannot show the definitions carry the meaning.

Independent rater

A rater who has never seen the taxonomy and works only from the definitions on screen.

Session

One rater's pass over a set of codes.

P1

The author's census of all 754 codes.

P3

The independent rater's pass over a 150-code subset, used for the agreement gate.

P6

The author re-rating 50 codes at least two days after P1, to measure their own consistency.

Decisions log

The dated record of every judgment rule made while rating, with the definition text it rests on.