9.6 — Bias, Filtering & Fair Data

Sampling bias, in your room. Who playtested your cave? Your friends, your table, people who already play platformers. If the class's data says "section 3 is easy," but every tester has 200 hours of Hollow Knight — is section 3 easy? The data is TRUE about the SAMPLE and misleading about the PLAYERS WHO WILL ACTUALLY PLAY. That gap is sampling bias.

Bias	In your project
Selection bias	Playtesters who volunteered are more forgiving (self-selection)
Survivorship bias	You count the players who stayed to the end. The ones who quit mid-cave left no row at all
Observer effect	Testers play differently when the designer watches (2.4 — why we sit silently)
Confirmation bias	You tune what you already suspected was broken

Filtering — the exam's favorite trap. A filter is a question you ask data: "show me only section 5." Filtering helps see patterns; it can also silently erase people. The classic: "let's drop the outlier runs" — until you realize the dropped runs are all the players who quit, and your average difficulty just got friendlier for no reason.

Step 1 — audit the playtest pool. For the playtests.csv dataset (or your own 7.7 table): list who was IN the sample and what kind of player was NOT. Write one sentence of scope: "This data represents ___, and does not represent ___."

Step 2 — find the missing rows. The dataset has completed_boss = FALSE rows. Who is ABSENT entirely? Players who stopped playing mid-cave — no row at all. For each of these absences, name the conclusion that the data would silently get wrong, and say which column you would have to start recording to stop it.

Check your answers name your own first

Then compare. Every one of these is a claim the table would have made happily without the missing rows in it.

• The average rating reads too high: the runs that never finished are the runs that scored low, and the players who quit for good are not in the table at all.
• The section times read too fast: a player who stopped mid-cave never timed the section that stopped them, so the file's times only describe the players who kept going.
• The difficulty curve reads too flat: the hardest section is the one where people stopped, and stopping is the one thing the file has no row for.

Step 3 — a fair-data fix, in pseudocode. Nothing here compiles or runs: this is the exam's arrow-and-DISPLAY notation used to write down what happens to the rows. It is data reasoning, not a program. Design the fix the exam wants — a filtered average that also REPORTS what it excluded:

kept ← [row FOR row IN runs IF row.completed_boss = TRUE]
dropped ← [row FOR row IN runs IF row.completed_boss = FALSE]
DISPLAY("avg win time: " + AVERAGE([row.time_seconds FOR row IN kept]))
DISPLAY("excluded: " + LENGTH(dropped) + " runs")

The second DISPLAY is the ethics: an aggregate that hides its filter is a claim wearing a lab coat.

Step 4 — algorithmic bias preview. Games do this too: your difficulty ramp assumes a player who finished the tutorial. Matchmaking assumes history predicts behavior. Recommendation algorithms trained on biased clicks show more of what was already popular. Same principle, bigger consequences — 9.10 takes it to the real world.