TALAMANA · THE AI LITERACY MAP FOR ARCHITECTURE AND DESIGN · Ethics & Provenance · AGE 11—15 · FACTUAL · DURABLE

Biased examples, biased machine

A machine leans the way its examples lean, and a person answers for it.

The idea

A machine learns from examples. If the examples lean one way, the machine usually leans that way too. A toy case: show it a thousand photos of sloping roofs and ten of flat roofs, and it will think houses usually have sloping roofs. That lean is called bias. The machine did not choose it. The examples carried it in — and the goal it was trained toward, and later corrections, can add a lean or remove one. A machine cannot answer for any of this. A person has to: whoever picked the examples, and whoever uses the answer.

Why it matters

When a result looks unfair, do not stop at "why is the machine unfair?" Ask what it learned from, and who is responsible now.

See it in the studio

Ask a drawing tool for "a house" and you may get a house from a country you have never visited. It is not ignoring you. It is showing you where most of its examples came from.

Watch for this

Saying "the computer decided, so it must be fair." Biased examples can make a machine biased, and even good examples do not guarantee a fair result — and examples are gathered by people.

Try it

Ask an image tool for "a school" five times. Count how many look like your school. Then write one line on where the examples probably came from.

Prove it

Take one lopsided result you have seen and trace it back: which examples would make a machine answer that way? Then name who should fix it.

How it works

Bias in the examples is one of the most common reasons a machine treats two groups differently — not the only one. What the machine was told to aim for, how it was corrected after training, and how its answers are used all matter too. It shows up in faces it recognises badly, accents it mishears, and buildings it cannot picture. Fixing it means changing the examples, changing the goal, or correcting the answer, and all of those are jobs for people. The lean can even start at the desk where people tag the examples. An Indian film, Humans in the Loop, follows a woman in Jharkhand whose knowledge of the forest does not fit the labels the machine allows. A later card on this map, The tilted mirror, shows what the lean does to pictures of Indian streets and houses.

What this idea builds on

What this idea opens up

Sources

Open this idea on the map · The complete map · Logika · RBDS AI Lab, India · revised every edition.

Age grows from 11 at the centre to 22 at the edge, and six sectors show the learning strands. Tab into the map and the arrow keys step from idea to idea, following the links where there is one. Enter opens the idea under the cursor, and E reads out its links and the reason recorded on each. Press slash for Search, question mark for the full key list, and Escape to leave. Open Ideas for the complete readable list, including what each idea builds on and what it opens up.

LOGIKA · RBDS AI LAB INDIA
ON-RAMP · AGE 11 · FIRST ENCOUNTERS, NOT GATES — IDEAS · — DEPENDENCIES
DONE
OPENS NEXT
SOLID — STANDS ON