TALAMANA · THE AI LITERACY MAP FOR ARCHITECTURE AND DESIGN · Judgment · AGE 17—19 · FACTUAL · DURABLE
Automation bias
People accept a system's answer over their own reading and skip the checks.
The idea
When a system gives an answer, people tend to accept it over their own reading, and to skip checks they would otherwise have made. Researchers call this automation bias. It has two forms: following a wrong suggestion, and failing to act because the system did not flag anything. It was studied in aircraft cockpits and hospital decision-support long before chatbots. It is a property of people working with machines, not of any one machine, which means it arrived at your desk with the first convenient tool.
Why it matters
The bias is in you. No update to the tool removes it. Only a habit does.
See it in the studio
The area schedule a tool computed does not match your rough count. You assume you miscounted. Three weeks later the municipal submission carries the tool's figure, and the tool had read one room twice.
Watch for this
The sentence "the software checked it." Software produced it. Checking is a separate act, and a person does it.
Try it
Ask a classmate to hand you a tool's answer to a task you can check by hand, such as five room areas from a plan, with one line quietly altered. You should not know which line. Notice whether you check, or defer. Then swap roles.
Prove it
Describe one moment you deferred to an output against your own reading, and what it would take for you to notice next time.
How it works
A 2012 systematic review of seventy-four studies of clinical decision-support found that wrong advice from a system raised the rate of wrong decisions, and that the effect shrank when people were trained for it, held accountable for their decisions, shown the system's confidence, and given information rather than a recommendation. Those four remedies transfer directly to a studio. A related 2025 observational study in one medical field found that clinicians' unaided performance fell after months of working with an AI assistant. That is nearby evidence that deference, kept up for long enough, becomes something worse. That is the next card.
What this idea builds on
What this idea opens up
Sources
Open this idea on the map · The complete map · Logika · RBDS AI Lab, India · revised every edition.