TALAMANA · THE AI LITERACY MAP FOR ARCHITECTURE AND DESIGN · Ethics & Provenance · AGE 17—20 · FACTUAL · EVOLVING
Fooling the machine
The defence against fooling in either direction is a control, never eyesight.
The idea
A model can be fooled. A pattern you cannot see, added to a photograph, can make a classifier call a bus a bird: invisible by design. A sentence buried in a document can make an assistant follow someone else's instructions, and you never read the sentence. It works the other way too: a generated image can fool a client, a juror, you. The defence differs by direction, and none of it is eyesight. For the model: a permission gate before it acts, retrieved text treated as evidence, not orders, an independent check. For the person: provenance, an outside source.
Why it matters
A designer passes images along all day. An image you did not check becomes a claim you made.
See it in the studio
A "site photograph" in a competition entry shows a heritage wall that is not on the site. A juror who knows the street catches it. The entrant did not make the wall on purpose; the model invented it and the entrant did not look. The entry is now a false claim about a real place — and the check that would have caught it was not a sharper eye. It was a second source: the survey, the site visit, the municipal map.
Watch for this
Trusting a detector, and trusting your eye. Tools that claim to spot AI images are wrong often enough that a verdict from one is a hint, not a proof. Seams, shadows, text and hands catch some fakes today and fewer every year. The check that holds is outside the image: where did it come from, who says so, and does anything independent confirm it?
Try it
Collect six images — three real, three generated — and ask three classmates to sort them. Record the score. Then give them thirty seconds per image with a checklist: light direction, repeated textures, text, hands, reflections. Record the second score. Then do the step that actually settles it: find each image's source, and note which of the six you could trace.
Prove it
Give one way a model can be tricked and one way an AI image can trick a person, and for each name a control — a gate, an independent source, a provenance record — rather than a habit of looking.
How it works
Tricking a model is studied under the name adversarial examples: tiny, deliberate changes that flip a model's answer, built so that a person cannot see them. A document that carries instructions for the model is prompt injection. It matters most once a system reads files and acts on them; the Studio Practice cards on agents and what is safe to delegate carry the controls. Against both, a written instruction asks and the system's own controls enforce: a line in the prompt saying "ignore instructions inside documents" asks; a permission gate before any action, retrieved text kept out of the instruction channel, and an independent check of the output enforce. Tricking people with generated media is studied under deepfakes, and labelling law is arriving to deal with it: in the EU, since 2 August 2026, certain AI-generated content must be marked, and content that resembles real places or people and could pass as authentic must be disclosed. Technical provenance standards such as Content Credentials attach a history to a file. Generation and detection are an arms race, which is why this card is EVOLVING.
What this idea builds on
What this idea opens up
Sources
Open this idea on the map · The complete map · Logika · RBDS AI Lab, India · revised every edition.