TALAMANA · THE AI LITERACY MAP FOR ARCHITECTURE AND DESIGN · Generative Mechanics · AGE 17—19 · FACTUAL · DURABLE
The joint text-image space
A reference hands the model a coordinate, and you get back only what it captures.
The idea
In a CLIP-style model, an image encoder and a text encoder are trained together so that a caption and its picture come out close, and a caption and the wrong picture come out far apart. Across hundreds of millions of pairs, a courtyard photograph lands near the sentence "a photograph of a courtyard". This is the second number-space on this strand; its job is comparison: is this image like that text? Many image generators read your prompt through such an encoder, which is partly why "make it like this reference" works. It is one family's space, not every model's.
Why it matters
In a pipeline built on this space, uploading a reference does not "show" the model anything. It hands the model a coordinate. What the coordinate captures — mood, palette, typology — is what you get back. What it does not capture is lost.
See it in the studio
You upload a photo of a Chettinad house for its courtyard proportion. The result gives you its colours and columns instead. In a CLIP-style space, palette and ornament dominated the photo's address. Proportion was a smaller part of it.
Watch for this
Saying "the model looked at my image". In a CLIP-style pipeline it found the image's coordinate, so say in words what you want from the reference as well, so that both addresses point the same way. And taking this for the space in which images are made. Generation happens in a different space — LATENT, next.
Try it
Upload one reference with no text. Then the same reference with one line naming the single quality you want. Compare. The difference is the text moving the address.
Prove it
Describe what it means that, in a CLIP-style model, an image and its caption are trained to line up, and explain how a reference image and a text prompt get compared inside that space.
How it works
The method is contrastive training (CLIP). An image encoder and a text encoder learn together, so that matching pairs score high and mismatched pairs score low. Many diffusion-based generators read the prompt through a text encoder of this kind; some newer ones use a language model's encoder instead. A multimodal chat model is a different design again. It does not drop your picture into this space; it converts image patches into tokens and feeds them in beside your words — ONE STREAM on this strand.
What this idea builds on
What this idea opens up
Sources
Open this idea on the map · The complete map · Logika · RBDS AI Lab, India · revised every edition.