Sold or Moved: Helping Store Agents See What the Till Misses
I give talks and workshops on this material for retail teams and conferences. See past talks or invite me to speak.
Series: Foundations of Agentic AI for Retail, Chapter 7 of 14. Updated for the Third Edition (October 2026).
From the book: Foundations of Agentic AI for Retail, Third Edition
A shopper picks up a product, then puts it back on a different shelf. On the product's home shelf, a camera flags a gap.
Was it sold, or only moved? The till never sees a move. An agent that reorders anyway, or steers the next shopper to a substitute, is acting on a guess.
An online store logs every click. A physical store has to infer what happened from sensor signals that arrive noisy and late.
Before a store agent acts on a shelf, it needs to know what actually happened there, and how sure it can be.
Why an empty facing is a question for your agent
In Chapter 7 I follow that misplaced product through three questions. Each belongs to a different layer of the agent's perception, and each layer stops somewhere.
| The question | What answers it | Where it stops |
|---|---|---|
| Is something out of place? | Sensors, fused into one view with a confidence | They see the empty shelf and the stray sighting but cannot link them |
| What is it, and where does it belong? | A knowledge graph of products and where they belong | It cannot say whether acting will change sales |
| Will acting on it change the outcome? | A causal model, across many incidents | A routine incident does not need one |
The first question starts with what a sensor owes the agent, because an agent should trust a reading before it sends an associate to an empty facing. The chapter writes that down as a sensor-to-agent contract.
The camera that watches the shelf also sees the shoppers in front of it. So the chapter also settles what stays in the store: the most sensitive and time-critical signals are processed at the edge, and only privacy-safe summaries and alerts travel upstream.
How the agent learns the product never left
The sensors now hold two facts that look unrelated: an empty facing on one shelf and an unexpected sighting in another section. Linking them takes memory, and Chapter 7 turns to a knowledge graph that can record where each product belongs.
In "Closing the Incident: A Design Trace," I put the rule this way: "A mismatch marks a placement discrepancy, not stock that has left the store."
That trace follows a hypothetical incident to an associate's confirming scan, 16 minutes after the first empty reading. Here is its shape.
The agent's loop has a latency budget, timed in milliseconds from capture to trigger.
The budget ends at the trigger, and the shelf is still empty then.
— Foundations of Agentic AI for Retail, Chapter 7
Availability returns only when the facing is filled and a later reading confirms it. So the chapter judges a perception system by shelf-unavailable minutes and associate minutes per incident, as well as by milliseconds.
When to ask whether acting helps at all
For one misplaced product, the third question can wait. "Closing the Incident" says it plainly: "No causal model decides a routine incident like this."
It returns across many incidents, as a question about the recovery policy. Once traffic, category, and time of day are accounted for, does faster recovery raise sales enough to justify the associate time?
The knowledge graph can't say. It records what exists and how things relate, but not why.
When I teach this, I use Pearl's ladder of causation, whose rungs are seeing, doing, and imagining. A descriptive graph stands on the first rung, and an agent that changes a price needs the second.
What the chapter gives you
- The sensor-to-agent contract and the Privacy-Preserving Edge Gateway pattern, for deciding what a reading must carry and what may leave the store
- The misplaced product as a running case, from the first empty reading to the associate's confirming scan
- Companion code behind every listing: a sensor processor, a retail knowledge graph, and a causal promotion analysis
- Exercises that build up to designing the shelf incident and a causal graph for its recovery policy
Go deeper
Chapter 7 of the Third Edition, out since October 2026, has the full method, the misplaced-product case, and the companion code. It also covers what this post only touched: "Operational KPIs for Perception Systems," the measures that surface drift early, and "Code Example: Causal Inference for Promotion Effectiveness," where several estimators are checked against the true effect. New in the Third Edition: Part II's running case, a camera's empty shelf and the question of whether the product was sold or only moved.