How a Learned Policy Earns the Right to Run Your Reorders
I give talks and workshops on this material for retail teams and conferences. See past talks or invite me to speak.
Series: Foundations of Agentic AI for Retail, Chapter 5 of 14. Updated for the Third Edition (October 2026).
From the book: Foundations of Agentic AI for Retail, Third Edition
One store, one product, and a plain rule that decides when to reorder it. If the stock on hand and the deliveries already on the way won't last until a new order could arrive, reorder. Otherwise, wait.
The rule is simple, and it works. So in Chapter 5 I gave it two challengers: a planner that looks a few weeks ahead before each decision, and a learned policy, a decision-maker trained by reinforcement learning in a simulator and then frozen. All three faced the same held-out scenarios, test runs none of them had trained on, with supplier delays and a promotion in the middle.
In one scenario, the learned policy waits three weeks, walks into the promotion short of stock, and messages customers instead of reordering. The rule reorders in the second week and never runs out.
The rule kept the job, and that is the chapter's answer in small: a rule keeps a retail decision until a challenger clearly beats a planning baseline on the same tests, and then clears the gates that follow.
Why the simple rule kept its job
The learned policy lost to the rule in all 30 scenarios.
The planner is the more interesting result. It had fewer stockouts, but its overall gain was too uncertain to establish, and it planned inside a copy of the true simulator, an advantage no real planner has. So the rule stays, and the planner's next step is a shadow trial, running beside the rule without acting.
Here are the three contenders side by side.
| Contender | How it decides | Verdict in the chapter |
|---|---|---|
| Hand-written rule | Reorders when stock on hand and on the way won't cover demand | Retained |
| Planner | Tries each action in a copy of the simulator, a few weeks ahead | Fewer stockouts, gain not established |
| Learned policy | Learned from reward in simulation, then frozen | Lost in every scenario |
The chapter sets the bar for any challenger in its first section, recalling the rule's Chapter 4 run.
Chapter 4's rule finished eight simulated weeks without a stockout. A learned policy that wants its job must beat that.
— Foundations of Agentic AI for Retail, Chapter 5
What a reward quietly teaches a markdown agent
Reinforcement learning learns from the outcomes of its own actions. Each action earns a reward, a score the team defines, and the policy drifts toward whatever scores well.
In my work, sell-through has at times competed with gross margin. Reward a markdown agent only for sell-through and it may learn to over-discount, hitting the volume target while harming margin and brand perception. The chapter shows a safer way to set the reward, and when to keep learning away from live customers.
In the section on learning through interaction, I put it this way: "Treat a penalty as a training signal and a bound as a rule the executed action must obey."
How a challenger earns its autonomy
I should be candid, as I am in the chapter's first section: I have not run a learned policy on live traffic. In my team's systems at OODARIS AI, reinforcement learning improves the agents' interpretations and decisions, but offline.
So the chapter's Agent Training Ladder adds autonomy only as the evidence improves. Ship gates decide whether a policy deploys at all.
The decision has a default winner.
The chapter argues that the strongest retail systems are hybrids. A planner defines a safe sandbox, and the learner works inside it.
What the chapter gives you
- A capstone that pits Chapter 4's reorder rule against a planner and a learned policy, with the full results
- The Agent Training Ladder and the ship gates a learned policy must clear before it acts on its own
- A store fulfillment planner with runnable companion code, and a candid reading of its run that holds dispatch until the promised times are verified
- Ten exercises, from quick checks to writing your own retain-or-replace recommendation
Go deeper
Chapter 5 of the Third Edition, out since October 2026, has the full method, the cases and the companion code. It also covers what this post only hinted at: the four patterns in "Hybrid Decision Approaches" that pair planners with learners, and "Planning and Optimization in Decision-Making," on turning a goal into a plan an agent can carry out. New in the Third Edition: where my team's learning runs (offline, from logged decisions), and why I have not run a learned policy on live traffic.