A decision engine for excess inventory
- What
- A simulation that works out the best way to clear overstocked products across about $50M of excess inventory: discount, liquidate, return to vendor, dispose or wait.
- Result
- About 600,000 simulations per run, in production since 2024. In 2026 I cut a full run to about 40 minutes and its compute credits by over 96%.
- My role
- I designed and built it in 2023 and 2024, handed it to its domain team, and was brought back in 2026 to own it.
- 8 × 36actions and scenarios costed per product
- 32 → 180parallel workers after the memory fix
- ~40 minfor a full run
- 96%fewer compute credits
Read the full story
The first version was about 3,000 lines of SQL. Costs such as storage fees change day by day as inventory ages, and SQL could not express that, so I proposed a simulation and built it in Python. For each product it works out the cost of 8 actions across 36 scenarios, at daily granularity over a two-year horizon, and recommends the action with the best net cashflow.
I built the foundations to be extended. The domain team took the engine over in mid-2024 and ran and extended it for two years.
When I came back to it in 2026, I found that memory, not hardware, was what limited its parallelism. Calling Python's gc.freeze() before forking let worker processes keep sharing the parent's memory instead of copying it, which took the job from 32 workers to 180. I then changed the unit of parallelism, removed wasted work, and extended the simulation to advertising.
Then I stopped. The parallel step had fallen to about 2.5% of the run's time, so making it free would have sped the whole run up by about 3%.
Python multiprocessing, SQL, Metaflow.