The Glovo lesson: automation needs a human veto

A Glovo delivery worker died in a road accident, and the company's automated system, seeing only an uncompleted delivery, fired him by text. No person chose that outcome. The system did exactly what it was built to do: enforce speed, at scale, without asking questions. The anatomy of the failure is worth memorizing: speed as the primary KPI, no human review of irreversible actions, no ethical safeguards, and no thought given to how the system behaves when reality goes wrong.

Why it matters to the business

Your automation speaks and acts for you, legally and reputationally. In 2024 a British Columbia tribunal held Air Canada liable for a bereavement-fare policy its chatbot invented; the company argued the bot was responsible for its own words, and lost. Customers are already wary of machines in the loop: a Gartner survey of 5,728 customers found 53% would consider switching to a competitor over AI in customer service, with the top concern being that reaching a human would get harder. One automated cruelty, even a rare one, confirms that fear publicly and permanently. And the lesson reaches past customers: gig workers, partners, and employees live inside these systems too.

How to use it

• Inventory every automated action that is irreversible or high-stakes: terminations, account closures, claim denials, suspensions.
• Require a human review before any of those actions fires, with real authority to stop it.
• Write failure-mode requirements that specify what the system does when data is missing, contradictory, or anomalous.
• Audit what each speed or throughput KPI rewards in its worst edge case, not its average case.
• Name an accountable owner and build a kill-switch for every automated decision system.

Where teams get it wrong

Teams review the happy path and ship. If an ethics review happens at all, it covers the customer-facing screens, while backstage automation acting on workers and partners goes unexamined. The gap stays invisible until an edge case meets an irreversible action, and by then the headline writes itself.

Ask your team

• List every irreversible action our systems can take without a human. Who signed off on each one?
• What does our automation do when its data is wrong, and when did we last test that?
• Which KPIs drive our algorithms, and what behavior do they reward in the worst case?

No one designed that outcome. The incentives did.