01
TRUSTED MODEL + INPUTNormal AI
—minutes
Run the fitted AI on the untouched held-out evidence.
Source trust100%
CASE-A-THON 2026/THE MEASURED ANSWER
Result: no-go for deployment
More data made delivery ETAs worse.
On this delivery dataset, a high-volume but unreliable feed pushed predictions farther from the real delivery time. The proposed safeguard helped too little and failed other tests.
RECOMMENDATION
Do not deploy yet.Improve the safeguard, then test it in a limited one-region shadow pilot.
Measured LaDe experiment31,415 orders · 24 matched runs · same future deliveries
01 · LIVE AI PREDICTION
Watch one delivery change
Watch one delivery change
when unreliable data reaches the AI.
This is real server-side inference. First run the trusted pipeline. Then run the full degraded scenario—retrained model plus matching degraded inputs—and test the safeguard.
REAL AI · PREDICTION CALCULATED NOW
READY TO RUN
02
DEGRADED TRAINING + INPUTAI after bad data
—minutes
Use the model retrained on the simulated unreliable feed and this delivery’s matching degraded inputs.
Change from clean—
03
BUNDLED SAFEGUARD POLICYAI with safeguard
—minutes
Weight trust, back off suspicious inputs, and check review risk.
Human review—
WHAT THIS DEMONSTRATESStart with the clean prediction.
Each step makes a fresh request to the online backend and evaluates the corresponding fitted model.
ACTUAL DELIVERY TIMEHidden until comparisonUsed only for retrospective evaluation—never as a prediction input.
What is live? Training happened beforehand for speed and repeatability. Every button evaluates the exported fitted trees on the selected held-out delivery in the online Worker. The real outcome is never a model input.
02 · FULL EXPERIMENTStart with the conclusion.
Start with the conclusion.
Open the detail when you need it.
The key scenario is already selected: the noisy feed, with the safeguard off. Turn the safeguard on to see whether it recovers the loss.
HOW TO READ THIS
Read the plain-language result first. The figures and timeline underneath are supporting evidence, not the headline.
STEP 1 · CHOOSE THE DATAWhat reaches the ETA model?
Step 2 · SafeguardWeight, back off, review
NOISY FEED · SAFEGUARD OFF
REAL EXPERIMENT RESULTSPLAIN-LANGUAGE RESULTThe unreliable feed makes ETAs worse.
The feed raises the average prediction miss by 2.1 minutes compared with the trusted-data reference.
Average prediction miss64.2 minlower is better
Sent to a dispatcher10.7%share needing human review
Explore the detailed resultsTimeline, large misses, repeatability, and data-quality signals
Predictions off by 15+ minutes86%large-miss rate
Consistency across repeat runs±0.3smaller variation is better
How far off were ETAs each day?Each point is one evaluation day, averaged across three repeat runs
average missAge of area history0 days
Noisy-feed share55%
Source trust score35%
Handled without review89%
THE FULL READINESS CHECK
The safeguard helps one test—and fails the overall decision.
We repeated the same comparison with trusted data, old data, the noisy feed, and both problems together.
No-go: the safeguard worsens the old-data and combined results, while the review policy sends every old-data and combined case to a dispatcher.
03 · THE MECHANISMFollow the failure,
Follow the failure,
not just the score.
01⇣Data enters
Area history freezes. A low-reliability feed grows to 55% of records.
→
02∿Features bend
Distance and historical ETA drift away from the clean reference.
→
03ƒModel learns
The noisy source contributes more examples than the accurate source.
→
04!Decision changes
ETA error and review workload change on the same future orders.
THE SAFEGUARD
Trust data in proportion to its evidence.
The safeguard changes influence rather than deleting every suspicious record.
1
Reliability weighting
Low-trust training rows count less.
2
Safe input backoff
Suspicious values move toward robust defaults.
3
Human deferral
High-risk outputs show evidence and request review.
04 · HUMAN REVIEWWhen the model hesitates,
When the model hesitates,
the person gets the evidence.
This is a walkthrough, not an illusion of autonomy. The dispatcher can accept, correct, or escalate—and every action is logged.
MODEL ETA
—minutes
Expected range —
WHY REVIEW?
Loading evidence…
Evidence available to the dispatcher
Data age—
Source reliability—
Unreliable feed share—
Ground-truth duration is hidden during review and used only for offline evaluation.
WHO REVIEWSQualified dispatcher
WHEN TO DEFEREvidence crosses approved risk threshold
WHAT GETS LOGGEDEvidence, action, correction, review time
05 · REQUIRED FILESMethods, results,
Methods, results,
and presentation.
Everything the competition requests: setup, data manifest, experiment details, governance, machine-readable results, presentation mode, and a recorded fallback.
Loading documentation…
MACHINE-READABLE EVIDENCE
Take the results with you.
10-MINUTE PRESENTATION
Present the same evidence shown above.
Use the concise slide mode for the live presentation, or the recorded version if the demonstration cannot run.