At CES 2026, Motional's CEO told a room full of reporters something most autonomous vehicle companies don't say out loud. Less than 1 percent of the driving data the company's fleet collects is actually useful. The other 99-plus percent is highway cruising, clear intersections, and routine merges, the kind of driving any reasonably trained system already handles without thinking.
That admission is why Motional built a system called Omnitag, whose entire job is mining the useful clips out of the haystack automatically. It is also the quiet, expensive truth sitting underneath nearly every serious autonomy program, in cars and in warehouses alike. Volume was never the hard part. Finding and capturing the rare moment that actually teaches the model something new is.
This article is about edge case data collection: the deliberate work of finding, triggering, staging, and sometimes simulating the scenarios that sit in the long tail of a model's experience. It is a close cousin of the general real-world robustness problem we cover in how physical AI handles unpredictable environments, but it is a narrower, sharper question. Not "how do we make robots robust to surprise in general," but "how do we actually get our hands on the rare, dangerous, boundary-case data before it bites us in deployment."
The short version
Edge case data collection is the practice of deliberately finding and capturing the rare, high-impact scenarios a robot or autonomous system meets in the real world but almost never sees in routine training data. It is not about recording more hours. It is about mining what you already recorded, building triggers that flag the moment something unusual happens, staging confirmed weak points on purpose, and simulating what is too rare or dangerous to stage for real, then holding a slice of it back so you can actually prove a fix worked.
The real problem isn't volume, it's visibility
Ask most teams how confident they are in their data and they'll point to a counter: hours logged, miles driven, episodes recorded. That number feels reassuring. It is also close to meaningless on its own, because the scenarios that actually decide whether a system is safe are rare by definition, and rare things stay rare no matter how long you keep the camera rolling.
Voxel51 frames this clearly: the bottleneck in autonomous driving has shifted from gathering data to finding the valuable cases already buried inside it, because collecting more just makes the search harder. A rare object, like a donkey wandering onto a road, is unusual. A rare scenario is often just ordinary objects in an unusual arrangement, like a cyclist carrying a ladder. Nothing about that scene looks exotic in a single frame. It is the combination that breaks a model, and combinations don't show up in a keyword search.
That search problem has a real cost attached to it. An iMerit product leader quoted in the same CES 2026 coverage put the automotive reliability bar at roughly 99.997 percent, a number that leaves almost no room for the scenarios a dataset never happened to contain. Dashcam network Nexar says it has collected tens of millions of edge case videos through crowdsourced fleets and uses on-device AI to pick out the valuable clips before they ever leave the device, because shipping everything back to a server would bury the useful footage just as badly as a messy hard drive would. Even the public datasets built specifically to showcase edge cases stay small by comparison. NATIX's open release on Hugging Face runs to 20 minutes of video covering 21 events, which is itself a demonstration of how thin this slice of the world really is.
The edge case capture ladder: five rungs from passive logging to deliberate provocation
Every method for finding rare scenarios trades effort for certainty. At the bottom, you log everything and hope the valuable moment is in there somewhere. At the top, you go out and deliberately provoke the scenario you already suspect is a problem. Thinking of the methods as a ladder, not a menu, makes it obvious why mature programs climb through several rungs on the same scenario instead of picking one and stopping.
Each rung trades speed and cost for certainty that the scenario is real and reproducible. Serious programs use several rungs on the same suspected weak point, not just one.
Passive logging
Record everything by default and sort it out later. Warehouse robotics company NoMagic calls its version of this the Library of Chaos, a continuously growing dataset built from live, 24/7 operations rather than staged experiments, capturing every unusual grasp and packaging failure as it happens. A Web Summit Vancouver recap of the company's work puts the scale at tens of millions of picks across hundreds of thousands of unique SKUs, with the edge cases that matter making up less than 1 percent of all operations. Cheap per hour, expensive to search.
Mining and flagging what you already recorded
Once passive logs pile up, the job shifts to search. Motional's Omnitag exists for exactly this: an automated system for mining useful clips out of fleet footage that is mostly routine. In a more academic version of the same idea, the AutoMine team used large language and vision-language models to write and refine code that mines critical scenarios directly from driving logs, scoring well in the Argoverse 2 Scenario Mining Challenge at CVPR 2026. On the labeling side, an Auto Trader report on autonomous vehicle edge cases describes a schema built around environmental conditions, road structure, traffic type, and foreign objects, first applied by hand and now suggested by a vision-language model that a human reviews.
Trigger-based capture from real incidents
Instead of searching logs after the fact, instrument the system to flag the moment itself: a disengagement, a human takeover, a near-miss. A 2026 UC Berkeley Institute of Transportation Studies report used large language models to read NHTSA crash narratives and cluster unusual crash groups, finding that edge cases disproportionately involve unusual crash-partner behavior, non-motorist involvement, roadway anomalies, and cases where the automated system had already disengaged. Each of those is a trigger a collection program can watch for directly, rather than waiting to stumble across it in a log review.
Deliberate reproduction and staging
Once a scenario is confirmed real, the next move is to recreate it on purpose and vary it. Human-interaction and collision-adjacent cases need a safety design before anyone stages them: reduced speeds, mannequins before live actors, and a reviewed protocol, the same discipline behind Gamasome's edge case data collection work. The point of staging is not one clean take. It is enough lighting, pose, and object variation around the confirmed scenario that a model learns the pattern instead of memorizing a single moment.
Synthetic generation for the untouchable tail
Some scenarios are too rare, too dangerous, or too illegal to stage for real at all. A corner-case ontology proposed in a 2022 ECCV workshop paper converts structured scenario descriptions into the OpenSCENARIO format and runs them in simulation to generate both test and training data. More recently, the Foresee framework uses a forecaster to predict a simulated vehicle's possible future states and flags near-misses as candidates worth testing further, reporting it exposes well over double the failures a random search finds in CARLA scenarios, while running faster. A University of Michigan study published in Nature Communications took a similar near-miss-in-simulation approach and reported a large improvement in safety performance, though the press coverage available doesn't specify the baseline behind that figure, so treat the exact number as directional rather than audited.
What people get wrong before they even start collecting
Treating rare events as noise
Some teams still run a cleanup pass that quietly drops the "weird" clips before training, on the assumption that outliers hurt generalization. NoMagic's framing of its own warehouse data is the opposite: edge cases aren't noise to filter out, they're the most valuable training signal available, because they're exactly where the model is currently failing.
Chasing volume instead of coverage
A dashboard that reports millions of hours collected looks impressive and tells you almost nothing about whether those hours included the lighting condition, the object arrangement, or the human behavior that actually breaks the system. Coverage against a written list of known risk categories is a better signal than raw duration.
Skipping the held-out set
Once a rare scenario is captured, the instinct is to throw every example straight into training. Without reserving some variations purely for evaluation, there is no honest way to tell a genuine fix apart from a model that simply memorized the clips it was shown.
Staging human scenarios without a safety design
Jumping straight to live actors for a collision-adjacent or human-interaction edge case is how staging sessions turn into incidents of their own. Mannequins, reduced speeds, and a reviewed protocol come first, and live actors only enter once the staged version is proven safe.
A field note on honesty
Even companies that have built their entire operating model around edge cases don't claim to have solved them. NoMagic, whose Library of Chaos exists specifically to surface rare warehouse failures, has said plainly that its own models aren't fully reliable on their own yet, which is exactly why a classical safety layer still catches what the learned model misses. Edge case programs that quietly imply "we found them all" are the ones worth doubting.
Our VLAs aren't at 99.9% success on their own yet. — NoMagic, quoted in Fortune, July 2026
Turning captured scenarios into something you can evaluate on
Finding a rare scenario and capturing footage of it is only half the job. The other half, the one almost every program underfunds, is structuring that footage so it can actually prove something later. A clip with no scenario tag, no severity score, and no held-out copy is just an interesting video.
| If you have... | What's actually missing | Start with |
|---|---|---|
| A pile of flagged clips from mining or triggers | A consistent way to know if two clips are the same scenario | A scenario taxonomy applied at tagging time, not after the fact |
| One confirmed example of a dangerous edge case | Enough variation for a model to learn a pattern, not one instance | Deliberate staging across lighting, pose, and object variation |
| A scenario too rare or risky to stage for real | Safe coverage without waiting for it to happen again | Simulation, validated against a small real-world sample |
| Training data that already includes some tail cases | Any way to prove a fix worked rather than just assume it | A held-out split from the main training set, reserved for evaluation only |
| A recurring "weird failure" with no clear cause yet | A trigger to catch it live instead of searching for it after the fact | Instrumented logging for disengagements, overrides, or confidence drops |
The categories a vendor blog from Macgence lists as common robotics edge cases, occluded objects in crowded scenes, sudden glare or shadow, damaged or irregularly shaped items, and workers stepping into a robot's path, are a reasonable starting taxonomy. The company also claims addressing those specific outliers improved deployment performance by 35 percent in one case, which is a single, vendor-reported figure that nobody has independently verified, so treat it as a plausibility signal rather than a benchmark.
What this means for a team starting today
Start by naming the categories of rare scenario your system actually faces, not a generic list copied from somewhere else, and tag every candidate clip to one of them the moment it's flagged. Build at least one trigger into your running system, a disengagement log, a confidence threshold, a human override, because waiting to find rare events through manual review of routine footage is the slowest and most expensive way to find them.
Decide which confirmed scenarios get staged for real and which get simulated first, based on how dangerous or rare they are rather than which is more convenient to set up this week. And before any of that data touches a training run, carve out a held-out slice you will never train on, because that is the only thing standing between "we fixed it" and "we hope we fixed it."
This problem shows up everywhere physical AI meets the real world, not only in cars. The automotive long tail we cover in data collection in the automotive industry is one of the clearest large-scale examples, where a system running at very high accuracy can still face a meaningful rate of unhandled scenarios per mile driven. And the five sources of surprise we map out in how physical AI handles unpredictable real-world environments are a useful lens for deciding which surprises deserve a dedicated edge case program and which are better solved by changing the environment instead. For teams building the broader data pipeline this feeds into, our guides to robotics data collection at scale and embodied AI data collection cover the volume side of the same problem this article's long tail sits on top of.
