Trench coats, umbrellas and bright sunlight can be the enemy of pedestrian detection.

Volvo Pedestrian and Cyclist Detection System - 2

Autonomous emergency braking (AEB) with pedestrian detection is one of the must-have safety features in modern cars, but it's not infallible. Whether it's false positives caused by reflective signs or systems failing during demonstrations, semi-autonomous safety assists can be fooled.

We're aware no system is perfect, as evidenced by this particularly public stuff-up, but a list published in Toyota's owner manual highlights the full list of scenarios where its pedestrian detection software might not actually detect a pedestrian. It's worth bearing in mind, these limitations aren't exclusive to Toyota, and similar lists exist for most brands.

The list of things it mightn't be able to recognise is below:

  • Pedestrians shorter than 1m or taller than 2m
  • Pedestrians wearing oversized clothing (raincoat), making their silhouette obscure
  • Pedestrians carrying large baggage, holding an umbrella (etc.) hiding part of their body
  • Pedestrians bending forward or squatting
  • Groups of pedestrians who are extremely close together
  • Pedestrians wearing white, who look extremely bright
  • Pedestrians in the dark (eg. at night, or in a tunnel)
  • Pedestrians whose clothing appears to be nearly the same colour/brightness as their surroundings
  • Pedestrians near walls, fences, guardrails or large objects
  • Pedestrians standing on a metal object like a manhole cover
  • Pedestrians walking fast, or abruptly changing their speed
  • Pedestrians walking out from behind a vehicle or large object

Exhaustive, right? The list was posted on Twitter by user Timothy B. Lee.

It's worth bearing in mind, these systems are only semi-autonomous, and don't claim to be perfect. Toyota relies on a radar sensor behind the badge, and a camera mounted behind the windscreen for the operation of its semi-autonomous assists, which include AEB, radar cruise control and lane-keeping assist.

Those sensors can be impacted by dust or road grime buildup, glare or complex visual scenes (people standing in front of odd-shaped objects, or holding things like umbrellas), potentially undermining their effectiveness. Even Tesla, which has painted itself as a leader in the autonomous field with Autopilot, has previously struggled with object detection.

These challenges are part of the reason Lexus international president, Yoshihiro Sawa, told media we need to be conservative with the rollout of autonomous driving in cities.

“Autonomous [technology] is not perfect yet... especially the city area, there are many unexpected situations happening," he said.

"Autonomous [driving] in city is a very dangerous situation sometimes, but freeway or toll road things are quite different. So I think first of all we focus on the freeway and highway to satisfy the autonomous [demands], but then later on we are going to introduce autonomous for city area," he continued.

"But before [that] we focus on safety to reduce the collision or accident or injury. Some may say that’s conservative but our priority is more on the safety issue.”