Audi A7 Piloted Driving concept: In-depth look at how an autonomous car is developed and how it works

At the show, we spoke to Daniel Lipinski, project leader for Audi’s automated driving projects, about how his team developed the A7 Piloted Driving car, nicknamed Jack, and all the ins and outs of the technology under its skin.

Compared to a regular A7 that you or I can buy, assuming we had enough dosh or credit, of course, the semi-autonomous A7 is equipped with a clutch of extra sensors: a laser scanner (below) embedded in both the front and rear bumpers, and mid-range radars at each corner of the vehicle.

The lasers monitor goings-on in front and behind the car, while the mid-range radars keep track of what’s happening to the sides. Jack also uses the latest megapixel camera from the second-generation Q7, while some other devices have been upgraded to the latest available versions.

Unlike production vehicles, the network of sensors in Jack have been engineered to provide a level of redundancy. Lipinski says that “this ensures that even if one sensor fails [the car] can operate for the next 15 to 20 seconds, and give the driver enough time to take over”.

All up these perception units provide the car with a 360 degree view of its surroundings and data from them is fed into the car’s decision making module. Housed in computers located in the boot, the decision making engine knows where the car wants to go and the car’s dynamic capabilities. As such, it has to continuously determine whether to stay in the same lane, plan trajectories around other vehicles, see whether it should pull into the slow lane to allow other vehicles to pass and, generally, avoid crashing into things.

The last item in Jack's chain of command is the execution module, which can alter the steering angle, and manipulate the brakes and accelerator at the decision making system’s behest.

Lipinski says that the overall route to CES was as simple going to Google Maps, but to ensure that the 885 kilometre trip between Stanford, California and Las Vegas, Nevada, went smoothly, the team had to do a bit of extra planning.

As a level 3 autonomous vehicle, the A7 Piloted Driving Concept has only been designed to handle regular highway conditions; the driver is responsible for getting the car to the highway initially. So, the first task was to mark all the stretches of the route that weren’t on the US' Interstate Highway System.

The team then had to note down the number of lanes for each highway section and the preferred lanes for the A7 to be in. Next up, they had to flag all the human-driven parts of the route, primarily construction zones or stretches of highway with poor lane markings.

In total, Lapinski claims, Jack was able to pilot itself for around 97 per cent of the journey.

Lastly the route had to be broken up by a series of layovers. This way the five journalists brought along for the trip could be rotated through the driver’s seat. These stops also ensured that the car didn’t run out of fuel and gave everyone a chance to stretch or visit the porcelain throne. The journey's waypoints had to be chosen with a little rigour as the A7 travelled as part of a convoy of 13 vehicles.

This contrasts with what rival Mercedes-Benz had to do in order to get its S500 Intelligent Drive to complete a recreation of Bertha Benz’ 1888 journey from Mannheim to Pforzheim. For that 100km trip, Mercedes-Benz enlisted the help of Nokia’s Here mapping division to build a virtual reality map of "every centimetre" of the course using lidar and aerial imagery.

Jack and his siblings were primarily developed in Germany. So, in order to get the A7 Piloted Driving Concept to work properly in the USA, Lipinski and his team had to tweak the car’s software.

Changes made include allowing Jack to overtake in the right, or slower, lane — this manoeuvre is a “big no no” in Germany. Additionally the system had to be informed that many right lanes in America can suddenly become exit ramps, and in some areas there are exit lanes on the left-hand side of the road.

Although it sounds oxymoronic, the five journalists Audi entrusted to fill the autonomous car's driver's seat had to go undergo a round of training and licensing. Firstly, Audi put the journos through the same prototype driving training course that the company's engineers and test drivers go through before they can drive development vehicles. This course involves learning about the car’s behaviour at the limit.

Each state in America that permits autonomous vehicles on public roads has different legal and licensing requirements. California, for example, mandates a specific training plan, while Nevada is a considerably more laissez-faire. In order to comply with Californian legislation, as well as for overall safety, each driver was also put through a programme at Volkswagen’s proving grounds in Arizona, where they were shown all of the auto-driving system's possible manoeuvres and how the car reacts if a sensor fails or the software crashes.

To offer an extra degree of safety, Lipinski or one of his engineers rode shotgun for the entire trip, and had access to emergency braking and steering controls.

Audi has been working on autonomous cars for at least 15 years, and in that time great strides have been made not only in the number of situations that can be comfortably handled, but also in overall refinement. According to Lipinski, it’s some of the smallest changes that have outsized impact in the latter regard.

He cites, as an example, how the team used to engineer its self-driving cars to stick to the centre of its current lane. That seems like a logical enough thing to do, but it felt weird to most test drivers because a majority of us tend to steer towards the open edge of the road when we’re in the outermost lane.

Another minor change prevented the autopilot from steering immediately, and sometimes brusquely, into the middle of the lane when it assumed control of the car.

With these changes, amongst many others, Lipinski says that most of Jack’s new drivers only take a few minutes to feel comfortable once autopilot mode has been engaged.

Inside Jack's cabin there’s a screen in the instrument cluster that displays which mode the car is in (manual or Highway Pilot) and whether it's safe to engage self-driving mode. There’s also a small LCD screen wedged between the central air vents showing what the car’s autopilot system is up to. A light bar at the base of the windscreen and spoken warnings are used to inform the driver if they shortly need to assume control.

Currently Audi’s autonomous prototype vehicles don't monitor the driver’s vital signs, although Lipinski believes that this will be necessary when this technology hits the market. Right now, if the car asks for you to take over and you fail to do so after a reasonable amount of time, the A7 will come to a complete stop, preferably after moving over to the shoulder first.

There’s no publicly stated target date for commercial availability of Jack’s Highway Pilot system, but it will hit the market well before Audi begins selling a fully automated car. It’s part of Audi’s “evolutionary approach” to self-driving vehicles, where less complex systems are completed and sold first. Over time all these systems will meld into one capable of piloting a level five or completely self-driving vehicle.

Until then, there are still plenty of challenges to be conquered, especially on urban and suburban streets where, Lipinski says, "there are so many unknowns” and a long laundry list of things that have to be worked on.