Grand Theft Auto V and other open world computer games are being used by researchers to train artificial intelligence systems that could one day power computers responsible for self-driving cars.
Commercially available assisted driving systems, as well as most prototype autonomous vehicle technologies, work best in closed environments, such as freeways and motorways, where the number of variables that need to accounted for by the computer are limited.
These types of roads are more tightly regulated than urban or suburban streets, where lanes are often unmarked, cars are parked, kids play ball, pets dash out, work crews toil away, cyclists abound, buildings encroach, and pedestrians cross whenever they think the coast is clear.
Machine learning, whether used for autonomous vehicles or other purposes, requires a large amount of data to be ingested, classified, and then verified.
This can be problematic for developers of self-driving vehicle technology as acquiring video of various real-world driving situations and tagging them up with the necessary data labels — such as tree, road, traffic light, pedestrian, car, building, footpath and so forth — is a time consuming and painstaking venture.
Researchers at Intel Labs and Darmstadt University have found a way to speed up the process by using the open world environment of Grand Theft Auto V to help train up their computer vision system.
They wrote software, which sits between the game and the operating system, and captures calls from the game to the underlying graphics hardware to draw shapes and other graphical elements. With this data the software is able to quickly, and in real time, classify the objects seen on-screen through the car’s front windscreen.
In the team’s recently released paper, using data gathered via GTA V “significantly increases accuracy” of the computer vision system.
A benefit of using the in-game world is that it contains a diverse variety of driving situations, from downtown traffic to clogged expressways, desert highways and foggy mountain passes, within relatively easy access. On top of this, the game also contains life-like renderings of buildings, cars, trucks, pedestrians, motorcylists, and other road users.
Alireza Shafaei, co-author of another paper, told the MIT Technology Review that, along with a professor at the University of British Columbia, they were able to show that “synthetic data is almost as good, or sometimes even better, than using real data for training”.
MORE: Autonomous driving news