Grand Theft Auto V and other open world computer games are being used by researchers to train artificial intelligence systems that could one day power computers responsible for self-driving cars.
Commercially available assisted driving systems, as well as most prototype autonomous vehicle technologies, work best in closed environments, such as freeways and motorways, where the number of variables that need to accounted for by the computer are limited.
These types of roads are more tightly regulated than urban or suburban streets, where lanes are often unmarked, cars are parked, kids play ball, pets dash out, work crews toil away, cyclists abound, buildings encroach, and pedestrians cross whenever they think the coast is clear.
Machine learning, whether used for autonomous vehicles or other purposes, requires a large amount of data to be ingested, classified, and then verified.
This can be problematic for developers of self-driving vehicle technology as acquiring video of various real-world driving situations and tagging them up with the necessary data labels — such as tree, road, traffic light, pedestrian, car, building, footpath and so forth — is a time consuming and painstaking venture.
Researchers at Intel Labs and Darmstadt University have found a way to speed up the process by using the open world environment of Grand Theft Auto V to help train up their computer vision system.
They wrote software, which sits between the game and the operating system, and captures calls from the game to the underlying graphics hardware to draw shapes and other graphical elements. With this data the software is able to quickly, and in real time, classify the objects seen on-screen through the car's front windscreen.
In the team's recently released paper, using data gathered via GTA V "significantly increases accuracy" of the computer vision system.
A benefit of using the in-game world is that it contains a diverse variety of driving situations, from downtown traffic to clogged expressways, desert highways and foggy mountain passes, within relatively easy access. On top of this, the game also contains life-like renderings of buildings, cars, trucks, pedestrians, motorcylists, and other road users.