AI Creates Fictional HD Images Using Real-World Street View Photos

Artificial Intelligence (AI) continues to advance at an unprecedented pace. Although it is not yet ready to create fictional worlds on its own, it is getting close. Recently, researchers at Stanford University and Intel debuted a new AI project that creates fictional street view images out of real-world photos, in high definition.

An AI that Creates Photorealistic Images

What researchers at Stanford and Intel created is essentially an imaginative AI that creates photorealistic street view-like images. The AI creates those images using a rough layout, and from memories of real streets on which it was trained.

According to researcher Qifeng Chen, the AI was fed 5,000 photos of German streets and then, with some human help, built images of nonexistent streets that resembled those it had been fed.

The AI took the layout it had been given and created images with the help of various labels researchers had added to the layout. For example, parts of the layout were labeled “cars,” others were labeled “road,” and so on. The technique, according to Chen, could eventually help create video game worlds that resemble Earth. The algorithm was even used to replace the world in Grand Theft Auto V.

The images, as seen in the video below, have the same quality as photos taken with a two-megapixel camera. The images are rendered in 1,024 x 2,048 HD resolution.

According to Noah Snavely at Cornell University, it is tricky getting AI agents to create such images, and indeed most existing approaches are incapable in this regard. Chen’s images are among the largest and most detailed available so far, and they open up the possibility that a person could describe a world and have the AI build it in virtual reality. He stated, “It’d be great if you could conjure up a photorealistic scene just by describing it aloud.”

According to experts, while these images might not replace high-end special effects, they would be great for virtual reality worlds since people do not expect complete photorealism in VR just yet. The researchers will present their work at the International Conference on Computer Vision taking place in Venice this October.

A Long Way to Go Before Building Photorealistic Worlds

Although these results are pretty astonishing, researchers acknowledge their system still has a long way to go before being able to build truly photorealistic worlds, as the images it produces right now are somewhat blurry and have a dreamlike quality. The AI does not know how to fill every single pixel just yet.

However, the AI was trained and tested on a relatively small database of only 5,000 photos of German streets. If it could capture the true diversity of our world, says Snavely, we would see its true potential. That may prove a challenge, however. Images are currently labeled by hand, meaning the database would need to be extremely detailed, and building such a database is extremely labor-intensive. Only time will tell what the future holds for this promising technology.