How photogrammetry brought Resident Evil's world of undeath to life

[sincity]

A symphony of staccato shutter snaps filled the air. Xenon gas strobed from dozens of camera flashes. Capcom was visiting a meat processing plant with a trove of camera equipment.

Iron-stomached photographers shot photo after photo of wild game meat along with ivory bones and flowing blood. Afterwards, they ordered tofu and pizza, but not to cleanse their palates. Those, too, were photographic fodder. 

The meat, bones, blood, and viscera translated into zombified flesh in their recent remake of Resident Evil 2. The tofu photographs helped them create the character Tofu from the game's eponymous Tofu Survivor mode. As for the pizza, that was actually one of the first images audiences saw in the game's E3 2018 trailer as a rat skittered across it.

The ability to scan objects and then separately collect texture data has existed for some time, but photogrammetry captures both at once. Streamlining that process allows us to create accurate, properly textured 3D models faster than ever before.

Members of the RE2 dev team

The process that Capcom employed to transduce multitudes of photos into what players saw reflected in their world of undeath is called photogrammetry, and Capcom has been tapping into the art and technique of this process since 2017's Resident Evil 7.

"The photos we take are able to capture fine-grained details such as an object's texture, which in turn allows us to create realistic, high-quality 3D models,” members of the Resident Evil 2 development team told us in an interview. 

“The ability to scan objects and then separately collect texture data has existed for quite some time, but photogrammetry captures both at once. Streamlining that process allows us to create accurate, properly textured 3D models faster than ever before."

Streamlining processes like rendering is key when you want to make big budget games: it reduces cost, saves time and ultimately, creates a better product - in this case, one of the highest-rated games of 2019.

Photogrammetry 101 

The field of photogrammetry uses photographs to take precise and advanced measurements and has applications that range from cartography to archaeology.

The game industry has, over the last several years, adopted the science of photogrammetry to create highly detailed 3D models like the those found in Resident Evil 2's art deco museum-turned-police station and the sanitized white laboratory stained in stark relief with arterial spurts of blood.

This process works by first taking an array of overlapping photos of a subject from all angles to build a comprehensive set of measurements of the subject in 3D space. The photos are then stitched together using specialized software which takes into accounts for scale, angle, and perspective. In that way, a series of 2D images become a seamless 3D reconstruction.

Overlapping aerial photographs can be used to create topographical maps, whereas 3D modelers more often get up close and personal with their subjects in employing this technique.

Photogrammetry is also remarkably scaleable. A cursory glance around the internet reveals tutorials on how to do rudimentary photogrammetry with a smartphone camera. Of course, the technology employed by AAA-development studios is a bit more advanced than that.

A match made in heaven (or... hell) 

This versatile approach to creating high-fidelity 3D models has allowed Capcom to push the boundaries into literally photorealistic territory. Not only can it be applied to architecture and organic materials, but facial capture and even details as minute as creases and wrinkles in clothing as well.

"For Resident Evil 2, we hired live models who fit the descriptions of our characters, fitted them with the appropriate costumes, and then used photogrammetry to capture the data from which we built the 3D models. The process was pretty similar to filming a live-action movie. Many of the objects in the game were created using the same process," the developers told us. 

"Something we had to contend with in this game was the difference in size between the stunt actors and the facial model actors. We had to find a way to link the two of them by performing some initial facial scans to use as a target. We also use photogrammetry to capture the contortions that appear in the actors' outfits as they move around and in their faces as they emote."

http://cdn.mos.cms.futurecdn.net/RxNiC4vUcdixiomwZMvWT5.jpg

To accomplish this, Capcom's 3D-scanning studio houses 141 commercially available digital single-lens reflex cameras across two booths. One booth is dedicated to creating lavishly-detailed full-body scans composited from photos taken across 103 camera, while the other 38 reside in a separate booth dedicated to facial scans. 

According to Capcom, the number of photos needed per object or person is directly proportional to the desired quality. They took, on average, about 100 photos for each model in the game for things like ...

Is the future of gaming picture-perfect? 

According to the Resident Evil 2 development team, there are a few drawbacks to employing photogrammetry - some objects can be cost-prohibitive to either procure or manufacture, while other objects that are constantly in motion (like trees and foliage that could sway in the wind) can introduce noise into the 3D data when converted if they move during the scanning process. 

That’s also not even mentioning the setup cost, which is rather sizable when you factor in a scanning studio and acquiring the equipment therein.

"However, assuming the object we want is easily deliverable, the quality of the final product compared to what's possible with traditional methods is more than worth the extra cost," according to Capcom. "We take the investment and operating costs of the photogrammetry studio into consideration as well, but we think this is an incredibly effective way to create assets, particularly in terms of the quality most contemporary games demand."

That is why photogrammetry has continued to leave a footprint on the game development industry, and why Capcom has said they will continue to utilize it for their future projects. 

Moreover, Capcom tells us, they are looking towards a future where it isn't something noteworthybut another commonplace tool in the belts of creators.

"Photogrammetry will eventually cease to be special, and in some ways it's already reaching that point. What's important is to have a kind of auteur mindset that is always aiming for higher quality and bringing even more value to the title. Photogrammetry is, in the end, just a part of the foundation used to create games and images."

http://feeds.feedburner.com/~r/techradar/digital-home/~4/XpISEYvGqYk