Realistic Image Synthesis

Brian Smits
University of Utah

What is Realistic Image Synthesis?

Image synthesis is the process of creating images. This is a very broad definition, so we limit it to the creation of images from a model of some scene. The image synthesis pipeline is shown in Figure Once a model is created, some global processing is often done. This processing prepares the environment for rendering through such things as the propagation of energy throughout the environment or optimizations for display. The rendering stage projects the processed model onto an image or set of images. Finally, the image must be displayed on some sort of output device.

Realistic image synthesis is the process of creating images that are, in some way, accurate representations of a real scene. Often, but not always, the images are meant to be viewed by a human observer. Therefore, the accuracy is with respect to the human visual system. Sometimes the image needs to be predictive, guaranteeing that the viewer would have the same visual experience if they were actually in the scene. In some cases the image need only be plausible, the viewer is convinced that the scene could actually be real.

Components of Image Synthesis

There are many different areas that overlap or are used in image synthesis.

Mathematics

Linear algebra, integral equations, probability.

Perception

How the eye works, what the brain ``sees''.

Physics

Optics

Algorithms

Much of graphics is about developing efficient algorithms to deal with the complexity of the environments.

Systems

Usually graphics projects quickly turn into multi-component systems. A good understanding of software engineering is useful too.

Why is it Needed?

Realistic image synthesis is a visualization tool for things the did, do, or might exist. This can reduce design time and improve the designs products, architecture, and environments. If the images are accurate enough, they can be used to guarantee certain visual information is available. This can increase immersion, and provide a certain level of confidence in simulators used for training purposes and education.

The groups that would most benefit from realistic image synthesis are:

• Designers concerned with appearance.
• Simulation and training.
• Education.
• Entertainment industry.

Target applications:

• Predicting visibility from a cockpit or a car. Is there glare obstructing the instruments? Can the driver see the child through the fog? What if the driver were 60?
• Architectural design. Is the building pleasing? Is there sufficient lighting for the tasks that will be performed? What can be seen from the inside looking out? What can be seen from the outside looking in?
• Special effects or advertising, where a virtual image is presented as something real and may be merged with live action, requiring matching the lighting of the set and the exposure of the resulting image.
• Virtual sets, which can eliminate the cost of maintaining physical sets over the lifetime of the show, and can allow effects physical sets can't, but which need to look rich enough to be real and allow the inclusion of filmed actors.
• The reconstruction of historic buildings or places for the purpose of education or archaeological research.