post production. Figure 1.14 shows the camera system, one of the original views, the depth and color views from the enhanced scene, including real and virtual content. Without range data, such scene composition is a tedious manual task, as used in traditional video post processing today. With the use of depth information, there is a wealth of opportunities to facilitate and to automate data processing.

Although many different techniques have been proposed for a wide variety of 3D imaging applications, many of them work well only in constrained scenarios and over a subset of the available datasets. Many techniques still fail on challenging data and techniques with more robustness and better overall performance still need to be developed. Deeper insights into various 3D imaging, analysis and application problems are required for the development of such novel techniques.

1.7 Book Outline

We conclude this chapter with a roadmap of the remaining chapters of this book. Although there is a natural order to the chapters, they are relatively self-contained, so that they can also be read as standalone chapters. The book is split into three parts: Part I is comprised of Chaps. 2–4 and it presents the fundamental techniques for the capture, representation and visualization of 3D data; Part II is comprised of Chaps. 5–7 and is concerned with 3D shape analysis, registration and matching; finally, Part III is comprised of Chaps. 8–11 and discusses various application areas in 3D face recognition, remote sensing and medical imaging.

1.7.1 Part I: 3D Imaging and Shape Representation

Chapter 2 describes passive 3D imaging, which recovers 3D information from camera systems that do not project light or other electromagnetic radiation onto the imaged scene. An overview of the common techniques used to recover 3D information from camera images is presented first. The chapter then focuses on 3D recovery from multiple views, which can be obtained using two or more cameras at the same time (stereo), or a single moving camera at different times (structure from motion). The aim is to give a comprehensive presentation that covers camera modeling, camera calibration, image rectification, correspondence search and the triangulation to compute 3D scene structure. Several 3D passive imaging systems and their realworld applications are highlighted later in this chapter.

In Chap. 3, active 3D imaging is discussed. These systems do project light, infrared or other electromagnetic radiation (EMR) onto the scene and they can be based on different measurement principles that include time-of-flight, triangulation and interferometry. While time-of-flight and interferometry systems are briefly discussed, an in-depth description of triangulation-based systems is provided, which have the same underlying geometry as the passive stereo systems presented in Chap. 2. The

characterization of such triangulation-based systems is discussed using both an error propagation framework and experimental protocols.

Chapter 4 focuses on 3D data representation, storage and visualization. It begins by providing a taxonomy of 3D data representations and then presents more detail on a selection of the most important 3D data representations and their processing, such as triangular meshes and subdivision surfaces. This chapter also discusses the local differential properties of surfaces, mesh simplification and compression.

1.7.2 Part II: 3D Shape Analysis and Processing

Chapter 5 presents feature-based methods in 3D shape analysis, including both classical and the most recent approaches to interest point (keypoint) detection and local surface description. The main emphasis is on heat-kernel based detection and description algorithms, a relatively recent set of methods based on a common mathematical model and falling under the umbrella of diffusion geometry.

Chapter 6 details 3D shape registration, which is the process of bringing together two or more 3D shapes, either of the same object or of two different but similar objects. This chapter first introduces the classical Iterative Closest Point (ICP) algorithm [5], which represents the gold standard registration method. Current limitations of ICP are addressed and the most popular variants of ICP are described to improve the basic implementation in several ways. Challenging registration scenarios are analyzed and a taxonomy of promising alternative registration techniques is introduced. Three case studies are described with an increasing level of difficulty, culminating with an algorithm capable of dealing with deformable objects.

Chapter 7 presents 3D shape matching with a view to applications in shape retrieval (e.g. web search) and object recognition. In order to present the subject, four approaches are described in detail with good balance among maturity and novelty, namely the depth buffer descriptor, spin images, salient spectral geometric features and heat kernel signatures.

1.7.3 Part III: 3D Imaging Applications

Chapter 8 gives an overview of 3D face recognition and discusses both wellestablished and more recent state-of-the-art 3D face recognition techniques in terms of their implementation and expected performance on benchmark datasets. In contrast to 2D face recognition methods that have difficulties when handling changes in illumination and pose, 3D face recognition algorithms have been more successful in dealing with these challenges. 3D face shape data is used as an independent cue for face recognition and has also been combined with texture to facilitate multimodal face recognition.