Fig. 11.16 Segmenting the left ventricle from MRI using the 3D balloons technique implemented in ITK-SNAP. (a) Initial balloon placement. (b), (c) Intermediate stages. (c) Final segmentation. Each image contains 4 panels: sagittal slice (top left), coronal slice (top right), axial slice (bottom right) and rendering of segmented surface (bottom left). The segmentation took approximately 5 seconds to complete

11.5.2 Fully Automatic Methods

Development of robust, fully automatic segmentation techniques has proved a challenging task and has been an active research topic for many years now. A common approach is to introduce some prior knowledge of the likely shape of the object being segmented. There are many different ways of introducing such knowledge. Here we discuss two of the most common ways, which introduce prior knowledge in slightly different ways.

11.5.2.1 Atlas-Based Segmentation

Atlas-based segmentation techniques introduce prior knowledge in the form of an atlas. Here, an atlas refers to a sample intensity image of the region of interest to-

gether with a corresponding segmentation. The segmentation need only be produced once, so a time-consuming (but accurate) manual or semi-automatic approach can be used. The basic approach of atlas-based segmentation can be summarized as follows:

•register the atlas intensity image to the subject image that we want to segment, then

•use the motion fields from the registration to propagate the atlas segmentation to the subject image.

An example for the purpose of hippocampus segmentation from brain MRI images was described by Carmichael et al. [17]. The atlas intensity image was registered to the subject image resulting in a geometric transformation. The atlas segmentation then underwent the same geometric transformation to produce the subject’s segmentation estimate.

Due to the sometimes large variation in anatomy, there may be some cases in which the registration phase fails. For this reason a more sophisticated scheme has been proposed using multiple atlas intensity images and segmentations. These are known as multi-atlases [68]. With multi-atlases, a new subject image can be registered to the atlas image to which it is most similar, increasing the chance that the registration will be successful. Alternatively, a database of images can be combined to form a single average atlas image [67] or a probabilistic atlas [22, 49].

Atlas-based segmentation has been applied to many different organs, such as the heart [49, 86], the brain [17, 18, 22], the prostate [41] and the liver and spleen [46].

11.5.2.2 Statistical Shape Modeling and Analysis

Another way to introduce prior knowledge into an automatic segmentation technique is to use a statistical shape model (SSM). SSMs explicitly capture the likely shape and variation in shape across a population. They have also been known as active shape models or point distribution models [21].

Based on a set of instances of a shape (the training set), SSMs use principal component analysis to compute a mean shape and its principal modes of variation. In medicine, the training set is typically derived from medical imaging data such as CT or MRI images.

The construction of a SSM involves a number of distinct steps. First, the n instances in the training set must be aligned to a common coordinate system. This can be done using image registration techniques. Next, for each shape instance we define a 1D shape vector, which we denote by x, where:

This vector represents a parameterization of the shape. Common parameterizations used for SSMs include surface landmark coordinates [21] or interior control points [70]. Whatever the parameterization used, the values must be extracted from the training images, either manually or automatically.

From the mean shape and the training set of shape vectors, we compute the covariance matrix, S,

The covariance matrix contains information about how the different parameters of the shape vectors vary with each other (or co-vary) away from their mean values. The eigenvectors of this covariance matrix summarize the major modes of variation of the shape vectors. Therefore, next we compute the non-zero eigenvalues of S and their corresponding eigenvectors. These, along with the mean shape, define the SSM. We denote the eigenvectors by φi and the eigenvalues by λi . The eigenvectors will be 1D vectors of the same length as the shape vectors. Intuitively, they represent the ways in which the shape parameters (e.g. surface landmarks) change together. Each eigenvector defines a particular change in the value of all shape parameters, e.g. a direction of variation for all surface landmarks. The larger the eigenvalue, λi , the more of the total variation of the shape population that the corresponding eigenvector represents. If there are n samples in the training set and the eigenmodes are sorted from largest to smallest eigenvalue, then the cumulative variance for the

variance matrix has rank n − 1 and only the first n − 1 eigenvalues will be nonzero.

Finally, we can compute a new shape instance from the SSM by defining a set of weights, bi , for the eigenvectors. For example, using the largest m eigenvalues, a new shape instance xˆ is produced as follows,

It is common to choose the value of m so that the eigenvectors used account for a certain percentage of the total variation (e.g. 95 %) in the shape population. Since the eigenvalues represent the variance of the data in the direction of the corresponding eigenvector, they can be used to compute the proportion of the total variance that the

Figure 11.17 shows an illustration of a SSM of the femur. The left column shows

and right columns show femur surfaces generated from the second and third most significant modes respectively.

Fig. 11.17 An illustration of the largest three modes of variation of a SSM of the femur (+3σ on top row and −3σ on the bottom for modes 1–3 (left to right). The first mode largely corresponds to size, whereas the second mode is dominated by the angle of the femoral head

SSMs can be applied to the problem of segmentation by using them to constrain a registration between the shape vector and a subject image of the organ. A similarity measure between the shape vector and the subject image is defined, typically based upon image intensities [15, 29] or gradients [24], or features extracted from the images [48, 78]. The SSM weights are typically optimized to produce a shape vector that maximizes this similarity measure.

SSMs have been applied to segmentation in a wide range of anatomy. For example, Fripp et al. [29] used SSMs to segment knee bones from MRI images, Tao et al. [78] used them for segmentation of sulcal curves from MRI images of the brain and de Bruijne et al. [14, 15] for segmenting abdominal aortic aneurysms from CT data. In addition, whole heart segmentation has been demonstrated from CT [24, 48], MRI [62] and rotational X-ray images [52].

Figure 11.18 illustrates a sample whole heart segmentation from CT produced using the SSM-based algorithm described in [24]. The segmentation is visualized using ITK-SNAP and shows three orthogonal views through the CT volume (topleft, top-right and bottom-right panels), overlaid with color-coded segmentations of the four chambers and major vessels of the heart. The bottom-left panel shows a 3D rendering of the segmented surfaces from an anterior-posterior view. A good review on the use of SSMs for 3D medical image segmentation can be found in [34].