370 |
H. Wei and M. Bartels |
•Finally a set of questions and exercises is provided in Sects. 9.8 and 9.9 respectively.
9.2 DEM Generation from Stereoscopic Imagery
Compared to the traditional manual methods that use human operators, automated methods of DEM generation from remote sensing provide efficient, economic and reasonably accurate products covering extended areas of the Earth’s surface. Remote sensing of the Earth’s surface started with photographic film cameras and has been evolving to digital cameras with selective sensing bands, for example, multispectral, thermal, hyperspectral, and radar. In this section, we discuss the issues associated with DEM generation from stereo images that are sensed from natural light in spaceborne missions. As with all stereo 3D reconstruction, two or more images that sense a scene with overlapping areas are required. A stereo image pair can be formed by along-track or across-track arrangement of sensors as shown in Fig. 9.2.
Along-track is defined by the forward motion of the satellite along its orbital path, whereas across-track refers to a satellite traveling on different orbits, hence images covering the same area are taken from different orbits. In general, a stereo pair captured in an along-track mission has a shorter time interval between two images than that captured in an across-track mission [101]. Thus variable weather conditions have less effect on along-track stereo pairs than on across-track stereo pairs in these passive imaging scenarios. The distance between two sensors is called the baseline (B), and the nadir distance (vertical distance) from satellite to ground is referred to as the height (H), as illustrated in Fig. 9.2. The base to height (B/H) ratio is a key parameter in DEM generation from stereoscopic imagery. It is a criterion for choosing an adequate number of stereo pairs from the same or different orbits. This section presents a literature review of the techniques used for DEM generation
Fig. 9.2 DEM generation from satellite stereoscopic imagery. The arrows refer to the satellite’s direction of flight. The low arrowed lines are the orthographic projection of the orbit onto the ground. Orbits 1 and 2 are two designated orbits that meet the requirements of stereo pairs. Left: Along-track. Right: Across-track
9 3D Digital Elevation Model Generation |
371 |
from remotely sensed stereoscopic imagery, followed by a discussion on quality evaluation of reconstructed DEMs. A step-by-step process to generate a DEM from a stereo pair is also presented.
9.2.1 Stereoscopic DEM Generation: Literature Review
The first space mission to provide stereoscopic imagery of the Earth’s surface was the American CORONA2 spy satellite program [52]. Over the past decades, a number of Earth observation satellites have been launched with high resolution imaging systems, such as Landsat (1972), IKONOS (1999), QUICKBIRD (2001), SPOT-5 (2002), ENVISAT (2002), ALOS (2006) and GeoEye-1 (2008). Stereo images acquired by these satellites can be along-track image pairs or across-track image pairs. DEM generation from a stereoscopic image pair involves the following processes [51, 68].
•Pre-processing of image pairs for noise removal: this is a process to mitigate the effects of noise introduced by the image sensors.
•Image matching: this is the process of finding corresponding points in two or more images and is implemented by either area-based or feature-based matching, or a combination of both.
•Triangulation process: image coordinates of matched points from the image pairs
are transformed into ground coordinates using the cameras’ interior and exterior parameters.3 This process involves geometric modeling of the satellite camera system and the ground coordinate system.
•Evaluation of the reconstructed DEM: this process can be achieved by means of ground control points (GCPs), if available.
As indicated in Marr and Poggio’s pioneering research on the computational theory of human stereo vision [109], there are two issues to address with respect to 3D reconstruction from stereo image pairs: correspondence and reconstruction (please refer to Chap. 2 of this book for the details). A key issue in automatic DEM generation is the process of image matching (solving the correspondence problem). Great efforts have been made by researchers from both remote sensing and computer vision communities in the 1980s [2, 11, 45, 57, 117, 130] to explore approaches in this field. In contrast to area-based cross-correlation, which dominated the field of image matching since the early 1950s, techniques developed in the period of the 1980s involved feature-based approaches. The combination of area-based and edgebased matching was attempted and applied by Förstner [45], Ackermann [2], and
2The CORONA program started in 1956 as a series of American strategic reconnaissance satellites. CORONA mission 9031 launched on 27th Feb. 1962 and was the first satellite providing stereoscopic images of the Earth.
3The terms ‘interior’ and ‘exterior’ are used in the DEM generation research community. In other research communities, such as computer vision, they are called ‘intrinsic’ and ‘extrinsic’ parameters, as discussed in Chap. 2.
372 |
H. Wei and M. Bartels |
Pertl [119] in DEM generation. Gruen developed a powerful model in which information from both image grey-level and first-order derivatives were incorporated for image matching [57]. With this model, adaptive least square correlation was performed to select the best match based on the fusion of point positioning with grey level information. It was claimed that the adaptive least square correlation provided a high matching accuracy. In terms of feature-based matching, Förstner and Gulch identified a series of feature points such as road intersections or centers of circular features which could be incorporated in matching algorithms [46]. Otto and Chau developed a region-growing algorithm for matching of terrain images [117]. They declared that their algorithm was an extension of Gruen’s adaptive least squares correlation algorithm so that whole images can be automatically processed, instead of only selected patches. It was demonstrated that the developed algorithm was capable of producing high quality and dense range maps when the scene being viewed had significant texture and few discontinuities. Feature-based algorithms in satellite stereo image matching complement the situations in which the scene has a sparse texture and presents large discontinuities.
In the 1990s, the development of passive stereo imaging techniques in the field of Computer Vision made it possible to have more automated solutions for DEM generation from stereoscopic imagery. Techniques, such as the stereo matching algorithm with an adaptive window [87], the coarse-to-fine pyramidal area correlation stereo matching method [116], and the robust approach for matching using the epipolar constraint [183] were adapted by the remote sensing community. By using the main principles of passive stereo vision from the Faugeras’ book [41], Gabet et al. worked out a solution for automatic generation of high resolution urban zone digital elevation models [51]. The work made use of an image sequence acquired with different base to height (B/H) ratios, hence, several stereo pairs are jointly used for DEM generation in a fixed area. With combinations of multiple algorithms covering both area-based and feature-based approaches, a fixed window size was used for cross-correlation in image matching. The authors claimed that the developed approach was universal to both airborne and spaceborne stereoscopic imagery, although only airborne data was tested due to the scope of the research. Wang [165] proposed an interesting structural image matching algorithm, in which an image descriptor was used for matching, which included points, lines, and regions structured by pre-defined relationships. The author demonstrated that the algorithm could achieve higher automation in DEM generation. The demand of automation for DEM generation within commercial software can also be seen in Heipke’s review paper [69] and significant improvements had been made to aerial stereo images in the 1990s.
In the 21st century, researchers have continued their efforts on automated DEM generation from satellite images and developed methodologies aimed at improving DEM accuracy and the level of automation. More robust computer vision algorithms were developed for stereo image matching [106]. Commercial software, such as PCI Geomatics, Desktop Mapping System, ERDAS Imagine, ENVI software, amongst others appeared on the market including algorithms for automated DEM genera-
9 3D Digital Elevation Model Generation |
373 |
tion from stereoscopic imagery. Hirano et al. [73] examined ASTER4 stereo image data for DEM generation. ASTER provides along-track stereo image data in nearinfrared with a 15 m horizontal resolution at a B/H ratio of 0.6. Computed elevations from commercial software were compared with results from topographic map and USGS5 DEMs at a few testing sites and conclusions were made that DEMs generated from ASTER could expect ±7 m to ±15 m elevation accuracy and up to 99 % correlation success rate with images of good quality and adequate ground control. Lee, et al. [98] argued that DEM generation from satellite images was timeconsuming and error-prone. This was due to the fact that most DEM generation software used for processing satellite images was originally developed for aerial photos taken by perspective cameras, while satellite images may be formed by linear pushbroom cameras. Hence, image matching and geometric modeling implemented in the software for aerial photos had to be modified for satellite imaging applications. In their paper, linear pushbroom cameras were modeled with the geometric properties in designing the matching strategy optimized in three aspects: conjugate search method, correlation patch design, and match sequence determination. It was claimed that the developed approach was universal for linear pushbroom images with various correlation algorithms and sensor models. DEM generation from SPOT-5 stereoscopic imagery was investigated in [21, 92, 125, 150]. SPOT-5 is equipped with two High Resolution Stereoscopic (HRS) cameras that are tilted ±20◦ to acquire stereo pairs of 120 km swath, along the track of the satellite with a B/H ratio of 0.8, and the nadir looking-HRG (high resolution geometric) panchromatic camera providing additional images. HRS has a horizontal resolution of 10 m and HRG has a resolution of 5 m. A summary of the SPOT-5 payload and mission characteristics is given in [21]. In the above work, bundle adjustments were conducted to correct cameras’ interior and exterior parameters in the geometric model. The best result was claimed in [150] with the vertical accuracy of 2.2 m for a smooth bare surface. In general, the DEM generated from SPOT-5 stereo images could achieve 5–10 m elevation accuracy with accurate and sufficient GCPs. DEMs generated from the IKONOS triplet (forward, nadir and backward) of stereoscopic imagery were investigated by Zhang and Gruen [182]. In their work, a multi-image matching approach was developed by using a coarse-to-fine hierarchical solution with an effective fusion of several matching algorithms and automatic quality control. It was reported that the DSM achieved 2–3 m elevation accuracy in the test area. With this accuracy, it is possible to consider DTMs to be extracted from the DSMs.
In June 2009, Japan’s Ministry of Economy, Trade and Industry (METI) and NASA6 jointly announced the release of Global Digital Elevation Model (GDEM) by stereo-correlating about 1.3 million scenes from ASTER data [85], as shown in Fig. 9.3. It has been indicated in its validation summary report [9] that ASTER
4ASTER: Advanced Spaceborne Thermal Emission and Reflection Radiometer, an imaging instrument flying on Terra satellite launched in December 1999 as part of NASA’s Earth Observing System.
5USGS: United States Geological Survey.
6NASA: National Aeronautics and Space Administration.