Building Detection Using Local Features and DSM

Data
1 Abdullah H. Özcan, 2 Cem Ünsalan, 3 Peter Reinartz
1
Tubitak BİLGEM, Turkey
2
Yeditepe University, Turkey
3
DLR, Germany
(e-mail: [email protected])

Abstract—Detecting and locating buildings in satellite images information on DSM data. After detecting buildings, they used
has various application areas. Unfortunately, manually detecting surface normals to extract rooftop geometries. Sirmacek et
buildings is hard and very time consuming. Therefore, in the al. [9] used DSM for detecting building ground floor shapes
literature several methods are proposed to automatically detect
buildings. These methods can be divided into two main groups. In using an active shape detection approach. Then, they used
the first group, researchers used panchromatic or multispectral derivative filters to extract roof ridge lines. This leads to
information to detect buildings. In the second group, researchers 3D building reconstruction. Galvanin and Poz [6] proposed a
used DSM data to detect buildings. In this study, we propose two method for rooftop extraction. They used DSM data to detect
novel methods to detect buildings by combining the panchromatic above ground objects. Therefore, they segmented DSM with
and DSM data. The first method uses corner points extracted by
Harris corner detection method. These corner points are used a recursive splitting technique and region merging process.
jointly with DSM data. Using a kernel based density estimation Awrangjeb et al. [1] proposed a method to separate buildings
method, possible building locations are detected. In the second and trees using DSM. They used height and width information
method, shadow of buildings are used in a similar way. We tested from DSM with a ground mask. They used the image entropy
both methods on WorldView-2 images and DSM data generated and color information to remove trees. For change detection of
from them.
Index Terms—Building detection; DSM; Kernel density esti-
buildings we worked in previous studies on fusion approaches
mation of using DSM and multispectral data [12] and [3], while this
study is concentrating on single building extraction.
I. I NTRODUCTION Most previous works assume that thresholding DSM pro-
vides sufficient information about the building shape. Un-
Detecting and locating buildings in satellite images has fortunately, using local thresholding for DSM data fails at
various application areas. Unfortunately, manually detecting industrial areas where big buildings are closely located. In
buildings is hard and very time consuming. Therefore, several these areas, the window size for local thresholding needs to
methods are proposed to automatically detect buildings in be very large. Also due to automatic DSM generation, some
the literature. These methods can be divided into two main unwanted outliers may occur. These are caused by matching
groups. In the first group, researchers used panchromatic or errors, temporal changes or applied interpolation techniques.
multispectral information to detect buildings. In the second These also affect the building detection process in the negative
group, researchers used DSM data to detect buildings. manner. As an example, closely located buildings in city areas
Automatic building detection using panchromatic or mul- cause uncertainty on building edges. The main reason for this
tispectral images has been studied extensively. Here, it is is the applied interpolation technique which causes a loss of
assumed that DSM data is not available. In our previous study, sharpness. Buildings also do not have clear rooftop contours
we analyzed and summarized the literature on panchromatic because of the mentioned reasons. Sometimes a group of trees
and multispectral image based building detection methods may look like a building and there is no easy way to separate
[11]. Therefore, we will not review them here. them. However, the height information in DSM is still very
There are several works using DSM for building detection valuable.
and 3D reconstruction. Most of these studies use the height The proposed methods in this study are an extension of
information to remove non-building structures. Then, they our previous works [11], [10]. There, we benefit from local
focus on the building shape and rooftop contours. Tournaire et features and shadow information extracted from panchromatic
al. [13] used point processes on digital elevation models. They images. In local feature based method, we generate a vector
calculated an energy function for fitting rectangles on buildings for each local feature. Each vector has a local position,
based on the adequacy of objects and prior knowledge to orientation, and weight. Based on their formation, for local
extract footprint of buildings. Ortner et al. [8] used two features generated from bright building corners, the vectors are
interacting spatial point processes on DEM to fit rectangular towards the building center. On the other hand, the generated
shapes on building segments. Brunn and Weidner [2] separated vectors for a dark building are away from building centers.
buildings and vegetation areas using height and geometric Both can lead to building detection. To do so, each local

978-1-4673-6396-9/13/$31.00 ©2013 IEEE 139

feature (vector) is used as an observation in non-parametric formation. Here, we used symmetric Gaussian probability
kernel density estimation. Here, the kernel is taken as a density function for kernel density estimation as follows.
symmetric Gaussian function with a variable variance. Modes
of the estimated density indicate possible building locations in ( )
1 (x − xm )2 + (y − ym )2
the satellite image. In shadow based building extraction, the p(x, y) = √ exp − (3)
multispectral information serves as a cue for possible nearby 2πσ 2σ 2
building locations. In our previous work, σ was a variable for a variable kernel
Our kernel density estimation based building detection function [11]. Here, we choose σ = w/3 which gives enough
method works fairly well on most of the satellite images. space for kernel formation. This method is applied to all corner
However, it may not work on images with complex building points, (xm (i), ym (i)), extracted. Summation of all kernel
structures. Besides, the closely located buildings also pose a densities gives the final density map as follows.
possible problem in detection. These problems occur mainly
from the interference from nearby trees, road edges, and
∑
N
1
similar objects. In this study, we propose two novel methods pmap (x, y) = √ exp(A(x, y)) (4)
to overcome the mentioned problems. In both methods, we i=1
2πσ
effectively fuse the panchromatic information and DSM data.
In DSM data extraction, we benefit from the stereo image where
pairs obtained from the WorldView-2 sensor. Detailed recon-
struction of objects from these images is possible with stereo (x − xm (i))2 + (y − ym (i))2
A(x, y) = − (5)
matching algorithms. Specifically, DSM used in this study is 2σ 2
obtained by the semiglobal matching algorithm [4]. In this In Eqn. 4, N is the number of corner points. As in our
method, the similarity value for two images is computed in a previous method, the formed density function can be used to
pixelwise manner. The matching cost between the image pixels detect building centers in the region. The final density map
are computed as an energy minimization problem. Finally, is multimodal since the number of buildings in the image
DSM is obtained by reprojecting the disparity image with is unknown. Local maxima of pmap (x, y) indicate possible
a desired grid spacing and cartographic projection. In the building locations. To obtain a more reliable result, we also
following section, we start with explaining our local feature perform a post-processing for eliminating some modes that are
based building extraction method. Then, we focus on the below a minimum probability value.
shadow based building extraction.
II. C ORNER P OINTS AND DSM BASED B UILDING B. Building Detection Examples
D ETECTION We summarized the local feature and DSM based build-
In this section, we explain our local feature and DSM based ing detection method in Fig. 1. In Fig. 1(a), corners on
method in detail. We start with explaining the method. Then, the panchromatic image detected by the Harris detector are
we provide a sample result on the application of our method. labeled. Each corner point is assumed to be on DSM in a w×w
subwindow as shown in Fig. 1(b). The voting directions for
A. Corner Points and DSM Data
the corner points are given in Fig. 1(c). Finally, the obtained
In this study, we extract local feature points using Harris kernel density map is given in Fig. 1(d).
corner detector [7]. Throughout the paper, we will call our We next take a sample test image given in Fig. 2. As can
local feature points as corner points. After extracting corner be seen in this figure, buildings are closely located. Therefore,
locations from panchromatic image, each corner location, DSM data or panchromatic image is not sufficient alone for
(xc , yc ) is assumed to be in the center of a w × w window, detecting buildings here. Moreover, buildings in this figure
Is , on DSM. The highest point in this window, (xm , ym ), is have different shaped and colored rooftops. In Fig. 2, we also
obtained based on its elevation data as follows. provided the voting directions obtained by the corner and DSM
data. As can be seen here, they indicate possible building
(xm , ym ) = argmax(Is (x, y)) (1) centers.
where, We provide the kernel density map obtained from the first
test image in Fig. 3(a). We also provide the detected buildings
xc − w/2 ≤ x ≤ xc + w/2 from this map in Fig. 3(b). As can be seen in this figure,
(2) buildings are detected by our method.
yc − w/2 ≤ y ≤ yc + w/2
We pick the second test image in Fig. 4. As can be seen
Then, (xm , ym ) is taken as a kernel formation location. here, buildings are again closely spaced. Moreover, they have
Before forming a kernel function, pre-elimination is done different color and size. We first provide the kernel density
using DSM data. If Is (xm , ym ) − Is (xc , yc ) < th, the height map in Fig. 4(a). Finally, we provide the detected buildings
difference between the maximum height point and the corner in Fig. 4(b). As in the first test image, buildings are detected
point on Is , then (xm , ym ) is not considered for kernel fairly well in the second test image.

140
(a) Corner points on panchromatic (b) Generated DSM. (a) The kernel density map. (b) Detected buildings.
image.
Fig. 4. Building detection results for the second test image.

III. S HADOW P OINTS AND DSM BASED B UILDING

D ETECTION
In the second method, we follow a similar strategy using
the shadow information extracted from panchromatic images.
In this method instead of using corner points, shadow points
are taken as local features. Then, they are jointly used with
(c) Voting directions. (d) Kernel density map. DSM data for kernel density estimation.

Fig. 1. Local feature point method summary. A. Extracting Shadow Points

There are several methods on shadow detection on satellite
imagery. These include thresholding, classification, region
growing, segmentation, and 3D modelling. Some of the meth-
ods use multispectral data. Some use only gray scale images.
In this work, we used thresholding on panchromatic images.
Classical problem for this method is the proper selection of
the threshold value to best separate shadow and non-shadow
areas.
Bimodal histogram splitting method gives a very good
solution for threshold selection. In Fig. 5 we provide the
histogram of the first test image where bimodal behaviour is
seen. Dare [5] proposed that taking the mean of the two peaks
gives accurate threshold level for shadow extraction for such
histograms.

Fig. 2. Voting locations for the first test image.

Fig. 5. Threshold selection for shadow extraction.

After thresholding, the image shadow and non-shadow areas

(a) The kernel density map. (b) Detected buildings. are labeled. Finally, we remove the small regions which have
Fig. 3. Building detection results for the first test image. less than 20 pixels in size. In Fig. 6, we provide the kernel
directions extracted from the extracted shadow pixels.

141
 (a) Shadow points on the panchro- (b) Generated DSM.
matic image.

Fig. 6. Kernel directions using the shadow pixels extracted for the first test
image.

B. Building Detection
To detect buildings using shadow pixels, we apply the fol- (c) Voting directions. (d) Kernel density map.
lowing steps. First, we select equally spaced points on shadow Fig. 7. Shadow points method summary.
areas. This step is shown in Fig. 7(a). As in the first method,
shadow points are assumed to be in the center of an w × w
window on DSM. This is shown in Fig. 7(b). As in Eqn. 1, the
maximum height index is obtained. Now, every shadow point
has a voting direction showing the possible building center as
shown in Fig. 7(c). Using Eqn. 3, a new symmetric Gaussian
kernel is formed. Repeating this process for every shadow
point and summing the kernel density gives the final kernel
density map. The final kernel density for our example is given
in Fig. 7(d). The final density map is multimodal and local
maxima of pmap (x, y) give the possible building locations.
Again, we do a post-processing for eliminating some modes
that are below a minimum probability.
(a) The kernel map. (b) Detected buildings.
We provide the building detection results using shadow
information on the same dataset. For the first test image, the Fig. 8. Shadow points method, building detection results for the first test
final kernel map is given in Fig. 8(a). Based on these, the image.
detected buildings are given in Fig. 8(b). We provide the kernel
map and the detected buildings for the second test image in
Fig. 9.

IV. T EST R ESULTS

In this section, we test our building detection methods on
larger WorldView-2 images for quantitative results. DSM data
is generated from these images. The test images are acquired
from residential areas. they have various building types. Our
test images contain 94 buildings. These have different rooftop
shapes, colors, and heights. To note here, if a building is
detected more than once than we accept it as a true detection.
We provide the building detection results using corner and (a) The kernel map. (b) Detected buildings.
shadow points in Figs 10 and 11 respectively. Fig. 9. Shadow points method, building detection results for the second test
We provide the test results (over 93 buildings) for our image.
methods in Table I. As can be seen in this table, using corner

142
 tunately, the DTM extraction process gives false results at
closely located building groups or at industrial areas. In these,
buildings are large and they have big height differences. To
avoid these disadvantages, we didn’t use nDSM data in our
methods.
V. C ONCLUSION
In this study, we propose two novel methods for building
detection in satellite images. In both methods, we effectively
fuse the panchromatic information and DSM data. In our first
method, we perform this fusion using corner points and DSM.
In our second method, we fuse the shadow information and
DSM to detect the buildings. The initial results indicate the
Fig. 10. Building detection results from the residential area using corner effectiveness of our methods in building detection.
points.
ACKNOWLEDGMENT
This work is supported by TUBITAK under project no
110E302.
R EFERENCES
[1] M. Awrangjeb, C. Zhang, and C. S. Fraser, “An improved building detec-
tion technique for complex scenes,” in IEEE International Conference
on Multimedia and Expo Workshops, 2012.
[2] A. Brunn and U. Weidner, “Extracting buildings from digital surface
models,” in Int. Archives Photogrammetry, Remote Sensing and Spatial
Information Sciences, 1997.
[3] H. Chaabouni-Chouayakh, I. Rodes-Arnau, and P. Reinartz, “Towards
automatic 3-d change detection through multi-spectral and digital ele-
vation model information fusion,” International Journal of Image and
Data Fusion, In Press.
[4] P. dAngelo and P. Reinartz, “Semi-global matching results on the isprs
stereo matching benchmark,” in Int. Archives Photogrammetry, Remote
Fig. 11. Building detection results from the residential area using shadow Sensing and Spatial Information Sciences, vol. XXXVIII-4/W19, 2011.
points. [5] M. P. Dare, “Shadow analysis in high-resolution satellite imagery of
urban areas,” in Photogrammetric Engineering and Remote Sensing,
2005, pp. 169–177.
[6] E. A. S. Galvanin and A. P. D. Poz, “Extraction of building roof contours
points and DSM data together, our true detection (TD) and from lidar data using a markov random field based approach,” IEEE
Transactions on Geoscience and Remote Sensing, vol. 50, no. 3, pp.
false alarm (FA) rates are 90.3% and 12.9% respectively. When 981–987, 2012.
we use shadow points and DSM data, the true detection and [7] C. Harris and M. Stephens, “A combined corner and edge detector,” in
false alarm rates are as 86.0% and 9.6%. Although the true Proceedings of the Fourth Alvey Vision Conference, 1988, pp. 147–151.
[8] M. Ortner, X. Descombes, and J. Zerubia, “Point processes of segments
detection performance decreased in the shadow based method, and rectangles for building extraction from digital elevation models,” in
the false alarm rate also decreased. The main reason for false Proceedings of ICASSP 2006, vol. 2, 2006.
alarms is trees and road segments. Multispectral information [9] B. Sirmacek, H. Taubenböck, P. Reinartz, and M. Ehlers, “Performance
evaluation for 3-d city model generation of six different dsms from air
may be used to eliminate these in future studies. Also if there and spaceborne sensors,” IEEE Journal of Selected Topics in Applied
are two adjacent buildings with very big height differences, Earth Observations and Remote Sensing, vol. 5, no. 1, pp. 59–70, 2012.
then most of the kernel vote directions will be towards the [10] B. Sirmacek and C. Ünsalan, “Building detection from aerial images
using invariant color features and shadow information,” in Proceedings
higher building. This may cause miss detections. Besides, the of ISCIS’08, 2008.
obtained results using both methods are very promising. [11] ——, “A probabilistic framework to detect buildings in aerial and
satellite images,” IEEE Transactions on Geoscience and Remote Sensing,
TABLE I vol. 49, no. 1, pp. 211–221, 2011.
B UILDING DETECTION PERFORMANCES FOR THE PROPOSED METHODS . [12] J. Tian, S. Cui, and P. Reinartz, “Building change detection based on
satellite stereo imagery and digital surface models,” IEEE Transactions
Method TD FA TD (%) FA (%) on Geoscience and Remote Sensing, In Press.
Corner points 84 12 90.3 12.9 [13] O. Tournaire, M. Bredif, D. Boldo, and M. Durupt, “An efficient
Shadow points 80 9 86.0 9.6 stochastic approach for building footprint extraction from digital eleva-
tion models,” ISPRS Journal of Photogrammetry and Remote Sensing,
vol. 65, no. 4, pp. 317–327, 2010.
To note here, although we used the height information for
kernel formation, we didn’t use normalized DSM (nDSM) data
for this purpose. nDSM is the difference of DSM and Digital
Terrain Model (DTM) of the interested region. Therefore, the
ground height will be referenced to zero at nDSM. Unfor-

143