Building Recognition by Consistent Line Clusters

1 Introduction

With the rapid growth of storage devices, digital photographs, and internet access, content-based image retrieval (CBIR) has become increasingly popular in recent years. Systems that can automatically analyze, categorize, and search image databases are under high demand.

Most current CBIR systems perform retrieval based primarily on low-level image features, such as color, texture, and shape. The concern with this approach is whether the low-level image features are powerful enough to capture the high-level concepts. As shown by the example in Figure 1, a user can query the image database using a sample image that contains a building and will expect to get other building images. Using the color histogram image distance measure, the user gets some very strange results including a pathway image and a national park image.

Figure 1: Sample results from a query to an image retrieval system using the color histogram feature.

The aim of our research is to develop an object-level CBIR system. In order to recognize common object classes, we must analyze, organize, and structure image primitives to create high-level object-oriented features that can be used to abstract the object-level concepts. We propose such a high-level image feature called consistent line clusters, which is a set of lines that are homogeneous in terms of their characteristics. We have applied this feature to a building recognition task and obtained satisfactory results.

2 The Proposed Algorithm

Lines are very important for object recognition. There are many algorithms to extract lines, but none of them address the questions of how to use the extracted lines to form more advanced features that can be used to recognize objects. Our solution is to group the lines into consistent line clusters and to use intra-cluster and inter-cluster relationships to recognize complex objects. We have applied this approach to building recognition and achieved promising results.

The detected lines from an image can be separated into different line clusters according to their features. There are several different line features that can be used to fulfill this task.

· Color: The lines in a line cluster have the same color features; such a line cluster is called a color-consistent line cluster;

· Orientation: The lines in a line cluster have the same orientation; in other words, they are parallel to each other. Such a line cluster is called an orientation-consistent line cluster;

· Position: The lines in a line cluster are in close proximity to each other; such a line cluster is called a spatially-consistent line cluster.

These features can be used individually or in different combinations. How these features are used to obtain consistent line clusters depends on the target objects.

In our algorithm, the color feature is used first to obtain color-consistent line clusters, then the orientation feature is used to classify each color-consistent line cluster to orientation-consistent line clusters, and at last the position features are employed to classify each color- and orientation-consistent line cluster to spatially-consistent line clusters. The whole procedure is illustrated in Figure 2.

Figure 2: The procedure for constructing consistent line clusters.

After the consistent line clusters are found, they must be used to recognize and locate the objects of interest. The properties of buildings and their distance from the camera lead to two different representations. Some buildings contain very rich structures, such as multiple windows and doors that are well-preserved in their images. Some buildings are very simple in structure or are far enough away that the detailed structure is lost. To handle both cases, two different criteria are used. The first criterion is based on the interrelationships among the line clusters; it is good at finding buildings with detailed structures. The second criterion is based on the intra-relationship of the line clusters, and it detects buildings with simple structures. Building recognition results using these two criteria are shown in Figure 3.

Figure 3: Building recognition results.