| Abstract |
Due to the ubiquity of GPS-equipped cameras, captured photos are automatically tagged with camera location (referred to as geo-tagged images) at the acquisition time. Subsequently, web and mobile applications are emerging that provide search capabilities for images that are similar to a given query image and within a given geographical region (referred to as spatial-visual search). In this talk, I focus on two research challenges to enable spatial-visual search. First, I will discuss the need for new index structures to expedite image search based on both spatial and visual relevance. Hence, I present a generic index structure which supports organizing the images using their spatial and visual properties, and also supports an adaptive distribution of images based on different local partitioning either spatially or visually. Second, I discuss the challenge of inferring a location for the legacy images that are collected without spatial metadata. To utilize and support spatial-visual search for such images, there is a need for an accurate image localization technique for non-geo-tagged images. Moreover, even for geo-tagged images, the camera location of an image can be different from the location of the scene depicted in the image (referred to as scene location) rendering its native localization inaccurate. To address this problem, I present a framework for image scene localization using a CNN-based classification. I conclude by showing how spatial-visual search can be used for retrieving images to be analyzed for various smart-city applications in Los Angeles (e.g., street cleanliness and graffiti recognition). © 2019 IEEE. |
