Visual Search. New Approaches in Content-Based Image Retrieval

Abstract

Visual Search +

New Approaches in Content-Based Image Retrieval

With the exponential growth in both creating and sharing digital photographs, the topic of visual search or Content-Based Image Retrieval (CBIR) has become increasingly important across all disciplines. As archiving digital content with manually typed keywords becomes unsustainable, CBIR will aid the automation of formal descriptions for imagery and quantify traits that are difficult to describe with natural language. Faces, for example, are rarely described and tagged with more detail than a name, date, and location. Missing from this knowledge are qualities of expression, composition, lighting and pose, key factors to filtering one image from that of a thousand.

Focusing on portrait imagery, Visual Search+ is a collaborative venture of the Graphics and Vision Research Group at the University of Basel and the Visual Communication Institute of FHNW HGK Basel. The goal of this project is to increase the synthesis between developing CBIR technology and the needs of an end user.

An existing analogy to this issue can be found in the color picker element, which allows the everyday computer user to select a color without being bothered by the mathematical representation required by the computer. From a design perspective, this will require creating a graphical user interface (GUI) consisting of new tools for searching through abstract sets of extracted information. Exploring the potential and usability of both 2D and 3D widgets, visual traits within the image will be retrieved through an abstracted visual representation of those very same qualities. From the technology side, a state of the art 3D Morphable Face Model used to analyze attributes within 2D photographs such as eye gaze, head tilt, age, sex, expression and lighting among others, will be used to generate an unlimited set of training imagery. This imagery can then be used to teach Fast Image Detectors how to describe and quantify visual attributes with concise speed and accuracy.

Through the success of this project, we hope to set a further citable example for the growing design-led research community and improve the integration of a users needs with technology as it is developed.