Visual Desktop Searching

Often when attempting to search for an application with which you have had little use it is often hard to remember specific details to create a search term from. This can lead to waisted time and often frustration as you can generally recall certain details regarding the application but not specifically its name. This is not helped by the general trend of naming applications with obscure often loosely connected names to the tasks they performed. The purpose of this search technique is to allow the incorporation of visual elements or descriptive words of the applications purpose to be used. For example if you have downloaded the application Spotify which allows you to listen to free music interrupted with adverts you may remember a few details but not what its called. Therefore in its simplest form the user can search using the keywords 'music', 'free', 'adverts' etc and be presented with the the application or those which are closely related. This can also include visual elements which are often more likely to be able to be recalled such at the fact that Spotify has a green logo that resembles a speaker. By attaching more keywords to the application more vague searching can be used to determine the required application. The difficulty with this project is deciding on the metaData that should be associated with each of the applications. The proposed and partially implemented solution is to use the ESP game structure where users attempt to describe what they associate with an application. The system as described later presents the user with the logo of the application with the name and a single screen shot of it in use. Both users must then enter suitable keywords which once matched are entered into a global database. These can then be combined and used to search for the specific application by future users within their desktop environment directly within spotlight.

The aim of this project is to improve desktop search techniques by combining visual elements to describe the desired content. The hope is to allow searching using less focused terms ensuring the desired application can be found by combining multiple keywords describing the purpose or visual associations with the item. The project also aims to construct a database of associations that users have with applications and in turn what is perceived to be the key defining elements of these.

The system is split with a central website supporting the ESP based game for metaData collection and a client application which updates each individual system with the associated tags. The keywords entered through the website are stored within a SQL database which is then accessed via a PHP API that returns data in XML format which can then be applied to each of the clients applications. When the client is executed it gathers all application names and versions and compares this to the central database. If the application hasn't been detected previously it is included within the ESP game allowing for tags to be created. If a record is available the various tags are added to the application within the apple namespace allowing spotlight to successfully identify and search the application. By integrating with the existing spotlight system users can combine their current search techniques with the new system ensuring that applications can be effectively identified.

The project hopes to be expanded as greater number of users submit details relating to their installed applications. As the database increases the hope is to allow tags to be added both through the ESP based game but also within the application allowing desktop users to specify their own custom tags to combine with the global database.

More Info

ESP Games, Search Test Page