Consider this scenario. An end consumer on visiting a online shopping mart, wants to see a particular product. Probably, also check out some discussions around the same. This product may be mentioned in a discussion on another product. Wouldn’t the consumer like to check it out too?
Our application is capable of storing all this information. But the big question is, is it capable of displaying it back, in a relevant and timely way?
It’s all about showing the end user, what he wants to see. Be it a business user for an internal application or the end consumer on the web or a phone, they all ask the same question –
Would the app show me the relevant and related information?
Traditionally, we would search the database by giving in a select query. This query would have some wild card comparisons against some fields. And well, hope that the entered text would match!
What did it involve
- The data is created or stored in the DB, this could be transactional data and/or business setup data.
- The data is (hopefully) indexed, but only for exact match
- Submit a query to the DB, if a match is found, it would return results. The match in traditional sql queries is very restrictive. It could either be a partial match or an exact match.
What’s different now
The search requirements have changed dramatically. With Google setting the standards, any application providing a search capability, cannot suffice with doing an exact match with a keyword, and hope to get away with it. The users expect more. The search query can be keyword or a question in a sentence form(natural language question). The expected behavior is that, even a single nonsensical word should retrieve some relevant data.
It is not difficult to enhance your current applications usability and depth by providing a search paradigm. This could be the X-factor of your application. Search engines are already available off the shelf. Integrate them with your application, and this small change could make a big difference to the end consumer’s user experience.
I am big Lucene fan, so would suggest, solr or elasticsearch search engines. Let the application and DBs do what they do best, store data, and retrieve them on the index already defined in the exacting way they are good at. But any other search requirement, could be done by our specialized guy – the search engine.
The search engine
Your application’s data needs to be fed to the search engine. This engine then indexes the data in its own way. It will analyze the data according to the requirements defined by you. You would need to tell it which data (fields in a model(s)) needs to be indexed.
Should the data be broken up in tokens or store as it is. Set up the tokenizers to be used – snowball, ngram, standard…and there is big list.(Phonetic search and spatial search capability formed the core of our last project.)
After the indexing is over, we can query it. The engine searches its indices and returns back with, the identifier to the DB record, along with how relevant was the match with the keyword being searched for.
In the query you could ask it to facet (group) it, highlight the matched word, or sort as per your needs.
Voila! Out comes the search results. It is astounding how the search engine can transform your database into a knowledge base.
Renu Yarday is a Senior Consultant in Ruby On Rails at Compassites. Her areas of expertise include Ruby On Rails, Java, Oracle and MongoDB. Prior to working at Compassites, She has worked at IFlex and Strata Retail and Technology Services.