If there is one thing you want when running a Commerce site it is a good search. People need to be able to easily find the products they need, and perhaps even up sell some things to them within the search. You also want to make sure that the search has a minimal amount of impact on the rest of the site. If you have tens of thousands of products you don't really want to have heavy searches going on across lots of attributes. Unfortunately if you use the Commerce search functionality it is heavy and it is quite basic, because at the end of the day its just SQL matches over the top of your data.
What you instead what is a way to rate the products and the matches found. So perhaps looking for "Harry" has a better match in the title of the product if its a book rather than the author or description. or maybe you want to match on common endings so I search for "roast" on a cookery site I may want roasting, roasted and roaster to come back as matches. Now you could invest in a product like Omnifind, but that does incur a cost and although it can do a lot for you an alternative free option may be to use Lucene.
This is an Open Source offering that can help move your search away from the database, and provide some really great functionality. Instead of looking at the data in the database Lucene builds indexes which you define across the data and then uses those indexes to search against. These can then either be held in memory or on disk depending on the resources available to your environment, and suddenly any searches are removed from being a Commerce overhead. The product provides all kinds of functionality for doing weighting of results and uses a stemming algorithm to provide words matches for you. The results returned from Lucene are then used to get the products data so things such as stock status can be shown, so we are mixing the two technologies together.
You can see this in action at both the TBP and Jeroboams sites, where searches are driven from Lucene. Try some different terms and you can see it is also possible to highlight your results. You could then take this a stage further and also have a weighting against your products if you wanted to move them up or down in the search results.
