Internet search has evolved from its early days. It has become smarter and more
natural, and people expect it to “just work.” But, anyone who has worked
behind-the-scenes with a search engine knows exactly how hard it is to get the
“right” results to show up at the right time.
Not to mention, what happens when the trends change, when your users’ favorite
weirdly-shaped dinosaur isn’t a T-Rex anymore? Spending countless hours tuning
the boosts before your user can find their favorite two-legged tiny-armed
dinosaur on the front page isn’t fun. What is cool is using Learning to Rank to
automate the process! In this talk, you will learn how Learning to Rank works
and how you can use it in Apache Solr — all from the Bloomberg team that built
and implemented it in the first place.
Description
Internet search has long evolved from days when you had to string up your query in just the right way to get the results you were looking for. Search has to be smart and natural, and people expect it to “just work” and read what’s on their minds.
On the other hand, anyone who has worked behind-the-scenes with a search engine knows exactly how hard it is to get the right result to show up at the right time. Countless hours are spent tuning the boosts before your user can find his favorite two-legged tiny-armed dinosaur on the front page.
When your data is constantly evolving, updating, it’s only realistic that so do your search engines. Search teams thus are on a constant pursuit to refine and improve the ranking and relevance of their search results. But, working smart is not the same as working hard. There are many techniques we can employ, that can help us dynamically improve and automate this process. One such technique is Learning to Rank.
Learning to Rank was initially proposed in academia around 20 years ago and almost all commercial web search-engines utilize it in some form or other. At Bloomberg, we decided that it was time for an open source search-engine to support Learning to Rank, so we spent more than a year designing and implementing it. The result of our efforts has been accepted by the Solr community and our Learning to Rank plugin is now available in Apache Solr.
This talk will be presented by two engineers working in the News Search Engineering team at Bloomberg - Sambhav Kothari and Diego Ceccarelli. The talk will serve as an introduction to the LTR(Learning-to-Rank) module in Solr. No prior knowledge about Learning to Rank is needed, but attendees will be expected to know the basics of Python, Solr, and machine learning techniques. We will be going step-by-step through the process of shipping a machine-learned ranking model in Solr, including:
- how you can engineer features and build a training data-set as per your needs
- how you can train ranking models using popular Python ML(machine learning) libraries like scikit-learn
- how you can use the above-learned ranking-models in Solr
Get ready for an interactive session where we learn to rank!
