Alright, you built an application to show off your impressive skills at data gathering and processing. Now you got data on ElasticSearch that you want to show, but your users may want to see too many data points and will shatter your DB performance. In this talk, we will present a project to formally measure the cost of queries before actually running them, and your app can decide, given a cost value, whether to launch a query or not.
ESCOVA leverages the ElasticSearch query parser to generate the parse tree of an ES query, and we use tree-based cost analysis in order to determine the cost of queries. Moreover, users may restrict their queries to only those of forms that have been previously whitelisted in order to avoid blocking the database. ESCOVA is available both as a plugin for an Elasticsearch as well as a library (that can, e.g., be deployed independently as a microservice).
The talk will describe: - How you can currently profile queries in Elasticsearch - A discussion about the pros and cons of static analysis/symbolic computation v. profiling - The inner structure of the project - Lessons learned on how to build an Elasticsearch plugin - Future work, questions and feedback
No specific prior knowledge required; topics will be presented before diving into them.
Speakers: Santiago Saavedra