Data systems that learn to be better

MIT News  August 10, 2020 Filtering data based on predicates is one of the most fundamental operations for any modern data warehouse. However, schemes are hard to tune and their performance is inconsistent. Automatically optimizing an index for a particular dataset and workload suffers in the presence of correlated data and skewed query workloads. Researchers at MIT have developed Tsunami, which addresses these limitations to achieve up to 6×faster query performance and up to 8× smaller index size than existing learned multi-dimensional indexes, in addition to up to 11× faster query performance and 170×smaller index size than optimally-tuned traditional indexes…read […]