Commoditizing Data Analytics in the Cloud

Introduction

The goal of this project is to commoditize large-scale data analytics in the cloud, which means (a) to reduce query processing cost to close to what the available hardware is theoretically capable of and (b) to prevent vendor lock-in by making it easy to move data between different clouds and systems. To achieve this, we will design and develop an open and cost-efficient analytical database system for the cloud called CODAC.

Components of CODAC

The CODAC system consists of three main components:

1. Intelligent Control Component
   This component automatically and transparently selects and manages the cheapest hardware instances for the given workload and makes migration to other (e.g., European) cloud vendors possible.

2. Highly-efficient Query Processing Engine
   This engine is capable of fully exploiting modern cloud hardware, ensuring efficient query processing.

3. Data Lake Storage Abstraction
   Based on open data formats, this abstraction enables cheap storage as well as modularity and interoperability across different data systems.

Conclusion

The resulting system therefore has the potential for making large-scale data analytics both cheaper and easier, boosting data-driven instruction and science.