Introduction
This is May from the GLB Division Lakehouse Department.
Based on reports from members participating in the local Data + AI SUMMIT 2023 (DAIS), Share "The Future of Data Management and Governance" about the future of data management and governance. This session will introduce a new technology called Lakehouse Federation, which aims to enable access to external data and improve performance and ecosystem. The target audience is technologists, data analysts, and data scientists interested in data management and governance. Let's take a look at the contents!
What is the Lakehouse Federation?
Lakehouse Federation is a breakthrough technology that paves the way for the future of data management and governance. This technology facilitates access to external data and streamlines data acquisition and analysis. It also improves data security, line-up and insight management, and enables data source consolidation.
Features that facilitate collaboration with external data sources
Lakehouse Federation offers the latest concepts, features and services, including:
- Data Integration: You can integrate various data sources and manage them centrally.
- Security: Easier access control and auditing of data.
- Scalability: It can efficiently process large amounts of data.
These capabilities make the future of data management and governance brighter.
Query planner innovation: 95% pushdown enabled
The query planner developed by Databricks is groundbreaking in that it can pull data from various systems such as Postgres and MySQL and push down almost 95% of queries. This greatly improves the efficiency of data analysis and data processing, and enables faster data access.
Summary
The talk covered the future of data management and governance and introduced the Lakehouse Federation. It was also shown that innovations in the query planner greatly improve the efficiency of data access. These technologies unlock the future of data management and governance, enabling more efficient data utilization. We will continue to pay attention to the evolution of such technology in the future.
Conclusion
This content based on reports from members on site participating in DAIS sessions. During the DAIS period, articles related to the sessions will be posted on the special site below, so please take a look.
Translated by Johann
D
Thank you for your continued support!
