Introduction and Agenda Overview
This session is titled "Building a Data and GenAI Ecosystem, Transitioning to Databricks". In this presentation, we will delve into the strategies adopted by Gerdau to construct a robust data and AI framework, which is essential for success in today's transformative tech environment.
Rodrigo Silveira, the Data & AI Chief Officer at Gerdau, along with Felipe Montalini, the head of Data and Architectural Engineering, will share valuable insights about their personal journeys as data analysts and the progressive steps Gerdau has taken to achieve the latest innovation, Data Lake 3.0. They will outline both past initiatives and future perspectives of their data endeavors.
The agenda for the day will start with a retrospect of Gerdau’s data analytics evolution, followed by an in-depth exploration of the latest initiative, the Data Lake 3.0 project, and the strategic plans moving forward in the field of data and AI.
This session aims to enhance businesses' ability to effectively harness data and artificial intelligence by providing a roadmap of lessons drawn from Gerdau's experience.
- Introduction and Agenda Overview
- Gerdau’s Journey in Data and AI Innovation
- Implementing Digital Twins at Gerdau and Its Impact
- About the special site during DAIS
Gerdau’s Journey in Data and AI Innovation
Gerdau’s Data Initiatives and Journey
From the onset, Gerdau has been rapidly adapting to the evolving landscape of data and AI technologies. In 2018, they began adopting Data Lake 3.0 and the business case for digital twins. However, their efforts did not stop there. They also released their first data product focused on supply chain optimization, named "Smart Decision."
By 2020, the need for further transformation became apparent. The platform underwent additional governance enhancements, introducing the concept of "Data Service," laying the foundation for a self-service data platform. In 2022, feeling a renewed need for change, Gerdau launched the "Data Lake 2.0" project, achieving a robust yet user-friendly platform. Felipe Montanini will provide details in the upcoming slides.
As we look at the actual flow of data analytics, we see Gerdau's operational modules and a process called "Mandala," providing an overview of how they have managed data handling through their journey.
Transition Challenges and Governance
The transition to Databricks posed several challenges while evaluating the existing architecture of Data Lake 2.0. The primary issues were the difficulty of data integration and the high costs related to maintaining customized in-house solutions, which required extensive management resources and significantly increased the Total Cost of Ownership (TCO).
Despite the inefficiencies of existing solutions, their complexity was evident; managing them was as complex as making coffee with lab equipment! Moreover, the steep learning curve for new partners and employees required about 3 to 4 months in the onboarding process to grasp the basics of Data Lake 2.0.
Facing these barriers, it became clear that a strategic reassessment was necessary, leading Gerdau toward adopting Databricks. This platform promises not only improved efficiency and scalability but also enhanced integration of data processing. Consequently, data visibility and accessibility significantly increased, fostering data-driven decision-making across the organization.
This transition also facilitated onboarding new technology, streamlining processes to reduce the time for new employees and partners to become productive.
This section highlighted the challenges Gerdau faced during the transition and the governance strategies implemented to address them. The shift to Databricks demonstrated that well-thought-out planning and execution could overcome technical barriers and enhance operational efficiency.
Cost Efficiency and Performance Enhancements
One of the key highlights of the "Creating a Data and GenAI Ecosystem Migrating to Databricks" session was the significant reduction in data processing costs and a successful scalability strategy across multiple providers, with Databricks being a central component of the data ecosystem, involving top cloud providers like AWS and Azure.
While specific cost-saving figures were not disclosed, the savings on data processing were described as "very significant." The tools provided by Databricks—such as Catalog, Overwatch, Autoserve, Delta Lake tables, and Warehousing—are being fully utilized by Gerdau. These tools have significantly optimized Gerdau’s data architecture and enhanced operational efficiency.
The company’s philosophy focuses on "continuous improvement," stressing the importance of striving for better rather than settling for "good enough." This principle is one reason Gerdau maintains a leading position in the competitive steel manufacturing industry.
As demonstrated in this session, strategic technology choices and solid partnerships with cloud providers are crucial for enhancing business efficiency and scalability. These insights are invaluable for other companies aiming to optimize their data and AI ecosystems.
Implementing Digital Twins at Gerdau and Its Impact
What is a Digital Twin?
A "Digital Twin" is a digital model that replicates a real-life device, system, or process. This innovative technology enables real-time data analysis and simulation, guiding operational optimization and preventive maintenance strategies, thereby improving overall business efficiency.
Implementation Example: The Delake 3.0 Project
Within Gerdau, the Delake 3.0 project serves as an excellent example of applying digital twin technology in the industrial sector. By creating a digital replica of the entire steel manufacturing process, Gerdau possesses the capability to simulate various operational scenarios and maintenance procedures. This ability not only enhances operational efficiency but also reduces downtime and smoothens the production cycle.
About the special site during DAIS
This year, we have prepared a special site to report on the session contents and the situation from the DAIS site! We plan to update the blog every day during DAIS, so please take a look.
