APC 技術ブログ

株式会社エーピーコミュニケーションズの技術ブログです。

株式会社 エーピーコミュニケーションズの技術ブログです。

Databricks_EN

How to connect Databricks and KX: PyKX with kdb Insights license

Introduction I am Jung from the GLB Business Unit, Lakehouse Section. In this article, I will introduce how to use KX's PyKX to connect Databricks and KX. We will perform verification using kdb Insights license of KX's product kdb+ on Data…

Introduction to KDB.AI (5) - Sample code (LangChain and RAG)

Introduction This is Jung from the Lakehouse Department of the GLB Division. In this article, I would like to introduce the sample code practice of KDB.AI that I introduced in the previous article. KDB.AI is a Vector database powered by kd…

Introduction to KDB.AI (4) - Music recommendation (Recommendation Systems)

Introduction Hello, this is Chen from the Lakehouse Department of the GLB Division. The Lakehouse department is conducting verification using KDB.AI on the Databricks platform. An article by our company's Jung introduces KDB.AI and how to …

Introduction to KDB.AI (3) - Sample code (pattern matching of sensor data)

Introduction This is May from the Lakehouse Department of the GLB Division. KX Systems leveraged the time series database kdb+ to release KDB.AI in September 2023. KDB.AI is a powerful knowledge-based vector database and search engine that…

Introduction to KDB.AI (2) - Sample code (document search)

Introduction This is Jung from the Lakehouse Department of the GLB Division. In this article, I would like to introduce the sample code practice of KDB.AI that I introduced in the previous article. KDB.AI is a Vector database powered by kd…

A Technical Deep Dive into Unity Catalog's Practitioner Playbook Part 3/3

Upgrading to Unity Catalog: A New Step to Strengthen Data and AI Governance ​This is Johann from the Global Engineering Department of the GLB Division.​ Hello everyone. Today, I will be discussing an essential tool for strengthening data a…

A Technical Deep Dive into Unity Catalog's Practitioner Playbook Part 2/3

Leveraging Unity Catalog: A New Approach to Data and AI Governance ​This is Johann from the Global Engineering Department of the GLB Division. ​ In this post, we will delve into an intriguing presentation on data and AI governance. The tal…

A Technical Deep Dive into Unity Catalog's Practitioner Playbook Part 1/3

Unity Catalog: A New Tool for Centralizing Data and AI Governance ​This is Johann from the Global Engineering Department of the GLB Division. ​ Hello everyone. Today, I bring you an intriguing talk on data and AI governance. The presentati…

Planning and Executing a Snowflake Data Warehouse Migration to Databricks Part 2/2

Planning and Executing Data Warehouse Migration from Snowflake to Databricks ​This is Johann from the Global Engineering Department of the GLB Division. ​ Today, I will be discussing data warehouse migration, specifically focusing on the t…

Planning and Executing a Snowflake Data Warehouse Migration to Databricks Part 1/2

Planning and Executing Data Warehouse Migration from Snowflake to Databricks: An In-depth Look ​This is Johann from the Global Engineering Department of the GLB Division.​ ​ In this article, I would like to discuss data warehouse migration…

Project Aqueduct: Simplify Making Delta Lake Tributaries

Introduction ​This is Johann from the Global Engineering Department of the GLB Division.​ ​ Today, we will be sharing information about a presentation on Project Aqueduct. This presentation discussed the simplification of building connecto…

Best Practices for Setting Up Databricks SQL at Enterprise Scale

Introduction ​This is Johann from the Global Engineering Department of the GLB Division. ​ Today, I had the opportunity to watch a presentation on best practices for leveraging Databricks SQL at an enterprise scale. This presentation expla…

Unlock the Next Evolution of the Modern Data Stack With the Lakehouse Revolution

Introduction This is May from the GLB Division Lakehouse Department. Based on reports from members participating in the local Data + AI SUMMIT 2023 (DAIS), Share "Unlock the Next Evolution of the Modern Data Stack With the Lakehouse Revolu…

Labelbox | Unlocking Enterprise AI with Your Proprietary Data and Foundation Models

Introduction This is May from the GLB Division Lakehouse Department. Based on reports from members participating in the local Data + AI SUMMIT 2023 (DAIS), the exponential progress of AI and computing and the emergence of basic models were…

Develop Like a Pro in Databricks Notebooks Part 2 of 2

New Features in Databricks Notebooks for Professional Development and How to Use Them ​This is Johann from the Global Engineering Department of the GLB Division. ​ Today, I will introduce the new features and usage of Databricks Notebooks,…

Develop Like a Pro in Databricks Notebooks Part 1 of 2

Introduction ​This is Johann from the Global Engineering Department of the GLB Division.​ Today, I bring you coverage of a presentation titled "Develop Like a Pro in Databricks Notebooks". This talk, delivered by Neha, the leader of the No…

Delta Sharing: The Key Data Mesh Enabler Part 1

Introduction ​This is Johann from the Global Engineering Department of the GLB Division.​ In this article, we will discuss the presentation "Delta Sharing: The Key Data Mesh Enabler," which explains the enablers of data mesh. This presenta…

Microsoft | Next-Level Analytics with Power BI and Databricks

Data Innovation with Power BI and Databricks ​This is Johann from the Global Engineering Department of the GLB Division. ​ I would like to provide an easy-to-understand explanation of Microsoft's presentation "Next-Level Analytics with Pow…

Data + AI Summit 2023 Keynote Day 2 Part 3

Practical Limitations and Potential Benefits of AI - Data + AI Summit Keynote Day 2 ​This is Johann from the Global Engineering Department of the GLB Division. ​ Today, we bring you a report from the Data + AI Summit Keynote Day 2, which i…

Data + AI Summit Keynote Day 2 Part 2

Efforts and Technological Innovations for the Democratization of Data and AI - Data + AI Summit Keynote Day 2 ​This is Johann from the Global Engineering Department of the GLB Division.​ Today, I would like to talk about the keynote presen…

Data + AI Summit Keynote Day 2 Part 1

Discussing the Future and Evolution of AI at the Data + AI Summit Keynote Day 2 ​This is Johann from the Global Engineering Department of the GLB Division. ​ Hello, everyone. Today, I would like to talk about the presentations at the Data …

The Hitchhiker's Guide to Delta Lake Streaming

Introduction This is May from the GLB Division Lakehouse Department. I will share "The Hitchhiker's Guide to Delta Lake Streaming" on improving the data collection process, as reported by members attending the Data + AI SUMMIT2023 (DAIS) i…

Delta Live Tables A to Z: Best Practices for Modern Data Pipelines Part 1

Introduction I'm Sasaki from the Global Engineering Department of the GLB Division. I participated in the local Data + AI SUMMIT2023 (DAIS) and wrote an article summarizing the content of the session. Articles about the session at DAIS are…

Real-Time Streaming Solution for a Call Center Analytics: Business Challenges and Technical Enablement

Introduction This is Abe from the Lakehouse Department of the GLB Division. I wrote an article summarizing the content of the session based on the report by Mr. Gibo, who is participating in Data + AI SUMMIT2023 (DAIS) on site. Articles ab…

The Future of Data Management and Governance

Introduction This is May from the GLB Division Lakehouse Department. Based on reports from members participating in the local Data + AI SUMMIT 2023 (DAIS), Share "The Future of Data Management and Governance" about the future of data manag…

Embracing the Future of Data Engineering: The Serverless, Real-Time Lakehouse in Action

Introduction This is May from the GLB Division Lakehouse Department. "Embracing the Future of Data Engineering: The Serverless, Real- Share Time Lakehouse in Action. This session was conducted by Frank Munz, a technical reporter specializi…

Fine Tuning and Scaling Hugging Face with Ray AIR

Introduction I'm Sasaki from the Global Engineering Department of the GLB Division. I wrote an article summarizing the content of the session based on the report by Mr. Gibo, who is participating in Data + AI SUMMIT2023 (DAIS) on site. Art…

Data Biases and Generative AI: A Practitioner Discussion

Introduction This is May from the GLB Division Lakehouse Department. We will share "Data Biases and Generative AI: A Practitioner Discussion" on data biases and generative AI based on reports from members participating in the local Data + …

The Best Data Warehouse is a Lakehouse: How Databricks Achieves Operational Efficiency With the Lakehouse Architecture

Introduction This is May from the GLB Division Lakehouse Department. Based on reports from members participating in the local Data + AI SUMMIT 2023 (DAIS), a new architecture that combines a data warehouse and a data lake, "The Best Data W…

Learn How to Reliably Monitor Your Data and Model Quality in the Lakehouse

Introduction This Chen from GLB Business Department Lakehouse Department. I watched the Data + AI SUMMIT2023 (DAIS) webcast session "Learn How to Reliably Monitor Your Data and Model Quality in the Lakehouse". I will tell you the contents …