APC 技術ブログ

株式会社エーピーコミュニケーションズの技術ブログです。

株式会社 エーピーコミュニケーションズの技術ブログです。

Databricks_EN

Databricks Streaming: Project Lightspeed Goes Hyperspeed

Preface In the beginning part of this session, we introduced what real-time data processing is, and how this phenomenon is related to today's business environment. Real-time data processing is characterized by the provision of ongoing data…

How to Create a Holistic Customer View to Drive Performance and Revenue

Preface Welcome to our discussion on the pioneering project ‘Graphite’ at Condé Nast. Today, we delve deep into the creation of this innovative and comprehensive data product, designed to drive performance and revenue. Our session begins w…

How to Migrate from Snowflake to an Open Data Lakehouse Using Delta Lake UniForm

Preface Hello, I'm Jonathan Brideau, a Senior Product Manager at Databricks, leading initiatives centered around Delta Lake, particularly the Universal Format, or UniForm. Today's session addresses the increasingly common scenario of trans…

Data Warehousing Performance, Scale and Security with Databricks SQL

Preface Data warehousing in enterprise and mission-critical environments demands special attention to cost-efficiency and security. In our session, we delved into how Databricks SQL meets these stringent requirements. With the increase in …

Building Enterprise-Grade GenAI Apps with MLflow and Vector Search

Preface In the initial segment of the session, participants were welcomed by Dennis Dymanski and Kulkichada, who introduced themselves and elaborated on their professional backgrounds and areas of expertise. Dennis Dymanski, the chief soft…

Leveraging GenAI to Accelerate Innovation

Preface This session aimed to reveal how the adoption of AI technologies like natural language processing and generative AI is redefining and streamlining approaches to handling product information. This not only enhances the effectiveness…

Introducing the Databricks AI Security Framework (DASF) to Manage AI Security Risks

Preface Databricks' security team has developed the Databricks AI Security Framework (DASF) in collaboration with top cyber security researchers from OWASP, Gartner, NIST, McKinsey, and several Fortune 100 companies. This framework is desi…

An In Depth Look at the New Features of Apache Spark 3.5

Preface Hello everyone, this is Daniel. Today, we're focusing on the release of Apache Spark 3.5. Although this session concentrates specifically on version 3.5, it's worth noting that this release is remarkably stable, setting the stage f…

Accelerating LLM Inference with vLLM

Preface Today's session was hosted by Johan from UC Berkeley and Kade from AnyScale, who both played pivotal roles in the development of vLLM. A short survey was conducted to identify participants who have contributed to, deployed, or are …

Databricks’ Journey: Driving Business Transformation with Data & GenAI

Preface This session revolves around Databricks' efforts to drive business transformation. We particularly delve into two key themes - initiating internal changes and elevating data literacy. By utilizing their unique Data Intelligence Pla…

Databricks Vector Search: What, Why and How

Preface As recently highlighted by Sonali, a recent Computer Science graduate from New York University, traditional keyword-based search methods often fall short in fully capturing the intent and relevance of a user’s queries, frequently y…

The C-Level Guide to Data Strategy Success with 3Ps - People Process and Platform - in a GenAI World

Preface In this section, we focused on the decisive roles of machine learning and data curation in Generative AI. A hands-on introduction to machine learning using Databricks' Mosaic AI was shared by the speaker, Craig. Passionate about AI…

Efficient MLOps: Developing and Deploying ML Models with Databricks

Preface Welcome to the world of MLOps. In this session, we kick off the discussion with insights and challenges faced while implementing an effective MLOps framework, joined by product owners Lavinia Guadaniolo and Alessandro from Plenitud…

Building Your First GenAI App using Databricks, MosiacML and MLRun

Preface Welcome to the world filled with insights of building GenAI applications. Led by Aaron, the co-founder of Iguago which was acquired by McKinsey, and Bruce Philp, a tech fellow at McKinsey & Company, this session explores the concep…

Scaling Marketing and Docs with a Privacy-Safe RAG Model

Preface In the digital age, one of the principal challenges a business faces is managing an increasing amount of unstructured data. Analysing activities such as internal documents, meeting notes, recorded calls, and transcriptions can be e…

Mitigating LLM Hallucination Risk Through Research Backed Metrics

Preface In the realm of LLMs, mitigating the risk of hallucination and ensuring the accuracy of outputs is fundamental. This section details the inherent challenges during the human evaluation phase of model development and deployment. Und…

LLMs in Production: Fine-Tuning, Scaling, and Evaluation

Preface In the highly competitive market of today, businesses increasingly rely on Large Language Models (LLMs) to enhance various aspects of their operations. This session focuses on fine-tuning LLMs to meet specific business needs and ap…

Rapid LLM Prototyping with OpenAI, Databricks, and Streamlit

Preface In the session on rapid LLM prototyping utilizing technologies such as OpenAI, Databricks, and Streamlit, participants gained insights into how Gjensidige, Norway's largest insurance company, operates and builds strategic collabora…

Building High-Quality and Trusted Data Products with Databricks

Preface Building data products necessitates safety and reliability. Databricks provides a pathway to the creation of high-quality data products that adhere to these standards. In this session, two domain experts, Karthik, and Pomerit, will…

Simplify GenAI App Development with Secure, Custom AI Agents

Preface In the session titled "Introduction to AI Agent Workflows and Tools," strategies for streamlining the development of generative AI applications were showcased. The speakers, Atriti from Mosaic AI and Bilal, a Product Manager from A…

What's New with Data Sharing and Collaboration

Preface In the inaugural segment of this "Latest in Data Sharing and Collaboration” session, we delve into the complexities of data sharing and how AI-powered and Databricks is revolutionizing this field. Preface The Ideal Environment for …

Building a Production Scale, Totally Private, OSS RAG Pipeline with DBRX, Spark, and LanceDB

Preface In this session, the CEO of NCD, a data-centered organization, spoke extensively about his experiences in developing tools for data science and AI projects. The discussion was centered around their process of building the wholly pr…

How to Train or Fine-Tune a Custom LLM on Your Data with Databricks

Preface In this session, we will explore strategic timing and motivations for training a custom LLM. Leveraging Databricks, which is designed to efficiently and easily handle large-scale data and complex computational tasks, significantly …

Data Modeling Made Simple: A Non-Technical Beginner’s Guide

Preface Data modeling, a technical approach to representing complex business data in a visually comprehensible form, serves not only as a means for businesses to better understand their data landscape but also as a bridge for communication…

What’s New in Unity Catalog—with Live Demos

Preface Dive into the forefront of data and AI governance advancements with the product team of Unity Catalog. Unity Catalog, designed specifically for businesses that have adopted the Databricks Data Intelligence Platform, is the only sol…

How McDonalds Uses ML to Optimize Restaurant Site Selection

Preface As McDonald's actively pushes to expand its global presence, the company utilizes advanced machine learning technologies to smartly determine potential new store locations. This segment delves into how mathematical models and sophi…

Path to Production: Databricks Project CICD for Seamless Inner to Outer Dev Loops

Preface At the outset of the session, we honed in on various challenges associated with implementing CI/CD in Databricks projects. Central to our discussions was the role of community collaboration, playing a pivotal part in overcoming the…

Summary of posts on DATA+AI Summit 2024 (DAIS2024, June 13, 2024)

Introduction AP Communications has developed a portal site for DATA+AI SUMMIT 2024 in collaboration with local participants and members watching from Japan to deliver information about the event. [ Our event site] Introduction June 13, 202…

Data + AI Summit Keynote, Thursday

Preface An incredible program awaits us, but first and foremost, we want to express our appreciation to all our partners. Without the partners, this program could not have been realized. We want to thank the GSIs, hyperscalers, and all the…

Summary of posts on DATA+AI Summit 2024 (DAIS2024, Virtual Sessions)

Introduction AP Communications has developed a portal site for DATA+AI SUMMIT 2024 in collaboration with local participants and members watching from Japan to deliver information about the event. [ Our event site] Introduction Virtual Sess…