APC 技術ブログ

株式会社エーピーコミュニケーションズの技術ブログです。

株式会社 エーピーコミュニケーションズの技術ブログです。

Databricks_EN

An In Depth Look at the New Features of Apache Spark 3.5

Preface Hello everyone, this is Daniel. Today, we're focusing on the release of Apache Spark 3.5. Although this session concentrates specifically on version 3.5, it's worth noting that this release is remarkably stable, setting the stage f…

Accelerating LLM Inference with vLLM

Preface Today's session was hosted by Johan from UC Berkeley and Kade from AnyScale, who both played pivotal roles in the development of vLLM. A short survey was conducted to identify participants who have contributed to, deployed, or are …

Databricks’ Journey: Driving Business Transformation with Data & GenAI

Preface This session revolves around Databricks' efforts to drive business transformation. We particularly delve into two key themes - initiating internal changes and elevating data literacy. By utilizing their unique Data Intelligence Pla…

Databricks Vector Search: What, Why and How

Preface As recently highlighted by Sonali, a recent Computer Science graduate from New York University, traditional keyword-based search methods often fall short in fully capturing the intent and relevance of a user’s queries, frequently y…

The C-Level Guide to Data Strategy Success with 3Ps - People Process and Platform - in a GenAI World

Preface In this section, we focused on the decisive roles of machine learning and data curation in Generative AI. A hands-on introduction to machine learning using Databricks' Mosaic AI was shared by the speaker, Craig. Passionate about AI…

Efficient MLOps: Developing and Deploying ML Models with Databricks

Preface Welcome to the world of MLOps. In this session, we kick off the discussion with insights and challenges faced while implementing an effective MLOps framework, joined by product owners Lavinia Guadaniolo and Alessandro from Plenitud…

Building Your First GenAI App using Databricks, MosiacML and MLRun

Preface Welcome to the world filled with insights of building GenAI applications. Led by Aaron, the co-founder of Iguago which was acquired by McKinsey, and Bruce Philp, a tech fellow at McKinsey & Company, this session explores the concep…

Scaling Marketing and Docs with a Privacy-Safe RAG Model

Preface In the digital age, one of the principal challenges a business faces is managing an increasing amount of unstructured data. Analysing activities such as internal documents, meeting notes, recorded calls, and transcriptions can be e…

Mitigating LLM Hallucination Risk Through Research Backed Metrics

Preface In the realm of LLMs, mitigating the risk of hallucination and ensuring the accuracy of outputs is fundamental. This section details the inherent challenges during the human evaluation phase of model development and deployment. Und…

LLMs in Production: Fine-Tuning, Scaling, and Evaluation

Preface In the highly competitive market of today, businesses increasingly rely on Large Language Models (LLMs) to enhance various aspects of their operations. This session focuses on fine-tuning LLMs to meet specific business needs and ap…

Rapid LLM Prototyping with OpenAI, Databricks, and Streamlit

Preface In the session on rapid LLM prototyping utilizing technologies such as OpenAI, Databricks, and Streamlit, participants gained insights into how Gjensidige, Norway's largest insurance company, operates and builds strategic collabora…

Building High-Quality and Trusted Data Products with Databricks

Preface Building data products necessitates safety and reliability. Databricks provides a pathway to the creation of high-quality data products that adhere to these standards. In this session, two domain experts, Karthik, and Pomerit, will…

Simplify GenAI App Development with Secure, Custom AI Agents

Preface In the session titled "Introduction to AI Agent Workflows and Tools," strategies for streamlining the development of generative AI applications were showcased. The speakers, Atriti from Mosaic AI and Bilal, a Product Manager from A…

What's New with Data Sharing and Collaboration

Preface In the inaugural segment of this "Latest in Data Sharing and Collaboration” session, we delve into the complexities of data sharing and how AI-powered and Databricks is revolutionizing this field. Preface The Ideal Environment for …

Building a Production Scale, Totally Private, OSS RAG Pipeline with DBRX, Spark, and LanceDB

Preface In this session, the CEO of NCD, a data-centered organization, spoke extensively about his experiences in developing tools for data science and AI projects. The discussion was centered around their process of building the wholly pr…

How to Train or Fine-Tune a Custom LLM on Your Data with Databricks

Preface In this session, we will explore strategic timing and motivations for training a custom LLM. Leveraging Databricks, which is designed to efficiently and easily handle large-scale data and complex computational tasks, significantly …

Data Modeling Made Simple: A Non-Technical Beginner’s Guide

Preface Data modeling, a technical approach to representing complex business data in a visually comprehensible form, serves not only as a means for businesses to better understand their data landscape but also as a bridge for communication…

What’s New in Unity Catalog—with Live Demos

Preface Dive into the forefront of data and AI governance advancements with the product team of Unity Catalog. Unity Catalog, designed specifically for businesses that have adopted the Databricks Data Intelligence Platform, is the only sol…

How McDonalds Uses ML to Optimize Restaurant Site Selection

Preface As McDonald's actively pushes to expand its global presence, the company utilizes advanced machine learning technologies to smartly determine potential new store locations. This segment delves into how mathematical models and sophi…

Path to Production: Databricks Project CICD for Seamless Inner to Outer Dev Loops

Preface At the outset of the session, we honed in on various challenges associated with implementing CI/CD in Databricks projects. Central to our discussions was the role of community collaboration, playing a pivotal part in overcoming the…

Summary of posts on DATA+AI Summit 2024 (DAIS2024, June 13, 2024)

Introduction AP Communications has developed a portal site for DATA+AI SUMMIT 2024 in collaboration with local participants and members watching from Japan to deliver information about the event. [ Our event site] Introduction June 13, 202…

Data + AI Summit Keynote, Thursday

Preface An incredible program awaits us, but first and foremost, we want to express our appreciation to all our partners. Without the partners, this program could not have been realized. We want to thank the GSIs, hyperscalers, and all the…

Summary of posts on DATA+AI Summit 2024 (DAIS2024, Virtual Sessions)

Introduction AP Communications has developed a portal site for DATA+AI SUMMIT 2024 in collaboration with local participants and members watching from Japan to deliver information about the event. [ Our event site] Introduction Virtual Sess…

A Practical Introduction to Machine Learning with Databricks Mosaic AI

Preface Welcome to "A Practical Introduction to Machine Learning Using Databricks Mosaic AI." Today, we are thrilled to host Craig, who oversees AI and machine learning product management at Databricks, to share insights into this burgeoni…

Accelerating Operational Excellence with Generative AI

Preface In the financial and insurance industries, where customer service demands swift and accurate responses, our company has implemented a Retrieval Augmented Generation (RAG) system. This AI-driven solution leverages deep learning tech…

MLOps and AI Governance in Healthcare: Providence's Use Case

Preface This session introduced the progress that Providence HealthCare has made over the past year, particularly focusing on the development of their AI/ML model marketplace and how they have operated it within Azure’s secure cloud enviro…

Rise of the Medallion Mesh

Preface In the swiftly evolving tech landscape of today, organizations face numerous challenges in fully harnessing the potential of data-driven endeavors. The session "Rise of the Medallion Mesh" directly addresses these challenges, focus…

Automating Unity Catalog Migration with UCX: Building Robust Python Applications on Databricks

Preface In the process of developing Terraform Providers and Python SDKs, UCX amassed invaluable knowledge around building robust Python applications on Databricks. Preface The Importance of Regular Integration Testing The Power of Pytest …

Dive into Building Production Quality Gen AI Applications

Preface In this session, emphasis was placed on deeply understanding the considerations of quality and performance required when transitioning from a prototype to a product. The speakers, Miles Curtis and Eric, provided expert insights int…

Data Platform Modernization with a Data Mesh Architecture

Preface In order to quantify the scale of their operations, Lazans shared a short video that clarified the organization’s core beliefs and targets. This video, made in Portuguese and subtitled in English, emphasized the fact that Bradesco …