APC 技術ブログ

株式会社エーピーコミュニケーションズの技術ブログです。

株式会社 エーピーコミュニケーションズの技術ブログです。

SIG の検索結果:

Best Practices for Setting Up Databricks SQL at Enterprise Scale

… SQL is designed with data security in mind, allowing companies to operate their data warehouses with confidence. Scalability: Databricks SQL supports enterprise-scale data warehouse configurations and can efficiently process large amounts …

Delta Sharing: The Key Data Mesh Enabler Part 2

…are his insights on data mesh and Delta Sharing. ​ The theme of this presentation is "Delta Sharing: The Key Data Mesh Enabler," aiming to explain the key enablers of data mesh. The target audience includes technical professionals intereste…

Data + AI Summit Keynote Day 2 Part 2

… bringing significant progress in the field. AI has existed alongside the computer industry and the internet for a long time, but it has only recently started to function due to the availability of large amounts of data. The concept of mult…

Data + AI Summit Keynote Day 2 Part 1

…ed out as significant barriers. ​ The Importance of Data in Driving Improvement and Impact in AI Use Cases ​ Finally, the importance of data in driving improvement and impact in AI use cases was emphasized. Data plays a central role in the …

The Hitchhiker's Guide to Delta Lake Streaming

…process Designing a practical data collection process This guidebook will help you improve your data collection process and ensure efficient data collection. Summary Improving the data collection process is important to improve data quality…

The Future of Data Management and Governance

…-up and insight management, and enables data source consolidation. Features that facilitate collaboration with external data sources Lakehouse Federation offers the latest concepts, features and services, including: Data Integration: You ca…

Fine Tuning and Scaling Hugging Face with Ray AIR

… AIR is designed to allow customization of various settings for model training and evaluation. This allows developers to choose the settings that best suit their needs. ​ Take advantage of the latest concepts and features ​ RAI AIR leverage…

Data Biases and Generative AI: A Practitioner Discussion

…dataset Design the generative AI algorithm with diversity in mind Evaluate bias in generated data and make improvements Through such efforts, it is expected that the development of generative AI, including diversity and ethnicity, will be p…

Data Application Bundles: A Standard, Unified Approach to Deploying Data Products on Databricks

…ludes redesigning and retesting existing data products. About the latest concepts, features and services Databricks Asset Bundles offer the latest concepts, features and services such as: Versioning: Databricks Asset Bundles support version…

Streaming Schema Drift Discovery and Controlled Mitigation

…thod of designing a data processing system to be flexible to schema changes. This allows the system to automatically react to schema drift and maintain data accuracy and consistency. About the latest concepts, features and services Recently…

Data Caching Strategies for Data Analytics and AI

…d cache design in AI training ​ AI training requires high-speed processing of large amounts of data. Therefore, optimizing data access patterns and cache design are important. Specifically, factors such as: ​ Data locality: By concentrating…

Unlocking the Value of Data Sharing in Financial Services with Lakehouse

…ain new insights. About the latest concepts, features and services Lakehouse offers the latest concepts, features and services. This further increases the value of data sharing in the financial services industry. Introduction of new data an…

Testing Generative AI Models: What You Need to Know

…e model design stage, apply methods to reduce bias, and introduce technologies to protect privacy. 3. Security and privacy risks Security and privacy risks are the risks that AI systems can be exploited by malicious attackers. Risks include…

Databricks Cost Management: Tips and Tools to Stay Under Budget

…ances can significantly reduce costs Resource monitoring and analysis: Monitoring and analyzing Databricks resources enables cost visualization and appropriate cost management. Understanding clusters and cost management is key In Databricks…

Factory of the Future: Case Study of Building Digital Twins Using Knowledge Graphs and Generative AI

…tes new designs and processes based on data, supporting the efficient reconstruction of factories. ​ Combining these technologies enables factory reconfiguration simulation using digital twins, enabling more efficient and flexible factory o…

Jet Streaming Data and Predictive Analytics

…al-time insight into flight operations and take appropriate countermeasures. Leveraging Lakehouse and Apache Spark™ Ascentia leverages Lakehouse and Apache Spark™. These technologies make it possible to efficiently process large amounts of …

De-Risking Language Models for Faster Adoption

…a quality significantly affects the performance and reliability of AI systems. By using high-quality data, you can reduce the risks of the system and develop a safer and more reliable system. ​ Involving Stakeholders ​ In the development of…

A Fireside Chat: Building Your Startup on Databricks

…usiness insights ​ Benefits of Partnering with Databricks ​ By partnering with Databricks, startups can enjoy the following benefits: ​ 1. Access to a larger customer base 2. Opportunities for joint collaboration 3. Improved data analytics …

Getting Insight From Anything: Gathering Data With IoT Devices and Delta Live

…Getting Insight From Anything: Gathering Data With IoT Devices and Delta Live." Collecting Data with Delta Live)” is an overview of the session. In this session, Andrew Blance explained how to collect data for data-driven decision-making an…

Getting Insight From Anything: Gathering Data With IoT Devices and Delta Live(インサイトを得る:IoTデバイスとデルタライブでデータを収集する)

…Getting Insight From Anything: Gathering Data With IoT Devices and Delta Live(インサイトを得る:IoTデバイスとデルタライブでデータを収集する)」というセッションの概要をご紹介します。 本セッションでは、Andrew Blance氏による、データ駆動の意思決定やビジネス問題の洞察を得るためのデータ収集方法について説明しました。データ収集における問題点とその解決策について議論が行われました。 データ駆…

Why a Major Japanese Financial Institution Chose Databricks to Accelerate its Data and AI-Driven Journey

…rivacy: Designed with a focus on protecting your data Taking advantage of these features, we provide an environment in which companies can efficiently analyze data. Democratizing Data and AI with Databricks By using Databricks for Trusted D…

Scaling Deep Learning Using Delta Lake Storage Format on Databricks

…d cache design in AI training ​ AI training requires high-speed processing of large amounts of data. Therefore, optimizing data access patterns and cache design are important. Specifically, factors such as: ​ Data locality: By concentrating…

Increasing Trust in Your Data: Enabling a Data Governance Program on Databricks Using Unity Catalog and ML-Driven MDM Part 2/2

…ain the design of the data hub and data lakehouse built by Comcast and introduce the efforts to improve data reliability through data matching and machine learning solutions. ​ Let's dive in! www.ap-com.co.jp ​ Data Hub and Data Lakehouse D…

Maximizing Value From Your Data with Lakehouse AI

…so made a significant investment in the Databricks CLI to code the MLOps infrastructure and provide deep integration with the CICD pipeline. This allows ML engineering teams to focus on high-value features and reap the benefits of having da…

Extending Lakehouse Architecture with Collaborative Identity

…g these insights, companies will be able to use data more effectively and maximize business value. We will continue to deliver the latest information in the field of data and AI, so please look forward to it! Conclusion This content based o…

Databricks As Code: How to Effectively Automate a Secure Lakehouse Using Terraform for Resource Provisioning

…product design Fleet health monitoring Improved customer experience By utilizing these data, Rivian aims to improve business efficiency and competitiveness. Data integration with Databricks Lakehouse Rivian uses Databricks Lakehouse to aggr…

Building a Real-Time Model Monitoring Pipeline on Databricks

…use the insights gained through it to improve AI products and machine learning models were explained. Model monitoring is an essential element for maintaining and improving the performance of machine learning models. By considering various …

Databricks Connect Powered by Spark Connect: Develop and Debug Spark From Any Developer Tool

…t and its Significance ​ With the introduction of Spark Connect, the Spark architecture was broken down into a single client and server, with a properly designed protocol introduced between them. This resulted in the following benefits: ​ A…

How Rec Room Processes Billions of Events Per Day with Databricks and RudderStack

…Extract insights using data science to improve user gaming experiences Optimize in-game features and content based on insights ​ Incorporating the Latest Concepts, Features, and Services ​ Rec Room's data team is further improving data proc…

The Future Brought by the Democratization of Data and AI - Data + AI Summit Keynote Day 1

… provides signals such as popularity, frequent users, update time, and upstream quality issues. This makes it easier to search for data within an organization and improves the efficiency of data utilization. ​ During the keynote, a demonstr…