Discover how Delta Lake simplifies the process of building data lakehouses and data pipelines at scale. With this practical guide, data engineers, data scientists, and data analysts will explore key data reliability challenges and learn to apply modern data engineering and management techniques. You'll also understand how ACID transactions bring reliability to data lakehouses at scale. This book helps you: <ul> <li>Understand key data reliability challenges</li> <li>Examine data management and engineering techniques using the modern data stack</li> <li>Realize data reliability improvements using Delta Lake</li> <li>Concurrently run streaming and batch jobs against your data lake</li> <li>Execute update, delete, and merge commands</li> <li>Use time travel to rollback and examine previous versions of your data</li> <li>Build a streaming data quality pipeline following the medallion construct</li> </ul>

Delta Lake: The Definitive Guide: Modern Data Lakehouse Architectures with Data Lakes

Computers & Technology

Lee, Denny

Haines, Scott

Wentling, Tristen

Babu, Prashanth

Software

O'Reilly Media

From the Back Cover\nLeverage Apache Spark within a modern data engineering ecosystem. This hands-on guide will teach you how to write fully functional applications, follow industry best practices, and learn the rationale behind these decisions. With Apache Spark as the foundation, you will follow a step-by-step journey beginning with the basics of data ingestion, processing, and transformation, and ending up with an entire local data platform running Apache Spark, Apache Zeppelin, Apache Kafka, Redis, MySQL, Minio (S3), and Apache Airflow. 
Apache Spark applications solve a wide range of data problems from traditional data loading and processing to rich SQL-based analysis as well as complex machine learning workloads and even near real-time processing of streaming data. Spark fits well as a central foundation for any data engineering workload. This book will teach you to write interactive Spark applications using Apache Zeppelin notebooks, write and compile reusable applications and modules, and fully test both batch and streaming. You will also learn to containerize your applications using Docker and run and deploy your Spark applications using a variety of tools such as Apache Airflow, Docker and Kubernetes. 
Reading this book will empower you to take advantage of Apache Spark to optimize your data pipelines and teach you to craft modular and testable Spark applications. You will create and deploy mission-critical streaming spark applications in a low-stress environment that paves the way for your own path to production. 
What You Will Learn Simplify data transformation with Spark Pipelines and Spark SQL 
Bridge data engineering with machine learning Architect modular data pipeline applications 
Build reusable application components and libraries Containerize your Spark applications for consistency and reliability Use Docker and Kubernetes to deploy your Spark applications 
Speed up application experimentation using Apache Zeppelin and Docker Understand serializable structured data and data contracts Harness effective strategies for optimizing data in your data lakes Build end-to-end Spark structured streaming applications using Redis and Apache Kafka Embrace testing for your batch and streaming applications Deploy and monitor your Spark applications\nIntermediate-Advanced user level

Modern Data Engineering with Apache Spark: A Hands-On Guide for Building Mission-Critical Streaming Applications

Computer Science

Apress

See the best price to sell, buy, or rent books by Haines, Scott. BookScouter helps to compare book prices from 25+ online bookstores and 30+ buyback vendors with a single search.

Books by Haines, Scott