Serverless computing enables developers to concentrate solely on their applications rather than worry about where they've been deployed. With the Ray general-purpose serverless implementation in Python, programmers and data scientists can hide servers, implement stateful applications, support direct communication between tasks, and access hardware accelerators. In this book, experienced software architecture practitioners Holden Karau and Boris Lublinsky show you how to scale existing Python applications and pipelines, allowing you to stay in the Python ecosystem while reducing single points of failure and manual scheduling. Scaling Python with Ray is ideal for software architects and developers eager to explore successful case studies and learn more about decision and measurement effectiveness. If your data processing or server application has grown beyond what a single computer can handle, this book is for you. You'll explore distributed processing (the pure Python implementation of serverless) and learn how to: <ul> <li>Implement stateful applications with Ray actors</li> <li>Build workflow management in Ray</li> <li>Use Ray as a unified system for batch and stream processing</li> <li>Apply advanced data processing with Ray</li> <li>Build microservices with Ray</li> <li>Implement reliable Ray applications</li> </ul>

Scaling Python with Ray: Adventures in Cloud and Serverless Patterns

Computers & Technology

Lublinsky, Boris

Karau, Holden

Networking & Cloud Computing

O'Reilly Media

Modern systems contain multicore CPUs and GPUs that have the potential for parallel computing. But many scientific Python tools were not designed to leverage this parallelism. With this short but thorough resource, data scientists and Python programmers will learn how the Dask open source library for parallel computing provides APIs that make it easy to parallelize PyData libraries including NumPy, pandas, and scikit-learn. Authors Holden Karau and Mika Kimmins show you how to use Dask computations in local systems and then scale to the cloud for heavier workloads. This practical book explains why Dask is popular among industry experts and academics and is used by organizations that include Walmart, Capital One, Harvard Medical School, and NASA. With this book, you'll learn: <ul> <li>What Dask is, where you can use it, and how it compares with other tools</li> <li>How to use Dask for batch data parallel processing</li> <li>Key distributed system concepts for working with Dask</li> <li>Methods for using Dask with higher-level APIs and building blocks</li> <li>How to work with integrated libraries such as scikit-learn, pandas, and PyTorch</li> <li>How to use Dask with GPUs</li> </ul>

Scaling Python with Dask: From Data Science to Machine Learning

Kimmins, Mika

Data in all domains is getting bigger. How can you work with it efficiently? Recently updated for Spark 1.3, this book introduces Apache Spark, the open source cluster computing system that makes data analytics fast to write and fast to run. With Spark, you can tackle big datasets quickly through simple APIs in Python, Java, and Scala. This edition includes new information on Spark SQL, Spark Streaming, setup, and Maven coordinates.
Written by the developers of Spark, this book will have data scientists and engineers up and running in no time. You’ll learn how to express parallel jobs with just a few lines of code, and cover applications from simple batch jobs to stream processing and machine learning.
<ul>
<li>Quickly dive into Spark capabilities such as distributed datasets, in-memory caching, and the interactive shell</li>
<li>Leverage Spark’s powerful built-in libraries, including Spark SQL, Spark Streaming, and MLlib</li>
<li>Use one programming paradigm instead of mixing and matching tools like Hive, Hadoop, Mahout, and Storm</li>
<li>Learn how to deploy interactive, batch, and streaming applications</li>
<li>Connect to data sources including HDFS, Hive, JSON, and S3</li>
<li>Master advanced topics like data partitioning and shared variables</li>
</ul>

Learning Spark: Lightning-Fast Big Data Analysis

Zaharia, Matei

Konwinski, Andy

Wendell, Patrick

Web Development & Design

Apache Spark is amazing when everything clicks. But if you haven’t seen the performance improvements you expected, or still don’t feel confident enough to use Spark in production, this practical book is for you. Authors Holden Karau and Rachel Warren demonstrate performance optimizations to help your Spark queries run faster and handle larger data sizes, while using fewer resources.
Ideal for software engineers, data engineers, developers, and system administrators working with large-scale data applications, this book describes techniques that can reduce data infrastructure costs and developer hours. Not only will you gain a more comprehensive understanding of Spark, you’ll also learn how to make it sing.
With this book, you’ll explore:
<ul>
<li>How Spark SQL’s new interfaces improve performance over SQL’s RDD data structure</li>
<li>The choice between data joins in Core Spark and Spark SQL</li>
<li>Techniques for getting the most out of standard RDD transformations</li>
<li>How to work around performance issues in Spark’s key/value pair paradigm</li>
<li>Writing high-performance Spark code without Scala or the JVM</li>
<li>How to test for functionality and performance when applying suggested improvements</li>
<li>Using Spark MLlib and Spark ML machine learning libraries</li>
<li>Spark’s Streaming components and external community packages</li>
</ul>

High Performance Spark: Best Practices for Scaling and Optimizing Apache Spark

Warren, Rachel

<div> If you're training a machine learning model but aren't sure how to put it into production, this book will get you there. Kubeflow provides a collection of cloud native tools for different stages of a model's lifecycle, from data exploration, feature preparation, and model training to model serving. This guide helps data scientists build production-grade machine learning implementations with Kubeflow and shows data engineers how to make models scalable and reliable. Using examples throughout the book, authors Holden Karau, Trevor Grant, Ilan Filonenko, Richard Liu, and Boris Lublinsky explain how to use Kubeflow to train and serve your machine learning models on top of Kubernetes in the cloud or in a development environment on-premises. <ul> <li>Understand Kubeflow's design, core components, and the problems it solves</li> <li>Understand the differences between Kubeflow on different cluster types</li> <li>Train models using Kubeflow with popular tools including Scikit-learn, TensorFlow, and Apache Spark</li> <li>Keep your model up to date with Kubeflow Pipelines</li> <li>Understand how to capture model training metadata</li> <li>Explore how to extend Kubeflow with additional open source tools</li> <li>Use hyperparameter tuning for training</li> <li>Learn how to serve your model in production</li> </ul> </div>

Kubeflow for Machine Learning: From Lab to Production

Grant, Trevor

Liu, Richard

Filonenko, Ilan

Computer Science

Fastdata Processing With Spark

Programming Languages

Packt Pub Ltd

Perform real-time analytics using Spark in a fast, distributed, and scalable way About This Book<ul>
<li>Develop a machine learning system with Spark's MLlib and scalable algorithms</li>
<li>Deploy Spark jobs to various clusters such as Mesos, EC2, Chef, YARN, EMR, and so on</li>
<li>This is a step-by-step tutorial that unleashes the power of Spark and its latest features</li>
</ul>Who This Book Is ForFast Data Processing with Spark - Second Edition is for software developers who want to learn how to write distributed programs with Spark. It will help developers who have had problems that were too big to be dealt with on a single computer. No previous experience with distributed programming is necessary. This book assumes knowledge of either Java, Scala, or Python.What You Will Learn<ul>
<li>Install and set up Spark on your cluster</li> <li>Prototype distributed applications with Spark's interactive shell</li> <li>Learn different ways to interact with Spark's distributed representation of data (RDDs)</li> <li>Query Spark with a SQL-like query syntax</li> <li>Effectively test your distributed software</li> <li>Recognize how Spark works with big data</li> <li>Implement machine learning systems with highly scalable algorithms</li> </ul>In DetailSpark is a framework used for writing fast, distributed programs. Spark solves similar problems as Hadoop MapReduce does, but with a fast in-memory approach and a clean functional style API. With its ability to integrate with Hadoop and built-in tools for interactive query analysis (Spark SQL), large-scale graph processing and analysis (GraphX), and real-time analysis (Spark Streaming), it can be interactively used to quickly process and query big datasets.
Fast Data Processing with Spark - Second Edition covers how to write distributed programs with Spark. The book will guide you through every step required to write effective distributed programs from setting up your cluster and interactively exploring the API to developing analytics applications and tuning them for your purposes.

Fast Data Processing With Spark: Perform Real-time Analytics Using Spark in a Fast, Distributed, and Scalable Way

Sankar, Krishna

See the best price to sell, buy, or rent books by Karau, Holden. BookScouter helps to compare book prices from 25+ online bookstores and 30+ buyback vendors with a single search.

Books by Karau, Holden