Big Data Analysis with Python

Book Description

Get to grips with processing large volumes of data and presenting it as engaging, interactive insights using Spark and Python.

Key Features

• Get a hands-on, fast-paced introduction to the Python data science stack
• Explore ways to create useful metrics and statistics from large datasets
• Create detailed analysis reports with real-world data

Book Description

Processing big data in real time is challenging due to scalability, information inconsistency, and fault tolerance. Big Data Analysis with Python teaches you how to use tools that can control this data avalanche for you. With this book, you’ll learn practical techniques to aggregate data into useful dimensions for posterior analysis, extract statistical measurements, and transform datasets into features for other systems.

The book begins with an introduction to data manipulation in Python using pandas. You’ll then get familiar with statistical analysis and plotting techniques. With multiple hands-on activities in store, you’ll be able to analyze data that is distributed on several computers by using Dask. As you progress, you’ll study how to aggregate data for plots when the entire data cannot be accommodated in memory. You’ll also explore Hadoop (HDFS and YARN), which will help you tackle larger datasets. The book also covers Spark and explains how it interacts with other tools.

By the end of this book, you’ll be able to bootstrap your own Python environment, process large files, and manipulate data to generate statistics, metrics, and graphs.

What you will learn

• Use Python to read and transform data into different formats
• Generate basic statistics and metrics using data on disk
• Work with computing tasks distributed over a cluster
• Convert data from various sources into storage or querying formats
• Prepare data for statistical analysis, visualization, and machine learning
• Present data in the form of effective visuals

Who this book is for

Big Data Analysis with Python is designed for Python developers, data analysts, and data scientists who want to get hands-on with methods to control data and transform it into impactful insights. Basic knowledge of statistical measurements and relational databases will help you to understand various concepts explained in this book.

Table of Contents

1. The Python Data Science Stack
2. Statistical Visualizations
3. Working with Big Data Frameworks
4. Diving Deeper with Spark
5. Handling Missing Values and Correlation Analysis
6. Exploratory Data Analysis
7. Reproducibility in Big Data Analysis
8. Creating a Full Analysis Report

中文：

书名：Big Data Analysis with Python

掌握如何处理大量数据，并使用Spark和Python将其呈现为引人入胜的交互式洞察力。

主要特点

• 获取对Python数据科学堆栈的实际操作和快速介绍
• 探索从大型数据集创建有用的指标和统计数据的方法
• 使用真实数据创建详细的分析报告

图书描述

由于可扩展性、信息不一致和容错，实时处理大数据具有挑战性。使用Python进行大数据分析教会您如何使用工具来控制这种数据雪崩。有了这本书，你将学习实用的技术，将数据聚合成用于后验分析的有用维度，提取统计测量，并将数据集转换为其他系统的特征。

这本书首先介绍了如何使用PANAS在Python中进行数据操作。然后你将熟悉统计分析和绘图技术。有了多项实际操作活动，您将能够使用DASK分析分布在多台计算机上的数据。随着学习的进行，您将学习当内存中无法容纳全部数据时，如何为绘图聚合数据。您还将探索Hadoop(HDFS和纱线)，它将帮助您处理更大的数据集。这本书还介绍了Spark，并解释了它如何与其他工具交互。

到本书结束时，您将能够引导您自己的Python环境，处理大文件，并操作数据以生成统计数据、指标和图表。

你将学到什么

• 使用Python读取数据并将其转换为不同的格式
• 使用磁盘上的数据生成基本统计数据和指标
• 处理分布在群集上的计算任务
• 将各种来源的数据转换为存储或查询格式
• 为统计分析、可视化和机器学习准备数据
• 以有效的视觉形式呈现数据

这本书是为谁而写的

使用Python进行大数据分析是为希望亲手掌握控制数据并将其转化为有影响力的见解的方法的Python开发人员、数据分析师和数据科学家而设计的。统计测量和关系数据库的基本知识将帮助您理解本书中解释的各种概念。

目录表

1. Python数据科学堆栈
2. 统计可视化
3. 使用大数据框架
4. Diving Deeper with Spark
5. 缺失值的处理和相关性分析
6. Exploratory Data Analysis
7. 大数据分析中的可重复性
8. 创建完整的分析报告