, ,

Data Quality Fundamentals

A Practitioner's Guide to Building Trustworthy Data Pipelines

Paperback Engels 2022 1e druk 9781098112042
€ 78,37
Levertijd ± 16 werkdagen
Gratis verzonden

Samenvatting

Do your product dashboards look funky? Are your quarterly reports stale? Is the data set you're using broken or just plain wrong? These problems affect almost every team, yet they're usually addressed on an ad hoc basis and in a reactive manner. If you answered yes to these questions, this book is for you.

Many data engineering teams today face the "good pipelines, bad data" problem. It doesn't matter how advanced your data infrastructure is if the data you're piping is bad. In this book, Barr Moses, Lior Gavish, and Molly Vorwerck, from the data observability company Monte Carlo, explain how to tackle data quality and trust at scale by leveraging best practices and technologies used by some of the world's most innovative companies.

- Build more trustworthy and reliable data pipelines
- Write scripts to make data checks and identify broken pipelines with data observability
- Learn how to set and maintain data SLAs, SLIs, and SLOs
- Develop and lead data quality initiatives at your company
- Learn how to treat data services and systems with the diligence of production software
- Automate data lineage graphs across your data ecosystem
- Build anomaly detectors for your critical data assets

Specificaties

ISBN13:9781098112042
Trefwoorden:databases, Data analyse
Taal:Engels
Bindwijze:paperback
Aantal pagina's:300
Uitgever:O'Reilly
Druk:1
Verschijningsdatum:16-9-2022
Hoofdrubriek:IT-management / ICT

Lezersrecensies

Wees de eerste die een lezersrecensie schrijft!

Inhoudsopgave

Preface
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments

1. Why Data Quality Deserves Attention—Now
What Is Data Quality?
Framing the Current Moment
Understanding the “Rise of Data Downtime”
Other Industry Trends Contributing to the Current Moment
Summary

2. Assembling the Building Blocks of a Reliable Data System
Understanding the Difference Between Operational and Analytical Data
What Makes Them Different?
Data Warehouses Versus Data Lakes
Data Warehouses: Table Types at the Schema Level
Data Lakes: Manipulations at the File Level
What About the Data Lakehouse?
Syncing Data Between Warehouses and Lakes
Collecting Data Quality Metrics
What Are Data Quality Metrics?
How to Pull Data Quality Metrics
Using Query Logs to Understand Data Quality in the Warehouse
Using Query Logs to Understand Data Quality in the Lake
Designing a Data Catalog
Building a Data Catalog
Summary

3. Collecting, Cleaning, Transforming, and Testing Data
Collecting Data
Application Log Data
API Responses
Sensor Data
Cleaning Data
Batch Versus Stream Processing
Data Quality for Stream Processing
Normalizing Data
Handling Heterogeneous Data Sources
Schema Checking and Type Coercion
Syntactic Versus Semantic Ambiguity in Data
Managing Operational Data Transformations Across AWS Kinesis and Apache Kafka
Running Analytical Data Transformations
Ensuring Data Quality During ETL
Ensuring Data Quality During Transformation
Alerting and Testing
dbt Unit Testing
Great Expectations Unit Testing
Deequ Unit Testing
Managing Data Quality with Apache Airflow
Scheduler SLAs
Installing Circuit Breakers with Apache Airflow
SQL Check Operators
Summary

4. Monitoring and Anomaly Detection for Your Data Pipelines
Knowing Your Known Unknowns and Unknown Unknowns
Building an Anomaly Detection Algorithm
Afgekapt op 60 regels

Managementboek Top 100

€ 78,37
Levertijd ongeveer 16 werkdagen
Gratis verzonden

Rubrieken

Producten

    Auteurs

      Trefwoorden

        Artikelen

          Data Quality Fundamentals