Data Science at the Command Line

4. Creating Command-Line Tools
Overview
Converting One-Liners into Shell Scripts
Step 1: Create a File
Step 2: Give Permission to Execute
Step 3: Define a Shebang
Step 4: Remove the Fixed Input
Step 5: Add Arguments
Step 6: Extend Your PATH
Creating Command-Line Tools with Python and R
Porting the Shell Script
Processing Streaming Data from Standard Input
Summary
For Further Exploration

5. Scrubbing Data
Overview
Transformations, Transformations Everywhere
Plain Text
Filtering Lines
Extracting Values
Replacing and Deleting Values
CSV
Bodies and Headers and Columns, Oh My!
Performing SQL Queries on CSV
Extracting and Reordering Columns
Filtering Rows
Merging Columns
Combining Multiple CSV Files
Working with XML/HTML and JSON
Summary
For Further Exploration

6. Project Management with Make
Overview
Introducing Make
Running Tasks
Building, for Real
Adding Dependencies
Summary
For Further Exploration

7. Exploring Data
Overview
Inspecting Data and Its Properties
Header or Not, Here I Come
Inspect All the Data
Feature Names and Data Types
Unique Identifiers, Continuous Variables, and Factors
Computing Descriptive Statistics
Column Statistics
R One-Liners on the Shell
Creating Visualizations
Displaying Images from the Command Line
Plotting in a Rush
Creating Bar Charts
Creating Histograms
Creating Density Plots
Happy Little Accidents
Creating Scatter Plots
Creating Trend Lines
Creating Box Plots
Adding Labels
Going Beyond Basic Plots
Summary
For Further Exploration

8. Parallel Pipelines
Overview
Serial Processing
Looping Over Numbers
Looping Over Lines
Looping Over Files
Parallel Processing
Introducing GNU Parallel
Specifying Input
Controlling the Number of Concurrent Jobs
Logging and Output
Creating Parallel Tools
Distributed Processing
Get List of Running AWS EC2 Instances
Running Commands on Remote Machines
Distributing Local Data Among Remote Machines
Processing Files on Remote Machines
Summary
For Further Exploration

9. Modeling Data
Overview
More Wine, Please!
Dimensionality Reduction with Tapkee
Introducing Tapkee
Linear and Nonlinear Mappings
Regression with Vowpal Wabbit
Preparing the Data
Training the Model
Testing the Model
Classification with SciKit-Learn Laboratory
Preparing the Data
Running the Experiment
Parsing the Results
Summary
For Further Exploration

10. Polyglot Data Science
Overview
Jupyter
Python
R
RStudio
Apache Spark
Summary
For Further Exploration

11. Conclusion
Let’s Recap
Three Pieces of Advice
Be Patient
Be Creative
Be Practical
Where to Go from Here
The Command Line
Shell Programming
Python, R, and SQL
APIs
Machine Learning
Getting in Touch

A. List of Command-Line Tools

Index