Building Recommendation Systems in Python and JAX

5. Putting It All Together: Content-Based Recommender
Revision Control Software
Python Build Systems
Random-Item Recommender
Obtaining the STL Dataset Images
Convolutional Neural Network Definition
Model Training in JAX, Flax, and Optax
Input Pipeline
Summary

II. Retrieval
6. Data Processing
Hydrating Your System
PySpark
Example: User Similarity in PySpark
DataLoaders
Database Snapshots
Data Structures for Learning and Inference
Vector Search
Approximate Nearest Neighbors
Bloom Filters
Fun Aside: Bloom Filters as the Recommendation System
Feature Stores
Summary

7. Serving Models and Architectures
Architectures by Recommendation Structure
Item-to-User Recommendations
Query-Based Recommendations
Context-Based Recommendations
Sequence-Based Recommendations
Why Bother with Extra Features?
Encoder Architectures and Cold Starting
Deployment
Models as APIs
Spinning Up a Model Service
Workflow Orchestration
Alerting and Monitoring
Schemas and Priors
Integration Tests
Observability
Evaluation in Production
Slow Feedback
Model Metrics
Continuous Training and Deployment
Model Drift
Deployment Topologies
The Evaluation Flywheel
Daily Warm Starts
Lambda Architecture and Orchestration
Logging
Active Learning
Summary

8. Putting It All Together: Data Processing and Counting Recommender
Tech Stack
Data Representation
Big Data Frameworks
Cluster Frameworks
PySpark Example
GloVE Model Definition
GloVE Model Specification in JAX and Flax
GloVE Model Training with Optax
Summary

III. Ranking
9. Feature-Based and Counting-Based Recommendations
Bilinear Factor Models (Metric Learning)
Feature-Based Warm Starting
Segmentation Models and Hybrids
Tag-Based Recommenders
Hybridization
Limitations of Bilinear Models
Counting Recommenders
Return to the Most-Popular-Item Recommender
Correlation Mining
Pointwise Mutual Information via Co-occurrences
Similarity from Co-occurrence
Similarity-Based Recommendations
Summary

10. Low-Rank Methods
Latent Spaces
Dot Product Similarity
Co-occurrence Models
Reducing the Rank of a Recommender Problem
Optimizing for MF with ALS
Regularization for MF
Regularized MF Implementation
WSABIE
Dimension Reduction
Isometric Embeddings
Nonlinear Locally Metrizable Embeddings
Centered Kernel Alignment
Affinity and p-sale
Propensity Weighting for Recommendation System Evaluation
Propensity
Simpson’s and Mitigating Confounding
Summary

11. Personalized Recommendation Metrics
Environments
Online and Offline
User Versus Item Metrics
A/B Testing
Recall and Precision
@ k
Precision at k
Recall at k
R-precision
mAP, MMR, NDCG
mAP
MRR
NDCG
mAP Versus NDCG?
Correlation Coefficients
RMSE from Affinity
Integral Forms: AUC and cAUC
Recommendation Probabilities to AUC-ROC
Comparison to Other Metrics
BPR
Summary

12. Training for Ranking
Where Does Ranking Fit in Recommender Systems?
Learning to Rank
Training an LTR Model
Classification for Ranking
Regression for Ranking
Classification and Regression for Ranking
WARP
k-order Statistic
BM25
Multimodal Retrieval
Summary

13. Putting It All Together: Experimenting and Ranking
Experimentation Tips
Keep It Simple
Debug Print Statements
Defer Optimization
Keep Track of Changes
Use Feature Engineering
Understand Metrics Versus Business Metrics
Perform Rapid Iteration
Spotify Million Playlist Dataset
Building URI Dictionaries
Building the Training Data
Reading the Input
Modeling the Problem
Framing the Loss Function
Exercises
Summary

IV. Serving
14. Business Logic
Hard Ranking
Learned Avoids
Hand-Tuned Weights
Inventory Health
Implementing Avoids
Model-Based Avoids
Summary

15. Bias in Recommendation Systems
Diversification of Recommendations
Improving Diversity
Applying Portfolio Optimization
Multiobjective Functions
Predicate Pushdown
Fairness
Summary

16. Acceleration Structures
Sharding
Locality Sensitive Hashing
k-d Trees
Hierarchical k-means
Cheaper Retrieval Methods
Summary

V. The Future of Recs
17. Sequential Recommenders
Markov Chains
Order-Two Markov Chain
Other Markov Models
RNN and CNN Architectures
Attention Architectures
Self-Attentive Sequential Recommendation
BERT4Rec
Recency Sampling
Merging Static and Sequential
Summary

18. What’s Next for Recs?
Multimodal Recommendations
Graph-Based Recommenders
Neural Message Passing
Applications
Random Walks
Metapath and Heterogeneity
LLM Applications
LLM Recommenders
LLM Training
Instruct Tuning for Recommendations
LLM Rankers
Recommendations for AI
Summary

Index
About the Authors