Enabling Microservice Success

Foreword
Preface
Why I Wrote This Book
Who Should Read This Book
Navigating This Book
Part I: Context
Part II: Organizational Structure and Culture
Part III: Building and Operating
Appendixes
Case Studies
Conventions Used in This Book
O’Reilly Online Learning
How to Contact Us
Acknowledgments

I. Context
1. Understanding Microservices
Defining the Microservices Architectural Style
A Suite of Services
Each Running in Its Own Process
Communicating with Lightweight Mechanisms
Built Around Business Capabilities
Independently Deployable
“Small”
With a Bare Minimum of Centralized Management
Heterogeneous
Forerunners and Alternatives
The Monolith
Modular Monoliths
Service-Oriented Architecture
The Microservices Ecosystem
Infrastructure as Code
Continuous Delivery
The Public Cloud
New Deployment Options
DevOps
Observability
Advantages of Microservices
Independently Scalable
Robust
Easy to Release Small Changes Frequently
Support Flexible Technology Choices
Challenges of Microservices
Latency
Estate Complexity
Operational Complexity
Data Consistency
Security
Finding the Right Level of Granularity
Handling Change
Require Organizational Change
Change the Developer Experience
In Summary

2. Effective Software Delivery
Regularly Delivering Business Value
High Deployment Frequency
Short Lead Time for Changes
Running Experiments
Separating Deploying Code from Releasing Functionality
Handling Work That Goes Across Team Boundaries
Adapting to Changing Priorities
Maintaining Appropriate Service Levels
When a Release Goes Wrong
Knowing When Something Important Is Broken
Restore Some Level of Service Quickly
Avoid Failure Cascades
Spending Most of Your Time on Meaningful Work
Not Having to Start Again
Keeping Risk at an Acceptable Level
How Microservices Measure Up
In Summary

3. Are Microservices Right for You?
Reasons to Choose Microservices
Scaling the Organization
Developer Experience
Separating Out Areas with Compliance and Security Requirements
Scaling for Load
Increasing Robustness
Increasing Flexibility
Conditions for Success
Domain Understanding
Products Not Projects
Leadership Support
Teams That Want Autonomy
Processes That Enable Autonomy
Technical Maturity
Managing Change
Sticking with a Monolithic Architecture
Enable Zero-Downtime Deployments
Build a Modular Monolith
Everything Is Distributed Now
The Rise of Cloud Native
SaaS Makes Sense
Recommendations
Starting from Scratch
Replacing an Existing Monolith
Measuring Success
In Summary

II. Organizational Structure and Culture
4. Conway’s Law and Finding the Right Boundaries
Conway’s Law
The Inverse Conway Maneuver
Possible Boundaries
Business Domains
Locations
Technologies
Compliance
Tolerance for Failure
Frequency of Changes
Recommendations
Identifying When Boundaries Are Wrong
In Summary

5. Building Effective Teams
Organizational Culture
Open
Learning
Empowering
Optimized for Change
The Westrum Model
Effective Teams
Motivated through Autonomy, Mastery, and Purpose
Aligned to Business Domain
Appropriately Sized
Cross-Functional and T-shaped
Strong Ownership
Long Lived
Sustainable Cognitive Load
High Trust and High Psychological Safety
Part of a Group
Optimizing for Flow
Stream-Aligned
Enabling
Complicated Subsystem
Platform
In Summary

6. Enabling Autonomy
What Is Autonomy?
Why Does Autonomy Matter?
Limits to Autonomy
The Right Amount of Communication
Interaction Styles
Collaboration
X-as-a-Service
Facilitating
Ways of Working That Support Autonomy
Aligning on Outcomes
Light Touch Governance
Trust but Verify
Agreeing and Aligning on Technology
The Role of the Individual Contributor
Minimum Viable Competencies
Making Space for Learning
Responsibilities of Autonomous Teams
Active Ownership
Communication and Cooperation
Compliance with Standards
Maintaining a Team Page
In Summary

7. Engineering Enablement and Paving the Road
What’s in a Name?
Building a Platform
Platform Services
Organization-Level Concerns
Building the Thinnest Viable Platform
Build for the Needs of the Majority
Platform as a Product
Beyond the Platform
Vendor Engineering
APIs, Templates, Libraries, and Examples
A Service Catalog
Insights
Paving the Road
What Capabilities to Include
Make It Optional
Keep It Small
How to Go Off Road
Bringing the Treasure Back
Internal Developer Portals
Building a Platform People Actually Use
Making Sure What You Build Meets a Need
Market It
Look for Signs You Are Getting It Wrong
Principles for Building a Paved Road
Optional
Provides Value
Self-Service
Owned and Supported
Easy to Use
Guides People to Do the Right Thing
Composable and Extendable
Measuring Impact
When to Invest in Engineering Enablement
In Summary

8. Ensuring “You Build It, You Run It”
Why Microservices Implies DevOps
Release on Demand
Work on Operational Features
Building Things Differently
Good Runbooks
Running on Someone Else’s Servers
Getting Comfortable in Production
Supporting Your System in Production
Assign Dedicated In-Hours Ops Support
Improve Alerts and Documentation
Identify the Haunted Forests
Practice
Out-of-Hours Support
Allow People to Opt Out
Formal Rotas Versus Best Endeavors
Make Sure Calls Are Rare
Only for Critical Systems
Provide Support and Guidance
Incident Management
Blameless Culture
Raising an Incident
Roles to Assign
During the Incident
After the Incident
Learning from Incidents
In Summary

III. Building and Operating
9. Active Service Ownership
Responding to the Log4Shell Vulnerability
A Counter Example: Equifax and a Struts Vulnerability
Ownership During Active Development
Strong Ownership
Weak Ownership
Collective Ownership
Once a Service Is Feature Complete
No Ownership
Nominal Ownership
Active Ownership
What Active Ownership Means
Code Stewardship
Upgrades and Patching
Migrations
Production Support
Documentation
Knowing Your Estate
Your Own Software
Dependencies
Third-Party Software
What You Need from a Service Catalog
Graph-Based Model
API-Driven
Extensible
Flexible Schema
Provides Different Views Across the Estate
Transferring Ownership
What Does a Good Transfer Look Like?
Meeting Quality Expectations
Operational Handover
Replacing
What to Do If You’re Struggling
Make the Business Case
Start with Critical Systems
Make Your Best Guess at Owners
Deliver Value from the Data
Aim for Continuous Improvement
Look for Teams That Are Overwhelmed
Services Shouldn’t Live Forever
In Summary

10. Getting Value from Testing
Why Do We Test?
Building the Thing Right
Building the Right Thing
Picking Up Regressions
Meeting Quality-of-Service Requirements
Shifting Testing Left
What Makes a Good Test?
Fast and Early Feedback
Easy to Change
Finds Real Problems
Types of Testing
The Testing Pyramid
Unit Tests
Service Tests
End-to-End Tests
Contract Tests
Consistency Tests
Exploratory Tests
Cross-Functional Testing
Testing in Production
Is It Safe?
Staging Is Not Production-Like
Your Customers Can Surprise You
You Can’t Test for Every Variation
You Don’t Have to Roll a Change Out to Everyone
Monitoring as Testing
Testing Your Infrastructure
Chaos Engineering
Testing Failovers and Restores
Quality Is About More Than Testing
What to Do If You’re Struggling
Not Enough Automated Testing
Tests That Aren’t Providing Value
In Summary

11. Governance and Standardization: Finding the Balance
Why Governance Matters
Know Your Estate
What Sort of Information Is Relevant?
Guardrails and Policies
Automating Guardrails
What to Include
The FT’s Guardrails
Aligning on Guardrails
Tech Governance Group
Benefits of the TGG
Choosing Technologies
The Technology Lifecycle
Save Innovation for Key Business Outcomes
Use Boring Technology
Limit the Alternatives
Be Clear on Where Duplication Is Acceptable
Expect Things to Change
Insight Leads to Action
Governance in Other Organizations
Governance at Monzo
Governance at Skyscanner
What to Do If You’re Struggling
In Summary

12. Building Resilience In
What Is Resilience?
Resilience for Distributed Systems
Resilience for Microservices
Understanding Your Service Level Requirements
Service Level Objectives
Error Budgets
Building Resilient Services
Redundancy
Fast Startup and Graceful Shutdown
Set Appropriate Timeouts
Back Off and Retry
Make Your Requests Idempotent
Protect Yourself
Testing Service Resilience
Make Building Resilient Services Easy
Building Resilient Systems
Caching
Handling Cascading Failures
Fallback Behavior
Avoiding Unnecessary Work
Go Asynchronous
Failover
Backup and Restore
Disaster Recovery
Building a Resilient Platform
Resilience to External Issues
Internal Tooling
Validating Your Resilience Choices
Chaos Engineering
Testing Backup and Restore
Practice Makes Perfect
Load Testing
Learn from Incidents
One Thing at a Time
What to Do If You’re Struggling
In Summary

13. Running Your System in Production
Operational Challenges of Microservices
Different Technologies Mean Different Support Knowledge Is Needed
Ephemeral Infrastructure
Rapid Change
Alert Overload
Complex Systems Run in Degraded Mode
Building Observability In
Logging
Monitoring and Metrics
Log Aggregation
OpenTelemetry
Focus on Events
Distributed Tracing
Archiving Observability Data
Building Your Own Tools
Spotting Issues
Getting Alerting Right
Healthchecks
Monitoring Business Outcomes
Understanding What Normal Looks Like
Mitigation
Troubleshooting
Maintaining Useful Documentation
Knowing What’s Changed
Problems with External Systems
Tooling Characteristics
Learning from Incidents
What to Do If You’re Struggling
In Summary

14. Keeping Things Up-to-Date
Why Is This a Challenge?
Minimizing the Impact of Change
Think About the Long Term
A Reason to Be on the Paved Road
Choose Managed Services and SaaS Options
Provide APIs
Immutable and Ephemeral Infrastructure
Decommission and Deprecate
Types of Change
Emergency Changes
Minor Planned Changes
Major Planned Changes
Responding to Change
Understand the Landscape
Define Guiding Policies
Making a Decision
Who Gets to Decide?
Scheduling Work
Managing Change
Clarity
Communication
Empathy
Execution
What to Do If You’re Struggling
In Summary
Afterword
Why Microservices?
The Importance of Flow
Support for Autonomy
The Rise of Platform Engineering
Wrapping Up

A. Microservices Assessment
Do You Need Microservices?
Scaling Challenges
Technical Reasons
Spotting Potential Pitfalls
Organizational Structure and Culture
Software Delivery Approach

B. Recommended Reading
Part I: Context
Part II: Organizational Structure and Culture
Part III: Building and Operating

Index
About the Author