Architecting IoT Solutions on Azure

Preface
Who Should Read This Book
Navigating This Book
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments

1. The IoT Landscape
Off Azure
IoT Devices
Edge Computing
Azure
IoT Messaging and Management
Data Processing
Data Persistence
Data Presentation Layer
Data Consumers
Monitoring, Logging, and Security
Conclusion

2. Azure-Centric IoT Devices
IoT Devices as the Nexus of Three Domains
Hardware
Software
Cloud
IoT Devices and Microsoft
Microsoft the Software Company
Microsoft as a Hardware Company
Microsoft as a Cloud Company
IoT on Azure: Microsoft’s Combination of Software, Hardware, and Cloud
Azure Sphere
Azure Sphere Hardware
Azure Sphere Software
Azure Sphere Cloud Services
What’s It For?
What Makes It Unique?
Azure MXChip
MXChip Hardware
MXChip Software
MXChip Cloud Services
What’s It For?
What Makes It Unique?
Kinect
Kinect Hardware
Kinect Software
Kinect Cloud Services
What’s It For?
What Makes It Unique?
Windows for IoT
Windows IoT Software
Windows for IoT Hardware
Windows for IoT Cloud Services
What’s It For?
What Makes It Unique?
Azure IoT Device SDKs
Supported Languages and Platforms
Real-Time Operating System (RTOS)
When All Else Fails, Use the APIs
Summary

3. How to Try Before You Buy, IoT Edition
Thinking Through Your Software
The User Experience
Collecting Data
Data Collected from User Inputs
Data Collected from the Environment
Data About the Device
Data About the Software on the Device
Device Simulators
Accelerate Development
Enable Feature Development Independent of Device Development
Enable Automated Testing
Device Simulator Best Practices
A Word About Device Simulator Services
Experiment Using Virtualization
Hardware Without Dev Boards
Creating a Device for the Examples
Setting Up the Sample Device or the Device Simulator
Dependencies
Cloning the Repository
Explore the Code
Committing to a Dev Board
Summary

4. The Device Lifecycle
Device Lifecycle Management
Research and Design
Three Phases of R&D
Hardware
Software
Manufacturing
Shipping
Claiming and Provisioning
Provision Devices with an IoT Hub through the Device Provisioning Service
Main Sequence
Communication
Twinning
Device Twinning with IoT Hub
Updates
Deprovisioning
Summary

5. Device Messaging
Synchronous Versus Asynchronous Messaging
Real-Time Versus Store-and-Forward Messaging
Bidirectional Versus Unidirectional Communication
Message Formatting
Properties
Body
Common Protocols
MQ Telemetry Transport (MQTT)
Advanced Message Queuing Protocol (AMQP)
Hypertext Transfer Protocol (HTTP)
Device-to-Cloud (D2C) Messaging
Telemetry
Events
File Uploads
Message Routing
Message Enrichments
Cloud-to-Device (C2D) Messaging
Commands (Direct Methods)
General Messages
Custom Solutions
Security
Device Management
Message Routing
Scaling
Integrations with Azure
Summary

6. Life on the Edge
Why Use Edge Computing?
Better Response Time to Events on Premises
Reliable Connectivity for Critical Services
Access to Critical Services Without Bandwidth Constraints
Aggregations and Filtering Data to Reduce Network Traffic
Compute Offloading to Save Network Bandwidth
Data Localization When Dealing with Data Sovereignty and Security Issues
A Disconnected Cloud That Performs with No Network Connection
Container Basics
Azure IoT Edge
IoT Edge Modules
Message Brokering
Data
Storage
Bringing the Cloud Closer with AI
Extensibility (Bring Your Own Code)
Creating an IoT Edge Device and Deploying a Module
Install Azure IoT Edge
Deploy a Module on IoT Edge
Azure Arc and Kubernetes
IoT Edge or Arc with Kubernetes?
Setting Up Arc with MicroK8S
Set Up MicroK8S
Connect MicroK8S to Azure Arc
Install the Device Simulator to MicroK8S Using the Azure Portal
Azure Data Box Gateway
Azure Stack
Azure Stack Hub
Azure Stack HCI
Azure Stack Edge
Summary

7. Scalable Data Architecture
General Principles for Data Storage
Partition Your Data Appropriately
Storage Is Cheap, Compute Is Expensive
Prefer Eventual Consistency over Strong Consistency
Separate Reads and Writes
Treat Writes as Idempotent
Store Data in an Optimized Form for Its Intended Use
Create Retention Policies
Landing Your Data
Azure Blob Storage
Azure Data Lake
General-Purpose Blob Storage Versus Data Lake
Set Up Azure Blob Storage to Land Data
What About File Shares or File Sharing Services?
Online Transaction Processor (OLTP) Versus Online Analytics Processor (OLAP)
OLTP Solutions on Azure
Cosmos DB
Azure Data Explorer (ADX)
Azure SQL
Open Source Databases
OLAP Solutions on Azure
Azure Synapse Analytics
HDInsight
Data Lakes Versus Data Warehouses
Lakehouse
Hybrid Transaction Analytics Processor (HTAP)
OLTP as OLAP
Creating an HTAP with Cosmos DB and Azure Synapse Analytics
Summary

8. Data Processing Architectures
Data Storage (Again)
Data Processing Cycle
Data Collection
Data Preparation
Data Input
Data Processing
Output
Storage
Data Movement
Hot Path
Cold Path
Warm Path
Which Type of Data Path Should You Use?
Data Architectures
Lambda Architecture
Kappa Architecture
Delta Architecture
Which Style of Architecture Should You Use?
Summary

9. Hot Path Data Processing
Messaging Platforms
Hot Paths
Azure Stream Analytics
Azure Functions
Azure Logic Apps
Azure Service Bus
Queues and Topics
Auto-Forwarders and Subscription Filtering
Topic Actions
Performance Considerations
Create a Service Bus with a Topic and Subscriptions
Create a New Route on IoT Hub
Create Some Functions Save Data
Summary

10. Cold Path Data Processing
Azure Data Explorer
Batch Processing on Azure
Azure Batch
Create a Batch Account
Set Up the Batch Job
Azure Data Factory
Create a Data Factory to Move Data
Create a Source Dataset from Your Storage Account
Create a Data Sink for Cosmos DB
Create a Data Flow to Move Data
Create a Pipeline to Move Data
Start the Data Flow
Summary

11. The Servicing Layer
Datasets
Data Formats for Datasets
Push-Style Delivery
Cold Path Push
Hot Path Push
Pull-Style Delivery
Azure Data Share
HTTP APIs
Hybrid approaches
Summary

12. Data Consumers
Reporting Tools
Business Intelligence Tools
Power BI
Connecting Power BI to Data
Applications
External Systems Integrations
Raw Data Consumers
Security and Privacy
Security
Data Privacy
Summary

13. Monitoring and Logging
Setting Goals with Service Level Agreements
Monitoring Your Solution
Azure Monitor
Data Collection
Azure Data Explorer (Log Analytics)
Kusto Query Language (KQL)
Monitoring and Alerting
Azure Application Insights
Instrumentation
Application Logging
User Analytics
Azure Security Center
Security Assessments
Security Monitoring
Incident Response
Azure Sentinel
Summary

14. IoT Security
Software Vulnerabilities
Malware
Botnets
Ransomware
Data Leaks
DoS
Insecure Communications
Device Spoofing
Insecure Data
Lax Access Controls
Physical Threats
Lax Network Security
DNS Threats
Man-in-the-Middle
Social Engineering
Advanced Persistent Threats
Managing Threats with Microsoft Defender for IoT
Summary

15. Further Reading
Devices
Edge and Containers
Containers and GitOps
Kubernetes on the Edge
Azure IoT Edge
IoT Management
Data Architecture
Conclusion

Index
About the Author