Prompt Engineering for Generative AI

Preface
Software Requirements for This Book
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments

1. The Five Principles of Prompting
Overview of the Five Principles of Prompting
1. Give Direction
2. Specify Format
3. Provide Examples
4. Evaluate Quality
5. Divide Labor
Summary

2. Introduction to Large Language Models for Text Generation
What Are Text Generation Models?
Vector Representations: The Numerical Essence of Language
Transformer Architecture: Orchestrating Contextual Relationships
Probabilistic Text Generation: The Decision Mechanism
Historical Underpinnings: The Rise of Transformer Architectures
OpenAI’s Generative Pretrained Transformers
GPT-3.5-turbo and ChatGPT
GPT-4
Google’s Gemini
Meta’s Llama and Open Source
Leveraging Quantization and LoRA
Mistral
Anthropic: Claude
GPT-4V(ision)
Model Comparison
Summary

3. Standard Practices for Text Generation with ChatGPT
Generating Lists
Hierarchical List Generation
When to Avoid Using Regular Expressions
Generating JSON
YAML
Filtering YAML Payloads
Handling Invalid Payloads in YAML
Diverse Format Generation with ChatGPT
Mock CSV Data
Explain It like I’m Five
Universal Translation Through LLMs
Ask for Context
Text Style Unbundling
Identifying the Desired Textual Features
Generating New Content with the Extracted Features
Extracting Specific Textual Features with LLMs
Summarization
Summarizing Given Context Window Limitations
Chunking Text
Benefits of Chunking Text
Scenarios for Chunking Text
Poor Chunking Example
Chunking Strategies
Sentence Detection Using SpaCy
Building a Simple Chunking Algorithm in Python
Sliding Window Chunking
Text Chunking Packages
Text Chunking with Tiktoken
Encodings
Understanding the Tokenization of Strings
Estimating Token Usage for Chat API Calls
Sentiment Analysis
Techniques for Improving Sentiment Analysis
Limitations and Challenges in Sentiment Analysis
Least to Most
Planning the Architecture
Coding Individual Functions
Adding Tests
Benefits of the Least to Most Technique
Challenges with the Least to Most Technique
Role Prompting
Benefits of Role Prompting
Challenges of Role Prompting
When to Use Role Prompting
GPT Prompting Tactics
Avoiding Hallucinations with Reference
Give GPTs “Thinking Time”
The Inner Monologue Tactic
Self-Eval LLM Responses
Classification with LLMs
Building a Classification Model
Majority Vote for Classification
Criteria Evaluation
Meta Prompting
Summary

4. Advanced Techniques for Text Generation with LangChain
Introduction to LangChain
Environment Setup
Chat Models
Streaming Chat Models
Creating Multiple LLM Generations
LangChain Prompt Templates
LangChain Expression Language (LCEL)
Using PromptTemplate with Chat Models
Output Parsers
LangChain Evals
OpenAI Function Calling
Parallel Function Calling
Function Calling in LangChain
Extracting Data with LangChain
Query Planning
Creating Few-Shot Prompt Templates
Fixed-Length Few-Shot Examples
Formatting the Examples
Selecting Few-Shot Examples by Length
Limitations with Few-Shot Examples
Saving and Loading LLM Prompts
Data Connection
Document Loaders
Text Splitters
Text Splitting by Length and Token Size
Text Splitting with Recursive Character Splitting
Task Decomposition
Prompt Chaining
Sequential Chain
itemgetter and Dictionary Key Extraction
Structuring LCEL Chains
Document Chains
Stuff
Refine
Map Reduce
Map Re-rank
Summary

5. Vector Databases with FAISS and Pinecone
Retrieval Augmented Generation (RAG)
Introducing Embeddings
Document Loading
Memory Retrieval with FAISS
RAG with LangChain
Hosted Vector Databases with Pinecone
Self-Querying
Alternative Retrieval Mechanisms
Summary

6. Autonomous Agents with Memory and Tools
Chain-of-Thought
Agents
Reason and Act (ReAct)
Reason and Act Implementation
Using Tools
Using LLMs as an API (OpenAI Functions)
Comparing OpenAI Functions and ReAct
Use Cases for OpenAI Functions
ReAct
Use Cases for ReAct
Agent Toolkits
Customizing Standard Agents
Custom Agents in LCEL
Understanding and Using Memory
Long-Term Memory
Short-Term Memory
Short-Term Memory in QA Conversation Agents
Memory in LangChain
Preserving the State
Querying the State
ConversationBufferMemory
Other Popular Memory Types in LangChain
ConversationBufferWindowMemory
ConversationSummaryMemory
ConversationSummaryBufferMemory
ConversationTokenBufferMemory
OpenAI Functions Agent with Memory
Advanced Agent Frameworks
Plan-and-Execute Agents
Tree of Thoughts
Callbacks
Global (Constructor) Callbacks
Request-Specific Callbacks
The Verbose Argument
When to Use Which?
Token Counting with LangChain
Summary

7. Introduction to Diffusion Models for Image Generation
OpenAI DALL-E
Midjourney
Stable Diffusion
Google Gemini
Text to Video
Model Comparison
Summary

8. Standard Practices for Image Generation with Midjourney
Format Modifiers
Art Style Modifiers
Reverse Engineering Prompts
Quality Boosters
Negative Prompts
Weighted Terms
Prompting with an Image
Inpainting
Outpainting
Consistent Characters
Prompt Rewriting
Meme Unbundling
Meme Mapping
Prompt Analysis
Summary

9. Advanced Techniques for Image Generation with Stable Diffusion
Running Stable Diffusion
AUTOMATIC1111 Web User Interface
Img2Img
Upscaling Images
Interrogate CLIP
SD Inpainting and Outpainting
ControlNet
Segment Anything Model (SAM)
DreamBooth Fine-Tuning
Stable Diffusion XL Refiner
Summary

10. Building AI-Powered Applications
AI Blog Writing
Topic Research
Expert Interview
Generate Outline
Text Generation
Writing Style
Title Optimization
AI Blog Images
User Interface
Summary

Index
About the Authors