Troubleshooting BGP

<p>Foreword xxii</p> <p>Introduction xxiii</p> <p><strong>Part I BGP Fundamentals</strong></p> <p><strong>Chapter 1</strong> BGP Fundamentals 1</p> <p>Border Gateway Protocol 1</p> <p>Autonomous System Numbers 2</p> <p>Path Attributes 3</p> <p>Loop Prevention 3</p> <p>Address Families 3</p> <p>BGP Sessions 4</p> <p>Inter-Router Communication 5</p> <p>BGP Messages 6</p> <p>OPEN 6</p> <p>Hold Time 6</p> <p>BGP Identifier 7</p> <p>KEEPALIVE 7</p> <p>UPDATE 7</p> <p>NOTIFICATION Message 8</p> <p>BGP Neighbor States 8</p> <p>Idle 9</p> <p>Connect 9</p> <p>Active 10</p> <p>OpenSent 10</p> <p>OpenConfirm 10</p> <p>Established 10</p> <p>Basic BGP Configuration 11</p> <p>IOS 11</p> <p>IOS XR 12</p> <p>NX-OS 13</p> <p>Verification of BGP Sessions 14</p> <p>Prefix Advertisement 17</p> <p>BGP Best-Path Calculation 20</p> <p>Route Filtering and Manipulation 21</p> <p>IBGP 22</p> <p>IBGP Full Mesh Requirement 24</p> <p>Peering via Loopback Addresses 25</p> <p>EBGP 26</p> <p>EBGP and IBGP Topologies 28</p> <p>Next-Hop Manipulation 30</p> <p>IBGP Scalability 31</p> <p>Route Reflectors 31</p> <p>Loop Prevention in Route Reflectors 33</p> <p>Out-of-Band Route Reflectors 33</p> <p>Confederations 34</p> <p>BGP Communities 37</p> <p>Route Summarization 38</p> <p>Aggregate-Address 39</p> <p>Flexible Route Suppression 40</p> <p>Selective Prefix Suppression 40</p> <p>Leaking Suppressed Routes 40</p> <p>Atomic Aggregate 40</p> <p>Route Aggregation with AS_SET 42</p> <p>Route Aggregation with Selective Advertisement of AS-SET 42</p> <p>Default Route Advertisement 42</p> <p>Default Route Advertisement per Neighbor 42</p> <p>Remove Private AS 43</p> <p>Allow AS 43</p> <p>LocalAS 43</p> <p>Summary 44</p> <p>References 45</p> <p><strong>Part II Common BGP Troubleshooting</strong></p> <p><strong>Chapter 2</strong> Generic Troubleshooting Methodologies 47</p> <p>Identifying the Problem 47</p> <p>Understanding Variables 48</p> <p>Reproducing the Problem 49</p> <p>Setting Up the Lab 49</p> <p>Configuring Lab Devices 52</p> <p>Triggering Events 56</p> <p>Sniffer-Packet Capture 57</p> <p>SPAN on Cisco IOS 58</p> <p>SPAN on Cisco IOS XR 60</p> <p>SPAN on Cisco NX-OS 62</p> <p>Remote SPAN 63</p> <p>Platform-Specific Packet Capture Tools 65</p> <p>Netdr Capture 66</p> <p>Embedded Packet Capture 68</p> <p>Ethanalyzer 70</p> <p>Logging 74</p> <p>Event Monitoring/Tracing 77</p> <p>Summary 81</p> <p>Reference 81</p> <p><strong>Chapter 3</strong> Troubleshooting Peering Issues 83</p> <p>BGP Peering Down Issues 83</p> <p>Verifying Configuration 84</p> <p>Verifying Reachability 87</p> <p>Find the Location and Direction of Packet Loss 88</p> <p>Verify Whether Packets Are Being Transmitted 89</p> <p>Use Access Control Lists to Verify Whether Packets Are Received 90</p> <p>Check ACLs and Firewalls in Path 91</p> <p>Verify TCP Sessions 94</p> <p>Simulate a BGP Session 95</p> <p>Demystifying BGP Notifications 96</p> <p>Decode BGP Messages 99</p> <p>Troubleshoot Blocked Process in IOS XR 103</p> <p>Verify BGP and BPM Process State 104</p> <p>Verify Blocked Processes 105</p> <p>Restarting a Process 106</p> <p>BGP Traces in IOS XR 106</p> <p>BGP Traces in NX-OS 108</p> <p>Debugs for BGP 110</p> <p>Troubleshooting IPv6 Peers 112</p> <p>Case Study–Single Session Versus Multisession 113</p> <p>Multisession Capability 114</p> <p>Single-Session Capability 115</p> <p>BGP Peer Flapping Issues 115</p> <p>Bad BGP Update 115</p> <p>Hold Timer Expired 116</p> <p>Interface Issues 116</p> <p>Physical Connectivity 117</p> <p>Physical Interface 117</p> <p>Input Hold Queue 117</p> <p>TCP Receive Queue 119</p> <p>MTU Mismatch Issues 120</p> <p>High CPU Causing Control-Plane Flaps 125</p> <p>Control Plane Policing 127</p> <p>CoPP on NX-OS 129</p> <p>Local Packet Transport Services 134</p> <p>Dynamic BGP Peering 138</p> <p>Dynamic BGP Peer Configuration 139</p> <p>Dynamic BGP Challenges 142</p> <p>Misconfigured MD5 Password 142</p> <p>Resource Issues in a Scaled Environment 142</p> <p>TCP Starvation 142</p> <p>Summary 143</p> <p>References 143</p> <p><strong>Chapter 4</strong> Troubleshooting Route Advertisement and BGP Policies 145</p> <p>Troubleshooting BGP Route Advertisement 145</p> <p>Local Route Advertisement Issues 145</p> <p>Route Aggregation Issues 147</p> <p>Route Redistribution Issues 150</p> <p>BGP Tables 152</p> <p>Receiving and Viewing Routes 154</p> <p>Troubleshooting Missing BGP Routes 156</p> <p>Next-Hop Check Failures 157</p> <p>Bad Network Design 160</p> <p>Validity Check Failure 162</p> <p>AS-Path 162</p> <p>Originator-ID/Cluster-ID 165</p> <p>BGP Communities 167</p> <p>BGP Communities: No-Advertise 167</p> <p>BGP Communities: No-Export 169</p> <p>BGP Communities: Local-AS (No Export SubConfed) 170</p> <p>Mandatory EBGP Route Policy for IOS XR 172</p> <p>Filtering of Prefixes by Route Policy 173</p> <p>Conditional Matching 174</p> <p>Access Control Lists (ACL) 174</p> <p>Prefix Matching 175</p> <p>Regular Expressions (Regex) 177</p> <p>UnderScore _ 179</p> <p>Caret ^ 180</p> <p>Dollar Sign $ 181</p> <p>Brackets [ ] 181</p> <p>Hyphen - 182</p> <p>Caret in Brackets [^] 182</p> <p>Parentheses ( ) and Pipe | 183</p> <p>Period . 183</p> <p>Plus Sign + 183</p> <p>Question Mark ? 184</p> <p>Asterisk * 184</p> <p>Looking Glass and Route Servers 185</p> <p>Conditionally Matching BGP Communities 185</p> <p>Troubleshooting BGP Router Policies 185</p> <p>IOS and NX-OS Prefix-Lists 186</p> <p>IOS and NX-OS AS-Path ACLs 188</p> <p>Route-Map Processing 191</p> <p>IOS and NX-OS Route-Maps 192</p> <p>IOS XR Route-Policy Language 196</p> <p>Incomplete Configuration of Routing Policies 198</p> <p>Conditional BGP Debugs 199</p> <p>Summary 203</p> <p>Further Reading 204</p> <p>References in This Chapter 204</p> <p><strong>Chapter 5</strong> Troubleshooting BGP Convergence 205</p> <p>Understanding BGP Route Convergence 205</p> <p>BGP Update Groups 207</p> <p>BGP Update Generation 212</p> <p>Troubleshooting Convergence Issues 216</p> <p>Faster Detection of Failures 218</p> <p>Jumbo MTU for Faster Convergence 219</p> <p>Slow Convergence due to Periodic BGP Scan 219</p> <p>Slow Convergence due to Default Route in RIB 222</p> <p>BGP Next-Hop Tracking 223</p> <p>Selective Next-Hop Tracking 225</p> <p>Slow Convergence due to Advertisement Interval 226</p> <p>Computing and Installing New Path 226</p> <p>Troubleshooting BGP Convergence on IOS XR 227</p> <p>Verifying Convergence During Initial Bring Up 227</p> <p>Verifying BGP Reconvergence in Steady State Network 228</p> <p>Troubleshooting BGP Convergence on NX-OS 234</p> <p>BGP Slow Peer 237</p> <p>BGP Slow Peer Symptoms 238</p> <p>High CPU due to BGP Router Process 238</p> <p>Traffic Black Hole and Missing Prefixes in BGP table 238</p> <p>BGP Slow Peer Detection 239</p> <p>Verifying OutQ value 240</p> <p>Verifying SndWnd 240</p> <p>Verifying Cache Size and Pending Replication Messages 241</p> <p>Workaround 242</p> <p>Changing Outbound Policy 242</p> <p>Advertisement Interval 243</p> <p>BGP Slow Peer Feature 245</p> <p>Static Slow Peer 245</p> <p>Dynamic Slow Peer Detection 245</p> <p>Slow Peer Protection 246</p> <p>Slow Peer Show Commands 246</p> <p>Troubleshooting BGP Route Flapping 246</p> <p>Summary 250</p> <p>Reference 250</p> <p><strong>Part III BGP Scalability Issues</strong></p> <p><strong>Chapter 6</strong> Troubleshooting Platform Issues Due to BGP 251</p> <p>Troubleshooting High CPU Utilization due to BGP 251</p> <p>Troubleshooting High CPU due to BGP on Cisco IOS 252</p> <p>High CPU due to BGP Scanner Process 253</p> <p>High CPU due to BGP Router Process 255</p> <p>High CPU Utilization due to BGP I/O Process 256</p> <p>Troubleshooting High CPU due to BGP on IOS XR 258</p> <p>Troubleshooting High CPU due to BGP on NX-OS 262</p> <p>Capturing CPU History 265</p> <p>Troubleshooting Sporadic High CPU Condition 265</p> <p>Troubleshooting Memory Issues due to BGP 267</p> <p>TCAM Memory 269</p> <p>Troubleshooting Memory Issues on Cisco IOS Software 269</p> <p>Troubleshooting Memory Issues on IOS XR 274</p> <p>Troubleshooting Memory Issues on NX-OS 278</p> <p>Restarting Process 281</p> <p>Summary 281</p> <p>References 282</p> <p><strong>Chapter 7</strong> Scaling BGP 283</p> <p>The Impact of Growing Internet Routing Tables 283</p> <p>Scaling Internet Table on Various Cisco Platforms 285</p> <p>Scaling BGP Functions 288</p> <p>Tuning BGP Memory 290</p> <p>Prefixes 290</p> <p>Managing the Internet Routing Table 290</p> <p>Paths 292</p> <p>Attributes 293</p> <p>Tuning BGP CPU 295</p> <p>IOS Peer-Groups 295</p> <p>IOS XR BGP Templates 295</p> <p>NX-OS BGP Peer Templates 296</p> <p>BGP Peer Templates on Cisco IOS 297</p> <p>Soft Reconfiguration Inbound Versus Route Refresh 298</p> <p>Dynamic Refresh Update Group 302</p> <p>Enhanced Route Refresh Capability 305</p> <p>Outbound Route Filtering (ORF) 309</p> <p>Prefix-Based ORF 309</p> <p>Extended Community—Based ORF 309</p> <p>BGP ORF Format 310</p> <p>BGP ORF Configuration Example 312</p> <p>Maximum Prefixes 316</p> <p>BGP Max AS 318</p> <p>BGP Maximum Neighbors 322</p> <p>Scaling BGP with Route Reflectors 322</p> <p>BGP Route Reflector Clusters 324</p> <p>Hierarchical Route Reflectors 331</p> <p>Partitioned Route Reflectors 332</p> <p>BGP Selective Route Download 339</p> <p>Virtual Route Reflectors 342</p> <p>BGP Diverse Path 346</p> <p>Shadow Route Reflectors 349</p> <p>Shadow Sessions 355</p> <p>Route Servers 357</p> <p>Summary 364</p> <p>References 365</p> <p><strong>Chapter 8</strong> Troubleshooting BGP Edge Architectures 367</p> <p>BGP Multihoming and Multipath 367</p> <p>Resiliency in Service Providers 370</p> <p>EBGP and IBGP Multipath Configuration 370</p> <p>EIBGP Multipath 372</p> <p>R1 373</p> <p>R2 374</p> <p>R3 374</p> <p>R4 375</p> <p>R5 376</p> <p>AS-Path Relax 377</p> <p>Understanding BGP Path Selection 377</p> <p>Routing Path Selection Longest Match 377</p> <p>BGP Best-Path Overview 379</p> <p>Weight 380</p> <p>Local Preference 380</p> <p>Locally Originated via Network or Aggregate Advertisement 380</p> <p>Accumulated Interior Gateway Protocol (AIGP) 381</p> <p>Shortest AS-Path 383</p> <p>Origin Type 383</p> <p>Multi-Exit Discriminator (MED) 384</p> <p>EBGP over IBGP 386</p> <p>Lowest IGP Metric 386</p> <p>Prefer the Oldest EBGP Path 387</p> <p>Router ID 387</p> <p>Minimum Cluster List Length 388</p> <p>Lowest Neighbor Address 388</p> <p>Troubleshooting BGP Best Path 389</p> <p>Visualizing the Topology 390</p> <p>Phase I–Initial BGP Edge Route Processing 391</p> <p>Phase II–BGP Edge Evaluation of Multiple Paths 392</p> <p>Phase III–Final BGP Processing State 394</p> <p>Path Selection for the Routing Table 394</p> <p>Common Issues with BGP Multihoming 395</p> <p>Transit Routing 395</p> <p>Problems with Race Conditions 397</p> <p>Peering on Cross-Link 402</p> <p>Expected Behavior 403</p> <p>Unexpected Behavior 406</p> <p>Secondary Verification Methods of a Routing Loop 409</p> <p>Design Enhancements 411</p> <p>Full Mesh with IBGP 412</p> <p>Problems with Redistributing BGP into an IGP 413</p> <p>Summary 417</p> <p>References 418</p> <p><strong>Part IV Securing BGP</strong></p> <p><strong>Chapter 9</strong> Securing BGP 419</p> <p>The Need for Securing BGP 419</p> <p>Securing BGP Sessions 420</p> <p>Explicitly Configured Peers 421</p> <p>IPv6 BGP Peering Using Link-Local Address 421</p> <p>BGP Session Authentication 424</p> <p>BGP Pass Through 426</p> <p>EBGP-Multihop 427</p> <p>BGP TTL Security 428</p> <p>Filtering 429</p> <p>Protecting BGP Traffic Using IPsec 431</p> <p>Securing Interdomain Routing 431</p> <p>BGP Prefix Hijacking 432</p> <p>S-BGP 439</p> <p>IPsec 439</p> <p>Public Key Infrastructure 439</p> <p>Attestations 441</p> <p>soBGP 442</p> <p>Entity Certificate 442</p> <p>Authorization Certificate 443</p> <p>Policy Certificate 443</p> <p>BGP SECURITY Message 443</p> <p>BGP Origin AS Validation 443</p> <p>Route Origination Authorization (ROA) 445</p> <p>RPKI Prefix Validation Process 446</p> <p>Configuring and Verifying RPKI 449</p> <p>RPKI Best-Path Calculation 460</p> <p>BGP Remote Triggered Black-Hole Filtering 463</p> <p>BGP Flowspec 467</p> <p>Configuring BGP Flowspec 469</p> <p>Summary 479</p> <p>References 480</p> <p><strong>Part V Multiprotocol BGP</strong></p> <p><strong>Chapter 10</strong> MPLS Layer 3 VPN (L3VPN) 481</p> <p>MPLS VPNs 481</p> <p>MPLS Layer 3 VPN (L3VPN) Overview 483</p> <p>Virtual Routing and Forwarding 483</p> <p>Route Distinguisher 485</p> <p>Route Target 485</p> <p>Multi-Protocol BGP (MP-BGP) 486</p> <p>Network Advertisement Between PE and CE Routers 487</p> <p>MPLS Layer 3 VPN Configuration 487</p> <p>VRF Creation and Association 488</p> <p>IOS VRF Creation 488</p> <p>IOS XR VRF Creation 489</p> <p>NX-OS VRF Creation 490</p> <p>Verification of VRF Settings and Connectivity 492</p> <p>Viewing VRF Settings and Interface IP Addresses 492</p> <p>Viewing the VRF Routing Table 494</p> <p>VRF Connectivity Testing Tools 495</p> <p>MPLS Forwarding 495</p> <p>BGP Configuration for VPNv4 and PE-CE Prefixes 497</p> <p>IOS BGP Configuration for MPLS L3VPN 497</p> <p>IOS XR BGP Configuration for MPLS L3VPN 499</p> <p>NX-OS BGP Configuration for MPLS L3VPN 500</p> <p>Verification of BGP Sessions and Routes 502</p> <p>Troubleshooting MPLS L3VPN 506</p> <p>Default Route Advertisement Between PE-CE Routers 508</p> <p>Problems with AS-PATH 509</p> <p>Suboptimal Routing with VPNv4 Route Reflectors 514</p> <p>Troubleshooting Problems with Route Targets 520</p> <p>MPLS L3VPN Services 524</p> <p>RT Constraints 534</p> <p>MPLS VPN Label Exchange 538</p> <p>MPLS Forwarding 541</p> <p>Summary 542</p> <p>References 542</p> <p><strong>Chapter 11</strong> BGP for MPLS L2VPN Services 543</p> <p>L2VPN Services 543</p> <p>Terminologies 545</p> <p>Virtual Private Wire Service 548</p> <p>Interworking 549</p> <p>Configuration and Verification 550</p> <p>VPWS BGP Signaling 558</p> <p>Configuration 560</p> <p>Virtual Private LAN Service 561</p> <p>Configuration 562</p> <p>Verification 564</p> <p>VPLS Autodiscovery Using BGP 569</p> <p>VPLS BGP Signaling 580</p> <p>Troubleshooting 586</p> <p>Summary 588</p> <p>References 589</p> <p><strong>Chapter 12</strong> IPv6 BGP for Service Providers 591</p> <p>IPv6 BGP Features and Concepts 591</p> <p>IPv6 BGP Next-Hop 591</p> <p>IPv6 Reachability over IPv4 Transport 596</p> <p>IPv4 Routes over IPv6 Next-Hop 601</p> <p>IPv6 BGP Policy Accounting 604</p> <p>IPv6 Provider Edge Routers (6PE) over MPLS 607</p> <p>6PE Configuration 611</p> <p>6PE Verification and Troubleshooting 615</p> <p>IPv6 VPN Provider Edge (6VPE) 620</p> <p>IPv6-Aware VRF 622</p> <p>6VPE Next-Hop 623</p> <p>Route Target 624</p> <p>6VPE Control Plane 624</p> <p>6VPE Data Plane 626</p> <p>6VPE Configuration 627</p> <p>6VPE Control-Plane Verification 629</p> <p>6VPE Data Plane Verification 633</p> <p>Summary 639</p> <p>References 639</p> <p><strong>Chapter 13</strong> VxLAN BGP EVPN 641&lt;/p</p>