Full iBGP Mesh and Route Reflection
To avoid loops in a Full iBGP Mesh, BGP routers are only allowed to learn prefixes over iBGP from the router that advertises them or, alternatively, learn them via eBGP. In such cases, all BGP speakers within a single AS must be fully meshed so that any external routing information is re-distributed to all routers within that Autonomous System (AS). This becomes a serious scaling problem when there is a large number of IBGP speakers exchanging a large volume of routing information.
Route Reflection, described in RFC 4456, is a solution to the iBGP scaling problem described above. When the Route Reflector (RR) receives a route from an iBGP peer it selects the best-path for that route. If RR receives a route from a non-client iBGP peer, it reflects it to all of its clients. When RR receives a route from its client it reflects it to all client and non-client peers.
RR uses the same path selection process as normal iBGP speakers do. The path whose next hop is resolved through the iBGP route with the lowest metric is preferred. This ensures that traffic exits a network at the lowest cost via the shortest path – the so called “hot potato” routing, as traffic is directed to the best AS exit point.
Suboptimal Path Reflection
Suboptimal Path Reflection occurs when a RR is not topologically near to the clients. RR performs the best path selection based on its proximity in IGP but not the proximity of its clients. Let’s take a look at the following topology, with a router reflector (RR) in AS6500 and its clients R1, R2 R3 and R4 (Picture 1).
Picture 1: Network Topology
The RR learns the route 188.8.131.52/24 from ISP1 (AS6501) through R1 and from ISP2 through R2 (Picture 2).
RR# show ip bgp | begin Network
Picture 2: BGP Table of Router RR
The path to 184.108.40.206/24 through R1 (220.127.116.11) is preferred to the path via R2 (18.104.22.168) because the IGP metric 2 is lower than 4 (Picture 3). Therefore, the RR router installs the 22.214.171.124/24 route via R1 into its routing table. As the BGP advertises only the best path, the path via R2 is not being advertised to the clients.
RR# show ip bgp detail
Picture 3: Inspecting BGP Table of RR for Route 126.96.36.199/24
|Note: The RFC7911 describes a BGP extension that allows the advertisement of multiple paths (ADD-PATH) for the same prefix (NLRI). We have covered the BGP ADD-PATH topic in a separate article.|
It is important to mention here that although a path to a certain prefix is optimal for the Route Reflector based on the lowest IGP metric criteria, this might not be true from the client’s perspective. For instance, the R3 router has installed a path to 188.8.131.52/24 via R1 received from RR (Picture 4). However, it is obvious that a path via R2 would be definitely shorter thus truly optimal if received.
RP/0/0/CPU0:R3# show bgp | begin Network
Picture 4: BGP Table of R3
The solution to this problem might be hierarchical RRs placed in close proximity to clients, so that a best path advertised by RR would represent the best path for clients as well. Another solution is to configure RR to advertise multiple paths (ADD_PATH) to clients for the same prefix so that they can select the best path based on its own perspective. However, pushing all external paths from RR to clients may may be a challenge as the client’s resources (CPU, RAM) can be depleted by a larger BGP table size.
BGP Optimal Route Reflection
BGP Optimal Route Reflection is defined in an I-E Draft. Instead of sending the best-path based on RR point of view, optimal BGP path selection is based on a client’s perspective. The RR sends a specific best path to a specific BGP border router that is calculated based on the position of the router in the topology. Therefore, the RR sends a different best path to different BGP border routers. For the BGP Optimal Route Reflection to work properly, the RR must have a complete view of the network topology from the IGP perspective. Hence, network must run a link-state routing protocol (OSPF or IS-IS). Also, the border routers must be RR clients. The objective is that each ingress BGP border router can have a different exit or egress BGP router for the same prefix. If the ingress border router can always forward the traffic to the closes AS-exit router, then a requirement of the hot-potato routing is fulfilled.
The RR runs a Shortest Path First (SPF) calculation with the client as the root of the tree and then calculates the cost to the BGP next-hop. The IGP cost for use of BGP best path selection is stored and the route with the lowest IGP metric is chosen and advertised to a particular client. The SPF calculation is independent of BGP and runs only if ORR is enabled on the RR.
As for Cisco family products, Optimal Route Reflection (ORR) has been supported only by IOS-XR, since version 5.3.1. The OSPF, IS-IS and BGP-LS can be used to acquire IGP topology information.
Now let’s discuss the configuration of the RR and R3 routers. RR is configured as a Route Reflector with enabled BGP-ORR. The R3 router is a client of the RR. Both routers are running IOS-XR, version 6.1.8.
Router RR Configuration
interface Loopback0 ipv4 address 184.108.40.206 255.255.255.255 interface GigabitEthernet0/0/0/0 description Link to R1 ipv4 address 10.0.0.2 255.255.255.252 interface GigabitEthernet0/0/0/1 description Link to R4 ipv4 address 10.0.0.5 255.255.255.252
Configure OSPF as an IGP used in AS6500. Redistribute OSPF information into BGP with distribute bgp-ls command.
router ospf 1 distribute bgp-ls router-id 220.127.116.11 address-family ipv4 unicast area 0 mpls traffic-eng interface Loopback0 network point-to-point interface GigabitEthernet0/0/0/0 network point-to-point interface GigabitEthernet0/0/0/1 network point-to-point
Define a primary root device for each of ORR group that we want to use to compute the IGP cost. The secondary and tertiary root device can be configured as well. Now enable optimal-route-reflection for each neighbor.
router bgp 6500 bgp router-id 18.104.22.168 address-family ipv4 unicast optimal-route-reflection r1 22.214.171.124 optimal-route-reflection r2 126.96.36.199 optimal-route-reflection r3 188.8.131.52 optimal-route-reflection r4 184.108.40.206 neighbor 220.127.116.11 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r1 route-reflector-client neighbor 18.104.22.168 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r2 route-reflector-client neighbor 22.214.171.124 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r3 route-reflector-client neighbor 126.96.36.199 remote-as 6500 update-source Loopback0 address-family ipv4 unicast optimal-route-reflection r4 route-reflector-client mpls traffic-eng
To check ORR SPF database for the group 3, issue the command below (Picture 5). The actual Root for SPF is 188.8.131.52. The OSPF cost to the iBGP next-hops is computed from the R3’s perspective. The cost 4 is computed for the neighbor 184.108.40.206 (R1) and the cost 2 for 220.127.116.11 (R2).
RP/0/0/CPU0:RR# show orrspf database r3
Picture 5: Router RR ORR SPF Database for Group 1
RP/0/0/CPU0:R3# show route ospf
Picture 6: R3 OSPF Routes
Router R3 Configuration
interface Loopback0 ipv4 address 18.104.22.168 255.255.255.255 interface GigabitEthernet0/0/0/0 ipv4 address 10.0.0.10 255.255.255.252 interface GigabitEthernet0/0/0/1 ipv4 address 10.0.0.13 255.255.255.252 router ospf 1 distribute bgp-ls router-id 22.214.171.124 address-family ipv4 unicast area 0 mpls traffic-eng interface Loopback0 network point-to-point interface GigabitEthernet0/0/0/0 network point-to-point interface GigabitEthernet0/0/0/1 network point-to-point router bgp 6500 bgp router-id 126.96.36.199 address-family ipv4 unicast neighbor 188.8.131.52 remote-as 6500 update-source Loopback0 address-family ipv4 unicast mpls traffic-eng
Finally, we will inspect the BGP table of R3 to check whether a suboptimal path to the prefix 184.108.40.206/24 via the BGP next-hop 220.127.116.11 (R1) is changed to the the shorter path via 18.104.22.168 (R2).
RP/0/0/CPU0:R3# show bgp
Picture 7: R3 BGP Table
The router R3 has installed a path to the prefix 22.214.171.124/24 via router R2 (BGP next-hop 126.96.36.199).
BGP Optimal Route Reflection is a new routing technology that still exists as an I-E draft. ORR should be tested in networks with 10k IGP nodes to check how it converges in such networks, as each next-hop change triggers SPF calculations. Instead of running ORR on hardware routers it should be run on a virtual Route Reflector with enough CPU and RAM. Nevertheless, the concept has been implemented by major network vendors such as Cisco, Juniper and Nokia already. It will be interesting to watch how it progresses.
Boost BGP Preformance
Automate BGP Routing optimization with Noction IRP