Traffic engineering outgoing traffic
Once connections to two ISPs are operational, it is common to find that the traffic ratio between the two ISPs is suboptimal, so you may want to perform traffic engineering. Usually, there is either a lot more outgoing traffic than incoming traffic or the other way around. In networks where the majority of the traffic volume is in the outgoing direction, there is usually no need to perform traffic engineering for incoming traffic, even if incoming traffic isn’t very well-balanced. For instance, if the network has 1.2 Gbps of outgoing traffic and 150 Mbps of incoming traffic, it doesn’t really matter that 120 Mbps traffic arrives through ISP A and 30 Mbps through ISP B, as the 1.2 Gbps outgoing traffic is what determines what capacity the connections to each ISP need to be and how much each ISP will charge.
Traffic engineering outgoing traffic is a lot easier than traffic engineering incoming traffic for two reasons: the network has control over its own outgoing traffic, and there are 600,000 prefixes that can be manipulated for traffic in the outgoing direction, but possibly only a single prefix that can be manipulated in the incoming direction.
Suppose more outgoing traffic flows through ISP B than through ISP A, so we want a certain number of prefixes to be more attractive through ISP A. A rather blunt way to do this is to increase the local preference for certain paths/prefixes through ISP A. For instance, the following configuration increases the local preference for paths to or through Level 3 (AS 3356) over ISP A to 110, so traffic to those destinations will flow over ISP A. Note that in order to change outgoing traffic, we need to manipulate incoming BGP updates.
!
router bgp 64496
neighbor 10.93.194.25 remote-as 65550
neighbor 10.93.194.25 description ISP A
neighbor 10.93.194.25 route-map traffic-eng-out-ispa in
!
ip as-path access-list 10 permit _3356_
!
route-map traffic-eng-out-ispa permit 10
match as-path 10
set local-preference 110
!
route-map traffic-eng-out-ispa permit 20
!
! the permit 20 clause is needed so that prefixes
! not matched by clause permit 10 are still added
! to the BGP table
!
Manipulating the local preference is a blunt tool because now the path over ISP A will always be preferred, even if the AS path over ISP A is much longer than the AS path over ISP B. A more subtle way to perform traffic engineering is to adjust the MED, which is only considered (with always-compare-med in effect) if the AS path is the same length. On router 2, we simply set the MED to 10 for all prefixes received from ISP B:
!
router bgp 64496
bgp always-compare-med
!
! without always-compare-med MED is only compared
! between routes received from the same AS
!
neighbor 172.31.84.17 remote-as 64511
neighbor 172.31.84.17 description ISP B
neighbor 172.31.84.17 route-map traffic-eng-out-ispb in
!
route-map traffic-eng-out-ispb permit 10
set metric 10
!
On router 1, we now also set the MED to 10 on prefixes from ISP A, except on paths that carry the community 3356:2. Those get an MED of 5, which makes those paths preferred over the ones with MED 10 from ISP B.
!
router bgp 64496
bgp always-compare-med
neighbor 10.93.194.25 remote-as 65550
neighbor 10.93.194.25 description ISP A
neighbor 10.93.194.25 route-map traffic-eng-out-ispa in
!
ip bgp-community new-format
!
ip community-list 80 permit 3356:2
!
route-map traffic-eng-out-ispa permit 10
match community 80
set metric 5
!
route-map traffic-eng-out-ispa permit 20
set metric 10
!
Setting up traffic engineering takes a significant amount of time and effort. Some networks have fairly stable traffic flows so traffic engineering doesn’t have to adjusted often. Other networks have less predictable traffic patterns and need frequent traffic engineering adjustments. A good alternative to managing traffic engineering manually is to use a route optimizer product such as the Noction Intelligent Routing Platform. The Noction IRP continously monitors traffic flows and how the paths over each ISP perform and automatically reroutes outgoing traffic to take advantage of the best performing paths while staying within traffic volume commitments.