Understanding BGP Communities

After reading the title of this article you may be thinking of small neighborhoods in the world’s most-connected cities where BGP-minded people live together, or maybe online places where BGP connoisseurs hang out, such as the North American Network Operators Group (NANOG) mailinglist. But what I want to talk about today is the BGP communities attribute. In 1996, a few years after the BGP-4 specification was published, RFC 1997 added the communities attribute to BGP.

What are BGP Communities?

Communities are basically labels that are attached to BGP routes. A few of these labels have pre-defined meanings. The well-known communities are:

• NO_EXPORT (0xFFFFFF01)
• NO_ADVERTISE (0xFFFFFF02)
• NO_EXPORT_SUBCONFED (0xFFFFFF03)
• NOPEER (0xFFFFFF04)

The NO_EXPORT community tells a router it should only propagate any prefixes this BGP community is attached to over iBGP, and not propagate it over eBGP to external autonomous systems. (NO_EXPORT_SUBCONFED does something similar in networks using confederations to limit the number of iBGP sessions.) NO_ADVERTISE goes a step further and tells the router to not advertise the prefix over BGP at all. Most, if not all, routers automatically honor these communities when they’re present. So if you want to overrule this behavior, you need to filter them out. NOPEER was defined later and indicates that a prefix “need not” be advertised over peering relationships.

With the well-known communities out of the way, let’s look at how user-defined communities work. First of all, many routers don’t automatically propagate communities. On a Cisco router, you’ll have to enable this explicitly for a BGP neighbor with the “send-community” keyword:

 

Communities are 32-bit values. By default, they show up as decimal numbers that can get pretty unwieldy. The “ip bgp-community new-format” line changes the display format of communities into two 16-bit values separated by a colon. (The well-known communities show up as “no-export” for NO_EXPORT et cetera.) The first 16-bit number is normally the AS number of the network that sets the community or looks for it, and the second number is one that conveys the intended information. For instance, Level3 (AS3356) tags all the prefixes it learns in Pittsburgh with community 3356:2040.

So if you had a reason to avoid paths that go through Level3’s network in Pittsburgh, you could look for 3356:2040 and give prefixes that have this community applied to them a lower local preference. That way, if your router has additional paths towards the prefix in question, it’ll use one of the alternatives. The following configuration accomplishes this:

 

The first thing that we do is apply route map peer2 to inbound updates from neighbor 10.0.0.2. This route map has two clauses: the permit 10 and the permit 20 part. 10 and 20 are basically line numbers, which allow you to (for example) add a clause 15 if you need to fit in something between 10 and 20. The permit 10 clause refers to the community access list 80, which permits all prefixes that are labeled with community 3356:2040. So if that community is present, the “match” part of the permit 10 clause is fulfilled, so the “set” part is executed, and the local preference for the prefix in question is set to 80. (Which is lower, and thus less preferred, than the default value of 100.) At this point, the matching prefix is added to the BGP table and no further processing is done for that prefix.

However, prefixes that don’t have community 3356:2040 and thus don’t match community list 80 pass through the permit 10 clause without any action. The permit 20 clause then catches them, because without a “match” statement, all prefixes match. Then, no action is performed, but because of the implicit match, the prefixes that get this far are added to the BGP table. Without the permit 20 clause, any prefixes not matched by the permit 10 clause would be filtered out because of Cisco’s implicit deny system: everything that isn’t explicitly allowed by the time we reach the end of a filter or route map is denied (and filtered out).

Now maybe avoiding paths through Level3’s Pittsburgh point-of-presence is not high on your list of priorities. However, there are many more things that can be accomplished with communities. Many large networks publish a list of communities that they set and honor, often in the routing registries. Like Level3 in the RIPE database. (If you’re interested in observing some communities in the wild, log into route-views.routeviews.org with telnet and type something like “show ip bgp 24.56.100.0”.)

I’ll discuss one more use of communities: setting up an active/standby scenario. Normally, if you have two connections to the internet, both of those will be used for both incoming and outgoing traffic. However, under some circumstances, it can be beneficial to have one connection handle all traffic and have the second one be just a backup that remains idle until the primary connection fails. For outgoing traffic, this is easy enough: lower the local preference for all prefixes learned over the backup line. However, for incoming traffic, the local preference needs to be lowered in the neighboring AS. Turns out, Level3 has three communities that do exactly that:

• 3356:70 – set local preference to 70
• 3356:80 – set local preference to 80
• 3356:90 – set local preference to 90

Unfortunately, there is no information on the local preference values Level3 uses for customer routes and peer routes. The former are always higher than the latter, and I’ll assume that peer routes normally have a local preference of over 70, so if we set our prefixes to 70, those prefixes have a preference lower than peer routes, so Level3 will actually prefer to reach us through our other ISP over a peer route rather than directly over our backup line. This is the configuration that accomplishes this:

 

We don’t apply any special settings to the BGP sessions towards the primary ISP, so the local preference learned over that BGP session will be 100. On the session to Level3, the backup ISP, we apply two route maps: one for incoming updates and one for outgoing updates. The peer2-in route map sets the local preference for all prefixes learned from Level3 to 70. So when given the choice, the router will send outgoing traffic through the primary ISP. Only when a destination is reachable through Level3 but not through the primary ISP will the traffic be sent over the backup link.

The peer2-out route map sets the community 3356:70 on the prefixes we send to Level3, which will make Level3’s routers lower the local preference within the Level3 network to 70 for those prefixes, so the backup configuration also applies to incoming traffic.

Have a look at some of the other communities Level3 (or your own ISP) makes available, such as the blackhole community that can be useful if there’s a denial of service attack towards some of your addresses. By having those addresses blackholed through the use of a blackhole community, at least the rest of your network will remain reachable. There are also communities for selectively applying AS path prepending, which can help with traffic engineering. Nevertheless, pay attention to The dark side of BGP community attributes as well.