Top Volume Autonomous Systems

3 Using IRP

3.1 Getting started

The Frontend is a web interface with a comprehensive set of reports, graphs and diagnostic information, which can reflect the current and historical network state, as well as the benefits of IRP network optimization.

3.1.1 Accessing the system

The system frontend is accessible over HTTPS. IRP’s Frontend can be accessed using any major browser, via the main IRP server IP address, or FQDN. E.g: http://irp-system.noction.com/. When opening the Frontend’s URL, a login form will pop up:

figure screenshots/frontend-login-form.png

Figure 3.1.1 Frontend login form

The username and password provided by the system administrator must be used. The “Remember me” check-box can be used as well.

3.1.2 System dashboard

After a successful login, the default Dashboard will become available. It contains a standard set of gadgets - elements, which can contain a reduced version of a report or a graph.

figure screenshots/frontend-default-dashboard.png

Figure 3.1.2 Default Dashboard

3.2 Frontend structure

The Frontend has several components that operate together and allow a regular user or administrator to access various reports, graphs, or configuration pages.

The menu bar (see Figure 3.3↓) allows quick access to any part of the system via its menu items.

Figure 3.2.1 Menu bar
The sidebar (See section 3.2.1↓) is present on the Dashboard or on any graph or report. Its contents depend on the current page, and can include filters, export functions, etc., as explained in the next section.
The status bar (see Figure 3.4↓), allows one to access the global search function (Section 3.2.3↓), as well as system events and notifications (See the Events↓ section).

Figure 3.2.2 Status bar

3.2.1 Sidebar

All dashboards, reports, graphs and events have a sidebar, located at the left side of the screen. Depending on the current page, the sidebar has a different set of controls, allowing users to manage the dashboard components or to filter the report and graph results.

Every sidebar contains Export, Email and Print buttons. Reports can be exported in .xls, .csv, .pdf formats, while the graphs and dashboard - only as .pdf .

Figure 3.2.3 Common sidebar buttons

The “Email” button allows scheduling email delivery of several reports or graphs, or even an entire dashboard (Figure 3.6↓). Multiple destination email addresses can be used. Most elements in this form have informative tool-tips, that can be seen when hovering over a specific control. Note that Email Reports can be removed from <Username> → Emailing from the top panel.

figure screenshots/sidebar-emailing-setup.png

Figure 3.2.4 Email schedule configuration

In the case of a report, custom filters can be applied, so that only specific information will be displayed in the report.

Figure 3.2.5 Report filters

On graphs, reports and dashboard, the time period which the information is being provided for, can be selected by using the “Timestamp” button. An intuitive control will open, and the desired time period can be selected.

figure screenshots/sidebar-timestamp-selector.png

Figure 3.2.6 Time period selection

The number of results can be adjusted on a report page, using the “Results per page” slider.

Figure 3.2.7 Adjusting the number of results per page

Dashboards have a different sidebar layout.

The Delete button cannot be found in the default dashboard. The default dashboard cannot be deleted.

System dashboard represents the overall system information which includes:

Configured maximum number of improvements (Max IPv6 Improvements will be shown if IPv6 is enabled)
Number of Routing Policies improvements
Number of improvements performed today
The amount of traffic improved today
Percentage of prefixes improved today
Number of operating iBGP sessions
Number of announcements to the edge router
Announcements to each from configured routers
Number of BGP sessions with the providers
Number of operating BGP sessions with the providers

figure screenshots/sidebar-dashboard-buttons.png

Figure 3.2.8 Dashboard buttons

3.2.2 Failover status

When failover is configured and operational the status of the components is highlighted below system dashboard for each master and slave nodes.

A valid failover license is required

Figure 3.2.9 Failover status of master node

Slave node status shall be considered together with component status in the dashboard. During normal operation components Core, Explorer and Irppushd are stopped.

figure screenshots/FailoverStatus-Slave.png

Figure 3.2.10 Failover status of slave node

3.2.3 Global search

The Frontend allows to search for historical information on optimized prefixes, AS Numbers and AS Names. The search function can be accessed using the Search box in the status bar.

Figure 3.2.11 Search box

Enter the needed prefix, AS number or AS name in the search box, then press Enter or click on the magnifying glass button.

While entering the search terms, results are dynamically loaded in a modal window (Figure 3.14↓).

figure screenshots/statusbar-dynamical-search-results.png

Figure 3.2.12 Dynamical global search results

When displaying the complete results, specific prefixes can be manually removed, or scheduled for re-probing, by using the buttons next to each result (Figure 3.15↓).

figure screenshots/global-search-results.png

Figure 3.2.13 Global search results

The Show all X results in Y category buttons can also be used, to display all matching results from the selected category (Improvements or AS Path problems)

3.2.4 Warning bars

The warning bars are shown in case the attention should be drawn to operational/performance issues, some specific IRP configuration settings or license/payment issues.

Figure 3.2.14 Precedence and improve in a group warning

Figure 3.2.15 Internal Monitor warning

Figure 3.2.16 Explorer performance warning

Figure 3.2.17 License expiration warning

3.3 Customizing the dashboard

The Default dashboard is the first screen that can be seen after a successful login to the system. It contains a default set of gadgets, that can be customized by adding additional gadgets, removing unnecessary gadgets, or modifying specific gadget’s settings.

3.3.1 Custom dashboard gadgets

3.3.1.1 Platform status

figure screenshots/widget-platform-status.png

Figure 3.3.1 Platform status

3.3.1.2 Interfaces status

figure screenshots/widget-interfaces-status.png

Figure 3.3.2 Interfaces status

3.3.1.3 List of improved prefixes

figure screenshots/widget-list-of-improved-prefixes.png

Figure 3.3.3 List of improved prefixes

3.3.1.4 AS-Path problems

figure screenshots/widget-as-path-problems.png

Figure 3.3.4 AS-Path problems

3.4 Reports

The IRP Frontend comes with a comprehensive set of reports, reflecting the current state of the network as well as overall statistics on the system performance. As explained in the Sidebar↑ section, the results can be filtered by specific criteria for each report, and the time period of the reported data can also be selected.

figure screenshots/frontend-reports-menu.png

Figure 3.4.1 Reports menu

3.4.1 Platform overview

The “Platform Overview” report provides information on the overall rate of improvements based upon latency, loss and cost. This report can be used for a quick overview of the overall system performance and improvements.

One can see here that almost all the improved routes had a loss drop of 20% or more. For more than 80% of the improved routes the loss was fully eliminated. There are also 7% of the improved routes, which were redirected from a complete blackout. The report also provides the amount of the accomplished route improvements made, based upon latency, and cost (if the Cost optimization is enabled, see Cost optimization↑).

figure screenshots/report-1-platform-overview.png

Figure 3.4.2 Platform overview

3.4.2 Platform performance

The “Platform Performance” report offers overall statistics on improvements made, based upon performance reasons, commit control and cost reasons on a daily, weekly and monthly basis.

figure screenshots/report-2-platform-performance.png

Figure 3.4.3 Platform performance

3.4.3 Loss Improvements

The Loss Improvements report highlights how much of the "Packet loss problem" IRP detected and was able to solve during a specific timeperiod.

Loss relative rates - highlights the percentual magnitude of the improvement in loss rates for groups of destinations before and after IRP assessed and re-routed the traffic. The groups represent:

All destinations that cover good and problematic destinations that registered traffic flowing to during the timeperiod;
Problem destinations that includes only routes with packet loss and improved by IRP;
Loss eliminated represents those problem destinations that IRP was able to improve to a better route with zero loss;
Loss reduced represents those problem destinations that IRP improved but was not able to reduce loss altogether.

figure screenshots/report-13-loss-improvements-relative-rates.png

Figure 3.4.4 Loss relative rates

Loss absolute rates - highlights the actual loss rate values before and after improvement for the same groups of destinations as above. While the previous part of the report shows big numbers one needs to know how big the packet loss problem actually is. The Loss absolute rates part of this report shows just that. It displays an actual 10% or 5% or 3% or 1% value depending on how well-behaved the network is. Values of 5-7% are usual. Values exceeding 10% are a bit worrisome. While it allows IRP to boast a bigger impact for your network it also means that the routing information you receive is not very accurate and that your providers might not be making the very best effort to service your traffic.

figure screenshots/report-13-loss-improvements-absolute-rates.png

Figure 3.4.5 Loss absolute rates

Problem destinations - highlights destination counts of affected routes in the groups as above. The charts present the rate of problem destinations identified by IRP and subsequently highlights how well IRP was able to improve on them. The counts allow you to infer whether there are 20 customers suffering minor packet loss (say, 5% each) or 2 customers suffering severe packet loss (for example 50% each). If for the top and middle graphs above IRP was able to reduce packet loss by 80% these pie charts allow you to infer that there were 20 or 2 customers affected and that for some of them IRP solved the problem completely and how many others are still suffering.

figure screenshots/report-13-loss-improvements-lossy-destinations.png

Figure 3.4.6 Loss-improved destinations

3.4.4 Latency Improvements

The Latency Improvements report presents an overview for average latency values before and after IRP optimization during a specific timeperiod and depicts them across the following populations:

Sample destinations that cover good and problematic destinations that IRP probed during selected timeperiod;
Problematic destinations cover those routes/prefixes that IRP assessed to have excessive latency rates and identified a better route via a different upstream provider;
20% or more cover those destinations where IRP was able to identify a route with latency improvements of 20% or more;
50% or more cover those destinations where IRP was able to identify a route with latency improvements of 50% or more.

figure screenshots/report-14-latency-improvements-values-ms.png

Figure 3.4.7 Latency values (ms)

Latency values (ms) - highlights the actual latency values before and after improvement for the same groups of destinations as above.

figure screenshots/report-14-latency-improvements-values-percent.png

Figure 3.4.8 Latency values (%)

Latency destinations - highlights destination counts in the groups as above.

figure screenshots/report-14-latency-improvements-destinations.png

Figure 3.4.9 Latency destinations

3.4.5 Probe conversion rates

Probe conversion rates - highlights the counts of networks/prefixes probed by IRP and their conversion rate into any of the possible improvements:

Commit
Performance
Cost

figure screenshots/report-15-probe-to-improvement-conversion-rates.png

Figure 3.4.10 Probe conversion rates

3.4.6 Current improvements

The "Current Improvements" report provides a list of the currently active improvements as per network. A drill-down capability to a specific ASN or prefix is also provided. The report shows the improved prefix and the providers from and to which the traffic was redirected. It also provides the reason, which the improvement was based upon, along with the performance values before and after the traffic was rerouted.

If necessary, specific improvements can be manually deleted by clicking the checkbox next to the needed prefix, afterward using the “Remove selected” button.

figure screenshots/report-3-current-improvements.png

Figure 3.4.11 Current improvements

In some specific cases, the “From” and “To” fields will contain the same provider. This is not a platform issue and is related to the Outage detection algorithm specifics. For more details see the Outage detection↑ and VIP Improvements↑ sections.

The “ASN” field can be empty in case there is no information about the AS number (a prefix belongs to) in the IRP ASN dictionary. As soon as the ASN information is available in the dictionary, the field will be filled accordingly.

Of note is that Current Improvements report also has a map view.

figure screenshots/report-3-current-improvements-map.png

Figure 3.4.12 Current improvements map view

The map view besides offering the usual zooming and panning capabilities also offers clustering and highlighting of improvements by either provider or improvement type.

3.4.7 VIP Improvements

The “VIP Improvements” report contains a list of VIP Prefixes/VIP ASNs that have been introduced by user and have also been translated by IRP. The list can be collapsed or expanded. Furthermore a single prefix or the whole ASN may be reprobed from this report by clicking the “Reprobe” link. The VIP Prefixes/ASN entries can be removed if necessary.

figure screenshots/report-3_1-vip-improvements.png

Figure 3.4.13 VIP Improvements

New VIP Prefixes/ASNs may be added to the system directly from this report using “Add VIP Prefix” or “Add VIP ASN” buttons.

figure screenshots/report-3_2-vip-improvements-add-vip-prefix.png

Figure 3.4.14 Add VIP Prefix/ASN

3.4.8 Improvements to Exchanges

The “Improvements to Exchanges” report contains a complete list of current improvements where the new provider is an Internet Exchange. The report includes data about improved prefixes and their relative traffic by means of last minute and average bandwidth usage. Average bandwidth is estimated based on current hour statistics.

figure screenshots/report-17-Improvements-to-Exchanges.png

Figure 3.4.15 Improvements to Exchanges

3.4.9 Historical records

The “Historical records” report contains a complete list of active and inactive improvements that were made by the system. The report can be sorted using any of the columns by simply clicking on the column header.

figure screenshots/report-4-historical-records.png

Figure 3.4.16 Historical records

In some specific cases, the “From” and “To” fields will contain the same provider. This is not a platform issue, and is related to the Outage detection algorithm specifics. Please see the Outage detection↑ section for more details

3.4.10 Providers overall

The "Providers Overall" report provides the number of prefixes that were rerouted from and to each of the providers. In the example below, the traffic exchange between the providers is balanced. However, if the difference between the outgoing and incoming traffic is significant, then the quality of that provider’s network should be questioned. The report also shows the number of improved prefixes, based upon performance and cost, for each of the providers.

figure screenshots/report-5-peers-overall.png

Figure 3.4.17 Providers overall

3.4.11 Provider performance

During the day IRP makes many probes to determine performance characteristics of alternatives routes.

Provider performance report presents an aggregated view of this measurements for providers. The data on the report highlights Packet loss and Average latency per provider putting the provider with best packet loss data at the top.

figure screenshots/report-16-provider-performance.png

Figure 3.4.18 Provider performance

Provider performance report has been extended and now also offers a daily performance view. The extended Provider daily performance report adds the following capabilities:

a Best (hypothetical) provider that estimates what might be possible to achieve on your network in terms of loss and average latency,
classification of destinations by distance from your network with the option to review a category of interest,
filtering by date range allowing review of past data or average performance over longer time intervals,
a table view with exact details of the data in the report.

Figure 3.4.19 Provider daily performance

Note that while some providers seem better performing (for example, NLIX or AMS-IX in the provided screen capture) the fact that they display a much smaller number of probes clearly indicates they are Internet Exchanges with only a few peers interchanging data with our network.

3.4.12 Cost overview

This report is available only if the system is running in Cost optimization mode (see Cost optimization↑).

The “Cost Overview” report provides details regarding the cost savings, resulted from running IRP for each provider. It displays the costs that would be normally paid to the providers and the costs incurred after IRP has optimized the routing. The system calculates the total transit cost while running in Cost improvement mode and compares it with the total transit cost that would be incurred without running IRP. This difference is shown on the right side of the table as Cost Savings. The total cost savings include the ones made by the Commit Control as well. If the provider’s billing periods differ, then it is possible to choose a billing period for each provider separately.

figure screenshots/report-6-cost-overview.png

Figure 3.4.20 Cost overview

3.4.13 Congestion and outages

This report is available only if Outage detection↑ is enabled.

The “Congestion and outages” report contains all the as-patterns defined as problematic by the system. The problem state is shown in the “Status” column, as follows:

- A problem on the specified AS-Pattern is suspected
- The problem was not confirmed by additional re-probing
- The problem was confirmed, and all related prefixes were rerouted
- The problem was confirmed, but no better-performing route has been found

figure screenshots/report-7-as-path-pattern-problems.png

Figure 3.4.21 AS-pattern problems

3.4.14 Top volume prefixes

The “Top volume prefixes” displays a list of prefixes sorted by the total volume transferred to these remote networks.

figure screenshots/report-8-top-volume-prefixes.png

Figure 3.4.22 Top volume prefixes

3.4.15 Top volume AS

The “Top Volume AS” reports show the Autonomous Systems with the highest volume of traffic coming from the monitored network. It can help you understand your traffic flow directions. If according to this report a specific AS has significantly more traffic than the others, then the network operator may consider getting a direct connection to this AS, to reduce traffic costs.

figure screenshots/report-10-top-volume-as.png

Figure 3.4.23 Top volume AS

3.4.16 Top problem AS

The "Top Problem AS" report reveals the most problematic Autonomous Systems that the monitored networks are sending traffic to. It does that by showing how many times the traffic going to that AS was rerouted by the IRP.

figure screenshots/report-11-top-problem-as.png

Figure 3.4.24 Top problem AS

3.4.17 Prefix statistics

This report lists specific prefixes with relevant monthly values such as traffic volume and number of improvements or current unique and top hosts values. To see top hosts expand the + button to expand to latest individual hosts and their corresponding traffic volume. Note that the upper limit for the number of top hosts is constrained by collector.export.top_volume_ips↓.

figure screenshots/report-9-prefix-statistics.png

Figure 3.4.25 Prefix statistics

Note that Prefix statistics report introduced a map view highlighting with heat-maps those regions of the world where either most traffic or highest number of improvements are made.

figure screenshots/report-9-prefix-statistics-map.png

Figure 3.4.26 Prefix statistics map view

3.4.18 AS statistics

The “AS statistics” report aggregates by Autonomous System the total number of improvements, the volume of traffic and the 95th for a selected past month. Note that the choice of month for the 95th and the time period of the report are independent and they can set non-overlapping ranges.

figure screenshots/report-12-as-statistics.png

Figure 3.4.27 AS statistics

3.4.19 Country statistics

The “Country statistics” report distributes traffic volume and number of improvements by country. It helps see to what regions most of a network’s traffic is addressed and also the relative number of suboptimal routes towards them that IRP identified and addressed by injecting improvements.

figure screenshots/report-country-statistics.png

Figure 3.4.28 Country statistics

Country statistics report also displays a map view highlighting countries with most/least number of improvements.

figure screenshots/report-country-statistics-map.png

Figure 3.4.29 Country statistics map view

3.4.20 Bandwidth by peer

The "Bandwidth by Peer" report lists statistics regarding communications with peers on Internet Exchanges. Filters and sorting help identify the required details.

figure screenshots/report-ix-peer-bandwidth.png

Figure 3.4.30 Bandwidth by peer

3.4.21 Current inbound improvements

The “Current inbound improvements” report provides a list of the currently active improvements. A drill-down capability to a specific ASN or prefix is also provided. The report shows the improved prefix and the providers from and to which the traffic was redirected. It also provides the reason, which the improvement was based upon, along with the performance values before and after the traffic was rerouted.

If necessary, specific improvements can be manually deleted by clicking the check-box next to the needed prefix, afterward using the “Remove selected” button.

figure screenshots/report-18-Inbound-CurrentImprovements.png

Figure 3.4.31 Current inbound improvements

Refer Inbound optimization↑ for details.

3.4.22 Inbound Improvements history

The "Inbound Improvements history" report provides a detailed list of past inbound improvement actions. Filtering for example by specific prefix or improvement type is provided. The report shows the improved prefix and the new prepends count towards a provider.

figure screenshots/report-inbound-hisotry.png

Figure 3.4.32 Inbound Improvements history

Refer Inbound optimization↑ for details.

3.4.23 Probes today

The "Probes Today" report provides probing details, including probes that did not warrant an improvement or failed completely. The report shows probed prefixes and actual selected probing IPs in that prefix with details about probe state and actual measurements across all providers. Missing details for some providers, for example for partial providers or Internet exchanges indicate that either the provider was stopped, the prefix is not serviced by the provider or entirely a probing IP was not identified and the probe has failed complete.

Icons and coloring are used in the report to highlight data for example if an improvement exists for the prefix, or what was the current provider during probing or whether there’s a 100% loss. Filtering and sorting can be used to highlight and group the data into different specific categories.

figure screenshots/report-probes-today.png

Figure 3.4.33 Probes today

3.4.24 Monitors status

The "Monitors Status" report highlights the current status of IRP monitors per provider. For Internet Exchanges the report highlights all the peers with a tooltip with details.

figure screenshots/report-monitors-status.png

Figure 3.4.34 Monitors status

The "Historical monitors" report supplements Monitors status by highlighting the time, monitor details and the new state facilitating the tracing of various issues on the network and in IRP configuration.

figure screenshots/report-monitors-history.png

Figure 3.4.35 Historical monitors

3.5 Graphs

The IRP graphs are visual representations of data found in the reports. They provide an opportunity to quickly identify problems in the network, as well as forecast periodic changes in the traffic patterns and network behavior. For each graph you can adjust the time period which you want to get the results for, and export them as PDF (see the Sidebar↑ section).

Unless otherwise noted, all graphs are plotted using 5 minute intervals.

figure screenshots/frontend-graphs-menu.png

Figure 3.5.1 Graphs menu

3.5.1 Improvements by type

The “Improvements by type” graph shows the improvements made by IRP based upon: performance, cost, commit, and outage detection. Various problem patterns can be detected in compliance with the IRP activity shown in this graph. The graph provides information on the types of improvements that are more frequent for the network in the current state. Different patterns can be noticed depending on the improvement mode that is currently running.

figure screenshots/graph-1-improvements-by-cause.png

Figure 3.5.2 Improvements by type

3.5.2 Performance improvements

The “Performance improvements” graph displays the prefixes improved based upon a performance reason. By following this graph, the solved loss and latency issues, occurring in the network, can be monitored. The graph also provides the average, maximum and minimum number of improvements for each of the improvement reasons during the selected time frame. The maximum peaks are indicators of a loss or latency problem in the network.

figure screenshots/graph-3-performance-improvements.png

Figure 3.5.3 Performance improvements

3.5.3 Probed prefixes and Improvements

The "Probed prefixes and Improvements " graph shows the number of improvements (Loss/Latency) per time unit as reported to the total probed prefixes.

figure screenshots/graph-4-improved-prefixes-within-probed.png

Figure 3.5.4 Probed prefixes and Improvements

3.5.4 New and retry improvements

After a particular improvement was made, IRP analyzes the path periodically. If the improvement is still valid it leaves it as it is. The “New and retry improvements” graph provides the number of improvements made for the first time as related to those, which were simply confirmed after re-probing.

figure screenshots/graph-5-new-and-retry-improvements.png

Figure 3.5.5 New and retry improvements

3.5.5 Improvements by probing source

The current graph displays the percentage of improvements that are differentiated by the probing source including Commit Control, Outage Detection, VIP Probing, Regular and Retry Probing.

figure screenshots/graph-16-improvements-by-probing-source.png

Figure 3.5.6 Improvements by probing source

3.5.6 Prefixes rerouted from provider

The current graph displays the number of prefixes that were rerouted from a specific provider in order to resolve performance or cost issues.

figure screenshots/graph-6-prefixes-rerouted-from-peer.png

Figure 3.5.7 Prefixes rerouted from provider

3.5.7 Prefixes rerouted to provider

The graph displays the number of prefixes that were rerouted to a specific provider in order to resolve performance or cost issues.

figure screenshots/graph-7-prefixes-rerouted-to-peer.png

Figure 3.5.8 Prefixes rerouted to provider

3.5.8 Improvements by provider

The “Improvements by provider” graph shows all the improvements made by IRP for each of the providers. You can monitor the issues occurring in the network, which triggered IRP to reroute the traffic. You can compare the providers based on how they perform in the context of performance and cost. It also provides the average, maximum and minimum number of improvements per provider for each of the improvement reasons.

Improvements by provider graph categorizes improvements by performance and cost. If cost mode is disabled this graph shows the same data as Prefixes rerouted to provider graph.

figure screenshots/graph-8-improvements-by-peer.png

Figure 3.5.9 Improvements by provider

3.5.9 Performance improvements by provider

Similar to the previous graph, the “Performance improvements by provider” displays a graphical representation of historical values for all latency and loss-based improvements for each provider. The charts are displayed next to each other for easier comparison and identification of regularities. Keep in mind that specific provider lines can be removed from the graph. This way we are able to focus only on the interesting data.

figure screenshots/graph-9-latency-improvements-by-peer.png

Figure 3.5.10 Performance improvements by provider

3.5.10 Commit improvements by provider

If the Commit Control algorithm is enabled (see Commit Control↑), this graph will display an overview of all the commit improvements that were enforced by IRP for a specific time period.

figure screenshots/graph-11-commit-improvements-by-peer.png

Figure 3.5.11 Commit improvements by provider

3.5.11 Total and Improved traffic

The Total and Improved Traffic graph allows you to see the positive impact of IRP on the network. It provides the amount of improved traffic, shown in blue color, as related to the total traffic, shown in green. Thus, you can monitor how much of the traffic is affected by IRP.

figure screenshots/graph-12-total-and-improved-traffic.png

Figure 3.5.12 Total and improved traffic

3.5.12 Bandwidth usage by provider

The Bandwidth Usage graph shows the total usage for each of the providers. It allows you to compare the current traffic volume to the current 95th percentile and the commit 95th. The graph provides average, maximum and minimum traffic usage values for each of the providers during the selected time period.

figure screenshots/graph-13-peers-bandwidth-usage.png

Figure 3.5.13 Bandwidth usage by provider

3.5.13 Providers bandwidth usage

The Providers bandwidth usage graph combines all usage lines in a single chart. It allows cross-comparison between providers themselves. The chart includes options to hide/show providers of interest during selected time period. The top and bottom charts display outbound and inbound traffic accordingly.

figure screenshots/graph-all-providers-bandwidth-usage.png

Figure 3.5.14 Providers bandwidth usage

3.5.14 Bandwidth usage and improvements

Bandwidth usage and improvements chart superimposes two charts for easy analysis and comparison of how improvement counts correlate with bandwidth.

figure screenshots/graph-14-bandwidth-usage-and-improvements.png

Figure 3.5.15 Bandwidth usage and improvements

3.5.15 Total bandwidth usage

figure screenshots/graph-15-total-bandwidth-usage.png

Figure 3.5.16 Total bandwidth usage

3.5.16 Inbound traffic distribution

The graph highlights distribution of inbound traffic across inbound prefixes and providers.

figure screenshots/graph-17-Inbound-Traffic-Distribution.png

Figure 3.5.17 Inbound traffic distribution

Inbound traffic distribution report includes a feature to cross-compare current values with a saved inbound traffic distribution from recent past. To cross-compare saved inbound traffic distribution with freshly retrieved statistics, save and compare traffic shape changes and if they conform to expectations.

Refer Inbound optimization↑ for details.

3.6 Routing Policies

To see the full list of routing policies configured in the system, select the Routing Policies option from the main menu and click on the Routing Policies option in the submenu. The list displays all the enabled and disabled routing policies configured in IRP.

Figure 3.6.1 Routing Policies

As shown in the screenshot above, the list provides the following details:

Priority of a policy.

Priority specifies what policy to use in case of overlapping prefixes as might be the case of a large aggregate prefix policy extending over an ASN policy

The prefix/ASN to which the policy was applied.
A note to describe the policy.
The ASN of the prefix to which the policy was applied.
The status of the VIP reprobing (enabled/disabled).
The type of the policy ( allow/deny/static).
Providers which are affected by the policy.
Time passed after the last probe was performed towards this prefix.
The time left before the next scheduled probe towards this prefix.
From and To details of an existing improvement relating to this policy.
The available actions for the policy. These include:
- On-demand probing
- Editing the routing policy
- Enabling the routing policy
- Disabling the routing policy

You can collapse all the policies by clicking the "Collapse All" button and then expand them back by clicking the "Expand all" button. You can use the "Remove all " button to delete all the policies. You can also select a set of policies and use the "Remove selected" button to delete them.

A new routing policy can be added by pressing the "Add routing policy" button. The following window will open up:

figure screenshots/routing-policy-window.png

Figure 3.6.2 Routing policy window

The following parameters should be configured to add a routing policy:

Prefix/ASN: The prefix/ASN to which the policy is applied
Note: A note to describe the policy
Policy Type: The type of the policy (allow/deny/static)
A community to mark improvements
A flag to set if policy is VIP
The policy status (enabled/disabled)
A flag to indicate if an ASN policy should propagate to downstream ASN.
A flag indicating if a global improvement allowed.
Priority slider or text-box will set a policy’s priority.
The Providers for which the policy should be respected
Cascade flag for ASN policies that indicate if the policy should be enforced only for this AS or also for downstream AS. If cascading is enabled it ensures that the policy is enforced for all traffic that will eventually be routed through this AS.

Refer policy configuration parameters Routing Policies settings↓ for details.

To see a list of Static Route policies, select Static Routes in the Routing Policies submenu. Alternatively the filter on the left side-bar can be used. A list like the one below will be displayed.

figure screenshots/routing-policies-static-routes.png

Figure 3.6.3 Static Routes

The list structure and the actions available are similar to those found in the Routing Policies list. Here you can also add a new Static Route by using the "Add static route" button.

You can select the VIP option in the Routing Policies submenu to access a list of all the policies with VIP reprobing enabled.

Figure 3.6.4 VIP policies

Here you can also enable VIP reprobing for a new prefix/ASN by using the "Add VIP" button.

3.7 Flowspec policies

To review Flowspec policies, select the Routing Policies option from the main menu and click on the Flowspec Policies option in the submenu.

Flowspec must be enabled for BGPd to announce them.

The list displays all the enabled and disabled Flowspec policies configured in IRP.

figure screenshots/flowspec-policies.png

Figure 3.7.1 Flowspec policies

As shown in the screenshot above, the list highlights:

A note to describe the policy.
A source ASN/prefix and port(s) for matching packets.
A destination prefix and port(s) for matching packets.
DSCP traffic classification value.
The policy action (Throttle, Drop, Redirect or Redirect IP).
Redirect IP or Provider for redirect policies.
Protocols of matching packets, for example TCP, UDP or ICMP.
Rate limits in Mbps for Throttle policies.

Flowspec policies like routing policies can be edited, enabled, disabled or deleted altogether.

A Flowspec policy is added by clicking on the designated button and specifying the following:

figure screenshots/flowspec-policy-window.png

Figure 3.7.2 Flowspec policy popup

The following parameters should be configured to add a routing policy:

Notes to describe the policy
Status to set if policy is Enabled or Disabled
Source ASN/Prefix/Port(s): The source ASN/prefix andport(s) of the IP packets that match. A prefix in CIDR notation or a single IP address should be provided. Multiple valid TCP/UDP ports can be provided as well as port ranges.

One Flowspec rule is created for each of the prefixes in the designated AS. ASN Flowspec policies can be setup only on the source of IP packets.

Ensure that routers are capable of processing a large number of Flowspec rules before setting policies for an AS consisting of a very large number of network segments (prefixes).

Destination Prefix/Port: The destination prefix/port attribute of the IP packets that match. Same rules as for Source Prefix/Port(s) apply.

Prefixes MUST not equal or include probing IP addresses. Refer to for example global.master_probing_interface↓, peer.X.ipv4.master_probing↓peer.X.ipv6.master_probing↓.

Protocols: Packet protocols that match the policy. Can be any combination of TCP, UDP and ICMP.
Policy Type: The type of the policy (Throttle, Drop or Redirect)
Provider specifies one of the provider identifiers where traffic will be redirected. The provider is set only for Redirect policies.

In order for Redirect policies to be implemented on routers a community value must be assigned to them and VRF must be configured on the router itself. Consult Noction support for details.

Rate limits the allowed bandwidth usage for matching traffic. The value is set only for Throttling policies. The rate specifies a number in the range 1-4200 Mbps.

In case the provided Flowspec policy attributes are incomplete or invalid on attempting to submit it a warning will be raised similarly as depicted in the popup.

For further details refer Flowspec configuration parameters for example: global.flowspec↓, core.flowspec.max↓, core.flowspec.max_ipv6↓, bgpd.peer.X.flowspec↓, peer.X.flowspec.ipv4.redirect_community↓, peer.X.flowspec.ipv6.redirect_community↓.

3.8 Throttling excessive bandwidth use with Flowspec

Throttling excessive bandwidth use with Flowspec↑ policies use the same Flowspec policies reports. Select the Routing Policies option from the main menu and click on the Flowspec Policies option in the submenu.

Flowspec must be enabled for BGPd to announce them.

The list displays besides all the other Flowspec policies the automatic throttling rules marked with Throttle at. These rules cannot be edited.

figure screenshots/flowspec-policies-Autothrottling.png

Figure 3.8.1 Throttling excessive bandwidth use with Flowspec policies

Each of the rules sets a specific destination prefix to control outbound bandwidth use and sets an Mbps rate based on past average bandwidth use.

Refer Core configuration↓ for configuration details.

3.9 Maintenance windows

To review Maintenance windows, navigate to Policies option in main menu and click on Maintenance Window option. The list displays all the current and future maintenance windows configured in IRP. Refer Maintenance windows↑ for details about maintenance windows.

figure screenshots/maintenance-windows.png

Figure 3.9.1 Maintenance windows

As shown in the screen-shot above, the list highlights:

Current color-coded status of the maintenance window where Red and Orange depict unloading and actual maintenance phases while Blue and Green depict future planned windows and past finished windows correspondingly.
Begin and End date-times of maintenance windows.
Provider or router under maintenance.
Withdraw Improvement option enables or disables withdrawal of existing Outbound Improvements to that provider.
Seconds to Unload defines time interval in seconds prior to beginning of the maintenance window when IRP starts to unload outbound traffic from provider.

If Seconds to Unload/Seconds to Prepend time intervals set to zero IRP does not perform corresponding action.

When seconds to unload is specified it should be at least 5 minutes (300 seconds) long with larger time intervals allowing IRP more time to assess and reroute traffic flowing through provider with scheduled maintenance window. Very short time intervals might not have the desired effect since usually an IRP optimization cycle takes 2-3 minutes to finish.

Seconds to Prepend defines time interval in seconds prior to beginning of the maintenance window when IRP announces all inbound prefixes with maximum prepends to provider in order to deflect inbound traffic towards other providers.

Maintenance windows can be added, edited or deleted using the designated action buttons.

A maintenance window is added by specifying the following:

Figure 3.9.2 Maintenance window popup

The parameters mentioned above including the option to choose a single provider or all providers on a router are provided for Maintenance window configuration.

Maintenance windows are also highlighted on IRP Frontend sidebar with the title of the sidebar box highlighting the status of the next maintenance window so that it is visible even if the contents of the sidebar is collapsed.

figure screenshots/maintenance-window-sidebar.png

Figure 3.9.3 Maintenance window sidebar

3.10 Events

The system events are diagnostic messages that were sent by the IRP Components (See also: IRP Components↑), as well as notifications caused by the BGP monitoring algorithm (See also: BGP Monitoring↑). The events are displayed in the status bar located at the bottom of the system frontend (See Frontend structure↑). Several or all the events can be marked as “Read”, so they will no longer be displayed in the status bar.

Figure 3.10.1 System events

3.11 Troubleshooting

The IRP Frontend provides several troubleshooting tools, that can offer you quick information on the specific remote networks.

3.11.1 Traceroute

A traceroute to a specific IP address or hostname can be run from the Frontend. Results will be displayed in the graphical and detailed formats, as in the examples below. The “Allow automatic improvement” checkbox can also be used for automatic improvement of the provided IP address.

figure screenshots/troubleshooting-traceroute.png

Figure 3.11.1 Traceroute

figure screenshots/traceroute-graphical.png

Figure 3.11.2 Traceroute results

figure screenshots/traceroute-details.png

Figure 3.11.3 Traceroute exploring details results

3.11.2 Looking glass

A ping, traceroute or MTR can be run from the Frontend via the default route or via a specific provider.

figure screenshots/troubleshooting-looking-glass.png

Figure 3.11.4 Looking glass

3.11.3 Whois

The “Whois” tool can query IP addresses, hostnames, network prefixes, or AS Numbers.

The following valid values can be entered in the search box:

Hostname: fully qualified domain name, e.g: “domain.com”
IP: valid IP address, e.g: “10.20.30.40”
IP Block: any valid prefix, in CIDR format, e.g: “10.20.30.40/24”
ASN: valid ASN, using the following format: “AS1234”

figure screenshots/troubleshooting-whois.png

Figure 3.11.5 Whois

3.11.4 Prefix probing

IRP features a manual prefix probing function. One can manually submit a specific IP address for probing and optimization by the system. Valid hostnames and IP addresses can be entered. The probing results will be displayed on the same page. Please note that IRP probes the exact entered IP address. In case the IP address doesn’t respond, IRP runs the indirect probing process and optimizes the whole prefix based on the indirect probing results.

Prefix probing allows to automatically or manually choose the best path for a particular IP address or prefix.

In order to check what are the possible paths for a particular prefix, enter the IP address or prefix into the field. If you prefer the prefix to be improved automatically, tick the check box “Improve automatically” and press the “Submit for probing” button. Otherwise, just press the “Submit for probing” button and wait until the system returns the probing results.

figure screenshots/troubleshooting-prefix-probing-input.png

Figure 3.11.6 Prefix Probing

The system returns the probing results by displaying the loss, latency and cost values for each of the providers.

figure screenshots/troubleshooting-prefix-probing.png

Figure 3.11.7 Prefix probing results

After the probing results are received, choose the path you consider the best one. Note, that IRP highlights the current and the best path. Use the “Reroute” button to redirect the traffic through the selected provider. You can also set the selected provider as static for the probed prefix by pressing the “Add static route” button or apply a specific routing policy to the probed prefix by clicking the “Add a custom policy” button.

3.12 Configuration editor

Starting with IRP version 1.7, all system parameters can be modified from the Frontend, using the “Configuration” menu.

Depending on the parameter type, several control types are available:

Text boxes, which can hold IP addresses, provider names and numerical values
Auto-complete boxes, used for parameters that can take a limited set of values
Auto-complete list boxes, usually containing a list of IPs / prefixes (in CIDR format) / ASNs
Buttons
Sliders, used to select a value from a predefined range
Drop-down lists
Toggle buttons (used for specific algorithms and BGP peers)

Provider control buttons, can be used to suspend, resume, stop, or remove a provider

3.12.1 Global configuration

Global settings can be adjusted from the Frontend, by using the “Configuration” menu.

Please refer to Global and Core Configuration↑ for a detailed parameters description.

Several global parameters can be configured by selecting their desired values and clicking on the “Submit” button.

Available parameters:

Improvement mode (global.improve_mode↓)
Management interface (global.master_management_interface↓)
BGP mode (Intrusive / Non-intrusive, see global.nonintrusive_bgp↓)
Probing interface(s). See global.master_probing_interface↓
Prefix aggregation. If this parameter is enabled, the system operates at aggregate level, up to the Maximum aggregate mask (see: global.aggregate↓, global.agg_ipv4_max↓, global.agg_ipv6_max↓)

figure screenshots/configuration-editor/global-settings.png

Figure 3.12.1 Configuration editor: Global settings

The list of ignored networks can be modified on the same page. The options allow listing as appropriate:

prefixes(global.ignorednets↓),
ASNs (global.ignored.asn↓),
BGP Community attributes that mark prefixes/routes to ignore (global.ignored_communities↓).

Setting prefixes and ASNs is possibleeither manually, or by importing a text file containing one IP/network in CIDR format or ASN per line.

figure screenshots/configuration-editor/global-settings-ignored-nets.png

Figure 3.12.2 Configuration editor: Ignored prefixes

Failover functionality is switched on or off (global.failover↓) as part of Global configuration too.

Failover license is required for Failover tab to be available.

Failover configuration covers the following parameters:

Failover timer - time period in seconds during which master is considered alive and before slave becomes active if there are no announcements from master (global.failover_timer_fail↓).
Failback timer - time period before slave node will release control back to master (global.failover_timer_failback↓). This is to protect from failover flapping between master and slave in case of master instability.
Slave IPv4 address - indicates to master what is the slave node (global.failover_slave.ip↓). Master uses this address to setup SSH connections and sync its own configuration files.
Slave SSH port - indicates the TCP port number for SSH on the slave node (global.failover_slave.port↓). Master uses this port to setup SSH connections and sync its own configuration files.

figure screenshots/configuration-editor/global-settings-failover.png

Figure 3.12.3 Configuration editor: Global failover

3.12.2 BGP and Routers configuration

BGP daemon-related parameters can be configured in the “Configuration → BGP” section.

BGPd parameters (See also: BGPd Configuration↑, BGPd settings↓):
- BGP mode intrusive/non-intrusive (global.nonintrusive_bgp↓)
- BGP monitor enabled/disabled, this option is useful during DoS/DDoS attacks. BGPd ICMP/SNMP monitors can be disabled on the fly, without restarting the BGPd daemon.
- AS-Path attribute a restore priority (bgpd.as_path↓)
- Remove prefixes on next-hop update (bgpd.improvements.remove.next_hop_eq↓)
- Remove prefixes on aggregate withdraw (bgpd.improvements.remove.withdrawn↓)
BGP monitoring settings (See also: BGP Monitoring↑):
- Guard time (bgpd.mon.guardtime↓)
- Holdtime (bgpd.mon.holdtime↓)
- Keepalive (bgpd.mon.keepalive↓)

figure screenshots/configuration-editor/bgp-settings.png

Figure 3.12.4 Configuration editor: BGP settings

BGP routers can be added, removed or modified on the same page.

A new BGP neighbor can be added using the “Add Router” button. Several parameters must be configured:

General settings:
- Router name
- ASN to be used for the iBGP session (bgpd.peer.X.as↓)
- Announced local-pref value (bgpd.peer.X.master_localpref↓)
- Local IP address (bgpd.peer.X.master_our_ip↓, bgpd.peer.X.master_our_ipv6↓)
- Neighbor IP address, usually the router’s IP (bgpd.peer.X.peer_ip↓, bgpd.peer.X.peer_ipv6↓)

figure screenshots/configuration-editor/bgp-add-peer-general.png

Figure 3.12.5 Configuration editor: New Router, general settings

Advanced settings:
- Announced communities to be appended to the Community attribute (bgpd.peer.X.master_communities↓)
- Keepalive (bgpd.peer.X.keepalive↓)
- Announced MED value (bgpd.peer.X.med↓)
- Router ID, mandatory for IPv6 only (bgpd.peer.X.slave_router_id↓)
- Update split feature (More details: bgpd.updates.split↓)

figure screenshots/configuration-editor/bgp-add-peer-advanced.png

Figure 3.12.6 Configuration editor: New Router, advanced settings

Failover settings - the same set of parameters shall be setup for both master and slave nodes. Use the toggle to switch between them.
- Announced communities to be appended to Community attribute (bgpd.peer.X.slave_communities↓, bgpd.peer.X.slave_communities↓)
- Announced LocalPref value (bgpd.peer.X.master_localpref↓, bgpd.peer.X.slave_localpref↓)
- Local IPv4 address (bgpd.peer.X.master_our_ip↓, bgpd.peer.X.slave_our_ip↓)
- BGP session password (bgpd.peer.X.master_password↓, bgpd.peer.X.slave_password↓)

figure screenshots/configuration-editor/bgp-settings-failover.png

Figure 3.12.7 Configuration editor: New Router, advanced settings

3.12.3 Collector configuration

Global collector parameters, as well as Span and Flow-specific settings can be adjusted by using the “Configuration → Collector” menu item.

Global collector settings:
- Top volume prefixes per cycle (collector.export.volume.high.top_n↓)
- Minimal traffic volume (collector.export.volume.min↓)
- Minimal traffic volume (%) (collector.export.volume.min_pct↓)
Flow collector settings:
- UDP port to listen for NetFlow or sFlow packets (collector.flow.listen.nf↓, collector.flow.listen.sf↓)
- Flow sources (collector.flow.sources↓)
SPAN collector settings:
- Capture interfaces (collector.span.interfaces↓)
- Mindelay algorithm enable/disable (collector.span.min_delay↓)
- Mindelay probing queue slots (collector.span.min_delay.probing_queue_size↓)

figure screenshots/configuration-editor/collector-settings.png

Figure 3.12.8 Configuration editor: Collector settings

The list of prefixes announced by the monitored network can be edited in the same form. Its contents can be also imported from a text file.

3.12.4 Core configuration

All Core parameters affecting the decision-making algorithms can be modified on the next page (“Configuration → Core”).

The following configuration parameters can be adjusted:

Max IPv4 improvements (core.improvements.max↓, core.improvements.max_ipv6↓)
Standard reprobing period (core.improvements.ttl.retry_probe↓) - the time period after which a specific improvement is being scheduled for re-probing.
VIP reprobing period (core.vip.interval.probe↓) - defines the VIP prefixes probing interval, in seconds.
Minimal probe lifetime (core.probes.ttl.min↓) - Ordinary probing will not be performed for a specific prefix if its probe age is lower than this value.
Allowed latency worsening (core.cost.worst_ms↓)
Exploring queue slots (core.eventqueuelimit↓)
Top-N relevant volume prefixes (core.improvements.retry_probe.volume_top_n↓)
Relevant loss for loss-based improvements (core.performance.loss_pct↓) - a prefix will be improved based on a loss cause only if the packet loss can be decreased by this value (in %)
Relevant RTT difference (core.performance.rtt.diff_ms↓)
Relevant RTT difference (%) (core.performance.rtt.diff_pct↓)
Maximum probe lifetime (core.probes.ttl.max↓)

figure screenshots/configuration-editor/core-settings.png

Figure 3.12.9 Configuration editor: Core configuration

Outage detection↑ algorithm can be enabled and configured on the same page. Outage detection algorithm can be enabled or disabled using the toggle On/Off buttons at the top of each form.

figure screenshots/configuration-editor/core-settings-outage.png

Figure 3.12.10 Configuration editor: Outage detection

Throttling excessive bandwidth use with Flowspec↑ can be enabled and configured on the same page. The feature can be enabled or disabled using the toggle On/Off buttons at the top of each form.

figure screenshots/configuration-editor/core-settings-overusage.png

Figure 3.12.11 Configuration editor: Overusage

The following configuration parameters can be adjusted:

Overusage interval (core.overusage.check_interval↓) sets the frequency in seconds of checking for excessive bandwidth use.
Overusage rule retention (core.overusage.hold_timer↓) sets how much time a rule is kept after bandwidth use returns to normal.
Prefix BW average time (core.overusage.out.average.period↓) sets the number of hours used to determine the average bandwidth use of a prefix.
Prefix relevant BW (core.overusage.out.average.relevant_min↓) sets the relevant bandwidth use in Mbps by a prefix before considering any rules for it.
Overusage throttle multiplier (core.overusage.out.threshold.throttle↓) sets the multiplier to apply to average prefix bandwidth use when setting a rule.
Overusage threshold multiplier (core.overusage.out.threshold.trigger↓) sets the threshold multiplier used to determine excessive bandwidth use.

Prefix relevant BW and Overusage threshold multiplier parameters control the number of Flowspec rules that will be generated automatically. Adjust these values to control the number of Throttling Flowspec policies.

Circuit issues detection ↑ parameters are listed and can be adjusted:

figure screenshots/configuration-editor/core-settings-circuit-issues.png

Figure 3.12.12 Configuration editor: Circuit issues detection

The following configuration parameters can be adjusted:

Delta loss to shutdown (core.circuit.high_loss_diff↓) sets threshold to initiate shutting down a provider with circuit issues as a difference between examined provider’s average loss and overall average loss during the given time horizon. Shutdown is attempted only for providers marked accordingly.
Delta loss to warn (core.circuit.warn_loss_diff↓) sets threshold to raise a warning regarding issues with a provider as difference between examined provider’s average loss and overall average loss during the given time horizon. Usually this threshold is significantly lower than the shutdown threshold.
Delta loss to restore (core.circuit.recover_loss_diff↓) sets low loss level threshold when IRP will restore full functionality over provider that had circuit issues.
Issues time horizon (core.circuit.hist_interval↓) sets how many minutes in the past IRP looks for probes with loss over both analyzed provider and all the other providers on the network.
Withdraw improvements on warn (core.circuit.withdraw_on_warn↓) instructs IRP to withdraw outbound improvements over provider with circuit issues when warning threshold is reached. By default improvements are withdrawn only when shutdown threshold is exceeded.
Restore after (core.circuit.recover_hold_time↓) sets the interval in seconds after which IRP should re-evaluate a provider’s circuit loss and attempt restoring it to normal function. This will be performed only for providers that are configured to attempt to restore after shutdown.
Restore interval (core.circuit.recover_monitored_intervals↓) sets the interval in minutes during which IRP will periodically re-evaluate a provider’s circuit loss and attempt restoring it to normal function. This will be performed only for providers that are configured to attempt to restore after shutdown.

The configuration parameters above are applied for all providers even though for individual providers different level of control can be set starting from disabling and ending with attempting both to automatically shutdown and later restore connectivity with it. See also Providers configuration↓

3.12.5 Commit Control configuration

Commit Control↑ algorithm can be enabled and configured on the Commit Control group of pages. It can be enabled or disabled using the toggle On/Off buttons at the top of the form.

If NetFlow is used to collect statistics Routers MUST be configured to export statistics every minute (or as often as possible). Some router models have default export intervals for either inactive or active flows of up to 1800 seconds. Big delays cause IRP to react very slowly to increased load and reduce the effectiveness of Commit Control feature.

The most important parameters for Commit Control are, as follows:

Commit Control probing queue slots (core.commit_control.probing_queue_size↓)
Minimal prefix bandwidth in Mbps (core.commit_control.agg_bw_min↓)

figure screenshots/configuration-editor/core-settings-cc.png

Figure 3.12.13 Configuration editor: Commit Control

Providers Overall block displays Commit Control configuration for all providers including their precedence. See details in the figure below.

Individual provider parameters can be adjusted as well as detail regarding current Commit Control changes applied by IRP can be accessed by following the provided links.

figure screenshots/configuration-editor/cc-providers-overall.png

Figure 3.12.14 Providers Overall

3.12.6 Explorer configuration

To adjust the Explorer-related parameters, the “Configuration → Explorer” menu item must be accessed. The Explorer Configuration↑ and Explorer settings↓ sections should be consulted for detailed Explorer configuration and parameters description.

Explorer parameters:

Infrastructure IPs (explorer.infra_ips↓)
Explorer worker threads (explorer.maxthreads↓)
Probing algorithms and Traceroute algorithms (explorer.probe.algorithm↓, explorer.trace.algorithms↓)
High volume task precedence (explorer.high_vol_precedence↓)
Process max collected IPs (explorer.max_collector_ips↓)
First probing packets amount (explorer.probing.sendpkts.min↓)
Adaptive probing packets count (explorer.probing.sendpkts.adaptive_max↓)
ICMP timeout (explorer.timeout↓)
Traceroute retry packets and packets per hop (explorer.traceroute.sendpkts↓, explorer.traceroute.retrypkts↓)
Traceroute minimum and maximum ttl (explorer.traceroute.ttl.min↓, explorer.traceroute.ttl.max↓)

figure screenshots/configuration-editor/explorer-configuration.png

Figure 3.12.15 Configuration editor: Explorer settings

3.12.7 Providers and Peers configuration

Providers can be added and modified via the “Configuration → Providers and Peers” menu.

3.12.7.1 Providers configuration

To add a new provider, the “Add provider” button must be used. Several parameters should be configured for the new provider:

General settings:
- Provider short name (peer.X.shortname↓)
- Router (peer.X.bgp_peer↓)
- Probing IPv4 address (peer.X.ipv4.master_probing↓) or Probing IPv6 address (peer.X.ipv6.master_probing↓)
- Provider’s routing domain (peer.X.rd↓)
- Provider cost per Mbps (peer.X.cost↓)
- Maximum load per interface (peer.X.limit_load↓)
- Provider description (peer.X.description↓)
- Flow agents (peer.X.flow_agents↓)
- Circuit issues detection (peer.X.circuit.control↓) indicating how IRP should react to excessive persistent loss over a particular provider.

figure screenshots/configuration-editor/add-peer-general.png

Figure 3.12.17 Configuration editor: Adding a new provider, General settings

Commit Control configuration:
- Provider cost per Mbps (peer.X.cost↓)
- Maximum load per interface (peer.X.limit_load↓)
- Provider 95th percentile (peer.X.95th↓)
- Provider inbound 95th percentile (peer.X.95th.in↓)
- Provider 95th calculation mode for inbound traffic (95th calculation modes↑)
- Commit Control status for this provider (peer.X.cc_disable↓)
- CC provider precedence - used for Commit Control and grouping (peer.X.precedence↓)
- Performance/Cost improvements within provider group flag in case Provider load balancing↑ is configured, (peer.X.improve_in_group↓, peer.X.precedence↓)
- Provider billing day (peer.X.95th.bill_day↓)
- Centile value (peer.X.95th.centile↓)

figure screenshots/configuration-editor/add-peer-cc.png

Figure 3.12.18 Configuration editor: Adding a new provider, Commit Control

SNMP-related settings:

Provider specific SNMP parameters are deprecated in IRP 3.7 and later. Refer SNMP hosts configuration↓ and SNMP Hosts settings↓ for SNMP configuration parameters used in IRP 3.7 and newer.

- Flag whether IPv4 or/and IPv6 is used
- Provider SNMP IPv4/IPv6 address (peer.X.snmp.ip↓, peer.X.snmp.ipv6↓)
- Provider SNMP community (peer.X.snmp.community↓)
- Provider SNMP interface name or index (peer.X.snmp.interface↓); the list of SNMP interfaces is automatically retrieved after Provider SNMP IPv4/IPv6 address field is filled
- SNMP version (peer.X.snmp.version↓)

figure screenshots/configuration-editor/add-peer-snmp.png

Figure 3.12.19 Configuration editor: Adding a new provider, SNMP settings

SNMP v3 uses additional parameters depending on security services used for statistics collection:
- SNMP security services selects if Authentication and/or Privacy services are used (peer.X.snmp.seclevel↓)
- SNMP authentication password if authentication is used (peer.X.snmp.auth_password↓)
- SNMP authentication protocol if authentication is used (peer.X.snmp.auth_protocol↓)
- SNMP encryption password if privacy is used (peer.X.snmp.priv_password↓)
- SNMP encryption protocol if privacy is used (peer.X.snmp.priv_protocol↓)
- SNMP Username if authentication is used (peer.X.snmp.auth_username↓)
- SNMP version (peer.X.snmp.version↓)
External Monitor settings:
- External monitor status flags for IPv4 / IPv6 (peer.X.mon.enabled↓, peer.X.mon.ipv6.external.enabled↓)
- ICMP/UDP ping monitored IPv4 / IPv6 addresses (peer.X.ipv4.mon↓, peer.X.ipv6.mon↓)

figure screenshots/configuration-editor/add-peer-monitoring.png

Figure 3.12.20 Configuration editor: Adding a new provider, External Monitor

Internal Monitor settings:
- Internal monitor status flags for IPv4 / IPv6 (peer.X.mon.enabled↓, peer.X.mon.ipv6.internal.enabled↓)
- BGP session monitoring IPv4 / IPv6 addresses (peer.X.mon.ipv4.bgp_peer↓, peer.X.mon.ipv6.bgp_peer↓)
- BGP MIBs for IPv4 / IPv6 (peer.X.mon.ipv4.internal.mode↓, peer.X.mon.ipv6.internal.mode↓)
- BGP Internal Monitor IPv4 / IPv6 addresses (peer.X.mon.snmp.ip↓, peer.X.mon.snmp.ipv6↓)
- BGP Internal Monitor SNMP Community (peer.X.mon.snmp.community↓)
- SNMP version used for monitoring (peer.X.mon.snmp.version↓)

figure screenshots/configuration-editor/add-peer-monitoring-internal.png

Figure 3.12.21 Configuration editor: Adding a new provider, Monitoring

SNMP v3 uses additional parameters depending on security services used for monitoring:

Provider specific SNMP parameters are deprecated in IRP 3.7 and later. Refer SNMP hosts configuration↓ and SNMP Hosts settings↓ for SNMP configuration parameters used in IRP 3.7 and newer.

- SNMP security services selects if Authentication and/or Privacy services are used (peer.X.mon.snmp.seclevel↓)
- SNMP authentication password if authentication is used (peer.X.mon.snmp.auth_password↓)
- SNMP authentication protocol if authentication is used (peer.X.mon.snmp.auth_protocol↓)
- SNMP encryption password if privacy is used (peer.X.mon.snmp.priv_password↓)
- SNMP encryption protocol if privacy is used (peer.X.mon.snmp.priv_protocol↓)
- SNMP Username if authentication is used (peer.X.mon.snmp.auth_username↓)
- SNMP version (peer.X.mon.snmp.version↓)
IPv4 / IPv6 settings:
- IPv4 / IPv6 diagnostic hops (peer.X.ipv4.diag_hop↓, peer.X.ipv6.diag_hop↓)
- Probing IPv4 / IPv6 address (peer.X.ipv4.master_probing↓, peer.X.ipv6.master_probing↓)
- Remote provider ASN (peer.X.ipv4.next_hop_as↓), used by the AS-Path restoration algorithm (bgpd.as_path↓)
- Router next-hop address (peer.X.ipv4.next_hop↓), that sets the next-hop value for injected routes related to this provider

figure screenshots/configuration-editor/add-peer-ipv4.png

Figure 3.12.22 Configuration editor: Adding a new provider, Commit Control

The same parameters can be adjusted for the already configured providers.

3.12.7.2 Internet Exchanges configuration

An Exchange is configured as a special type of provider. The important things to consider when setting up an Exchange are listed below.

It is recommended that Noction systems engineers guide you through Exchange configuration process.

IRP needs one ACL and one PBR rule for each Peering Partner on an Exchange. Internet Exchanges can have hundreds of Peering Partners. Ensure that the number of Access Lists, Access List entries and PBR entries will not exceed router hardware limitations.

Once the Exchange is setup IRP will retrieve the routing table on the router in order to setup Exchange peering partners. IRP will require access to the router in order to do so.

When configuring an Exchange its Diag Hop parameter must be provided. Diag Hop for an Exchange represents the list of direct-connected networks configured on the router’s interface towards the Exchange network. This parameter is labeled "IPv4 diagnostic hop" on IRP Frontend.

Probing IPs for Exchanges no longer require one IP address per peering partner since this number might be impractically big. Instead a combination of probing IP and DSCP value is used to configure the PBRs. A single probing IP in conjunction with DSCP can cover up to 64 peering partners. A sufficient number of probing IPs will be required in order to cover all peering partners on your exchange.

As a rule of thumb it is better to list the PBR rules for transit providers before the (large number of) rules for Internet Exchange peers. If the many PBR rules assigned to peers on a large Internet Exchange are placed at the beginning of the PBR list some routers evaluate all of them before finding the terms for transit providers and this consumes valuable router CPU resources.

figure screenshots/configuration-editor/ix-settings.png

Figure 3.12.23 An Internet Exchange is similar to a provider and it requires configuration of the Router, the Probing IPs and the other usual attributes

Once the Exchange is configured, IRP will need to reset its BGP session towards the router interconnecting with the Exchange in order to retrieve the required routing table information about all peering partners. Once the BGP session is restarted IRP will start fetching Exchange routing tables.

Give IRP sufficient time (~10 minutes) to fetch Exchange routing tables before continuing configuration.

IRP offers peering partner autoconfiguration. It retrieves the list of Next Hops (peering partners) on the Exchange with the corresponding list of prefixes individual peers accept.

figure screenshots/configuration-editor/ix-peering-partners-settings.png

Figure 3.12.24 The list of peering partners for the Exchange shows details about each of them and offers access to IRP Autoconfiguration feature and PBR ruleset generator for the router

Use the Autoconfiguration feature to create an initial configuration for IRP. Review Exchange peering partners before starting the Exchange. Consider enabling periodic auto re-configurations for selected IX in order to update periodically the value of BGP session monitoring IPv4 address from the Exchange and to add new peering partners. Refer global.exchanges.auto_config_interval↓ and peer.X.auto_config↓.

Keep in mind that Autoconfiguration might tamper with all the changes you’ve made directly within IRP config files for peering partners. So, use Autoconfiguration sparingly after you’ve applied manual changes or try avoiding direct configuration file changes altogether.

Once the Exchange is configured correctly within IRP you will need to apply the PBR rules on the router(s). IRP includes the functionality to generate PBR rules for different router vendors. You will need to review the generated ruleset and apply it on the router manually.

figure screenshots/configuration-editor/ix-pbr-settings.png

Figure 3.12.25 PBR Generator

It is possible that the Autoconfiguration feature on the Exchange has been run with incomplete routing tables or that new peering partners have been connected. This is especially true for very big Exchanges. In this case it is possible that some peering partners are neither in IRP nor router configurations. When IRP detects such a case it raises a warning about newly discovered peering partners like in the image below. At this stage you will need to both re-autoconfigure IRP and extract the PBRs for the router.

figure screenshots/configuration-editor/ix-a-new-peering-partner-warning.png

Figure 3.12.26 Warning about new peering partners that have been added to the Exchange

After its creation and up to its complete configuration the Exchange is kept in a Stopped state. When both IRP and the Router PBRs are configured IRP is ready to start optimizing Exchange traffic too. Keep in mind that starting the Exchange will require a BGP session restart.

We must mention that before applying changes to IRP configuration the changes are validated. If errors are detected these will be flagged and the previous good configuration will be kept intact so you will have the option to review the erroneous data and correct.

3.12.7.3 Switch a provider from one router to another

Switching a provider from one router to another is possible via manual IRP reconfiguration only.

Use the following steps to switch the provider from one router to another:

Suspend the provider using IRP Frontend “Providers and Peers” configuration section. IRP withdraws all existing and stops new improvements towards shutdown provider(s).
You can remove traffic from the provider (by denying incoming/outgoing announces in the BGP configuration) and then physically re-cable the provider from one router to another. Then configure BGP session towards the provider on new router.
The PBR rule for the provider should be configured on new router (move provider’s PBR settings from the old router to the new one).
Change IRP box local PBR rules if any.
Check if PBR works properly using traceroute tool.
Modify (/etc/noction/irp.conf) assigned router for the provider (refer to peer.X.bgp_peer↓).
Modify (/etc/noction/irp.conf) SNMP interface/IP/community for the provider (refer to peer.X.snmp.ip↓, peer.X.snmp.ipv6↓, peer.X.snmp.interface↓, peer.X.snmp.community↓, peer.X.mon.snmp.ip↓, peer.X.mon.snmp.ipv6↓, peer.X.mon.snmp.community↓, peer.X.snmp.version↓, peer.X.snmp.seclevel↓).
Reload BGPd configuration (affected BGP sessions could be reset).
Re-activate the provider using IRP Frontend “Providers and Peers” configuration section
Check IRP BGP Internal and External monitor states. They must be UP.

3.12.8 SNMP hosts configuration

Starting with version IRP 3.7 SNMP is configured as SNMP hosts and provider specific SNMP settings are being deprecated.

SNMP hosts are nodes on the network that provide or read SNMP data. Multiple SNMP hosts are configured in this section of configuration and references to them are used from elsewhere in the configuration. SNMP hosts can be configured by navigating via the “Configuration → SNMP hosts” menu.

3.12.9 Notifications and Events

Notifications are messages sent to alert about some events registered by IRP. There are two prerequisites in order to start using notifications:

Configure event parameters and channels
Subscribe for event notifications

IRP can send notifications as SMS, email and SNMP Trap.

3.12.9.1 Configure notification and events

Events configuration changes default threshold values when an event is raised. The most relevant events are presented in the following figure. For example Overloaded by (%) indicates that this event will fire when the aggregated outbound bandwidth usage for all providers exceeds by 10% the configured limits. Refer section 4.1.10↓ for details.

figure screenshots/configuration-editor/events-configuration.png

Figure 3.12.28 Configuration editor: Events configuration

IRP uses a local email server to send emails. Still, email sent by this server might trigger filtering policies enforced by some third parties that might block some important event notifications. It is possible to provide the details of another email server on your network that is protected for this. The following figure highlights the available email channel configuration parameters. Besides specifying the server email configuration sets the sender of email messages that will show in receiver’s inbox.

Only SMTP servers without authentication are currently supported.

figure screenshots/configuration-editor/email-configuration.png

Figure 3.12.29 Configuration editor: Email configuration

SMS configuration is mandatory for sending SMS notifications.

IRP supports the following SMS gateways:

Twilio
Plivo

A valid account with a supported SMS gateway is required in order to finish configuration. Check the following figure for details:

figure screenshots/configuration-editor/sms-configuration.png

Figure 3.12.30 Configuration editor: SMS configuration

The settings are:

SMS gateway - choose one of the supported gateways.
Account ID - the public identifier assigned to your account by the SMS gateway. The following figure highlights this value for Plivo.
Max message size - the maximum length of the SMS text. The SMS will be trimmed by IRP before sending. Subsequently the SMS gateway will split the SMS into multiple parts if required.
Secret - the secret identifier assigned to your account by the SMS gateway. The following figure highlights this value for Plivo.
From phone number - the phone number that shows as sender on the receiver’s mobile device. Use a phone number supported by the SMS gateway.

Refer section 4.1.10↓ for complete details about configuration parameters.

Some SMS gateways enforce the From phone number and will reject numbers that do not comply with their policy.

figure screenshots/configuration-editor/plivo-account.png

Figure 3.12.31 Plivo account ID and secret

SNMP Traps configuration requires a list of destination IP addresses and an SNMP community parameter to deliver SNMP traps. The following figure highlights the basic configuration. There is a series of advanced configuration parameters like the SNMP Trap port and rate of packets. Refer section 4.1.10↓ for details.

figure screenshots/configuration-editor/traps-configuration.png

Figure 3.12.32 Configuration editor: SNMP Traps configuration

Web Hooks configuration requires only a Webhook URL to be provided. There is a series of advanced configuration parameters in case their default settings are not satisfactory. Refer section 4.1.10↓ for details.

figure screenshots/configuration-editor/webhooks-configuration.png

Figure 3.12.33 Configuration editor: Web Hooks configuration

The settings are:

Webhook URL - the unique URL the team is assigned by Web hook provider. An example for Slack.com is shown.
Avatar icon URL - optional parameter that designates an icon (usually PNG or JPEG image are supported) assigned to the Webhook bot in the channel.
Avatar emoji - an alternative avatar specified in the form of an emoji in “:robot-face:” format.
Bot name - optional bot name assigned to Webhook bot.

Refer section 4.1.10↓ for complete details about configuration parameters.

Web hooks support has been tested and verified to work for Slack.com API.

3.12.9.2 Subscriptions

Notifications are sent only if a valid subscription has been created. Subscriptions, similar to regular emailing of reports can be found under <Username> → Notifications menu.

Notifications page provides an overview of existing subscriptions with features to create, edit, delete them. Check the following figure for a preview.

figure screenshots/configuration-editor/notifications.png

Figure 3.12.34 Notifications

Use Add new subscription or Events drop-down to create a new subscription or edit, delete existing subscriptions by clicking corresponding icons. Creating or editing a subscription will bring up a pop-up like the one in the following figure.

Figure 3.12.35 Subscribe notifications

Subscription details include:

Topic - the title given to subscription in Notification page and also a textual part of email and SMS notifications.

If SMS notifications are used it is advised to keep the topic short so that there is enough space left to include details about the event.

Interval between notifications - sets a rate limit of how often an email and/or SMS notification is sent. Value ’No limit’ for the interval depicts no rate limits and all events will trigger a notification.

SNMP Traps notification are not constrained by the interval between notifications. SNMP Traps are sent immediately as they are raised.

The rate limit is enforced per subscription so it is still possible to receive multiple notifications if the subscribed events are part of different subscriptions.

Destinations - IP address of a trap receiver for SNMP Trap notifications, email addresses, phone numbers and webhook channels. Multiple destinations of same type can be provided. At least one valid destination must be provided per subscription.

IRP parses and recognizes whether a provided destination is a valid IP address, email address, phone number or web hook channel. Web hook channels for example a recognized by detecting that first character of a channel name is the “#” mark.

Event filters - allows filtering of events by textual description or by IRP component. Remove the filters to see all the subscribed events.
Subscribed events - the full list of events supported by IRP and tick marks for the events in this subscription. At least one event is required for a valid subscription.

3.12.9.3 Notification text

When a subscribed event is fired IRP will send notifications. A sample notification text (email) looks like the following figure:

figure screenshots/configuration-editor/notification-text.png

Figure 3.12.36 Subscribe notifications

The email sample above identifies the different parts of the message:

IRP server name
Subscription topic as specified in the subscription
From email address - as configured in email channel
Time - date-time of the last event that caused the notification. In case of rate limitation this might be older than the time of the email.
Textual description of the event and any relating details.
Older events in this subscription that have been retained due to rate limitations. Keep in mind that all past events are aggregated into a single email or SMS message.

3.12.10 Security Configuration

Security configuration covers restrictions on the ranges of IP addresses that can access IRP’s Frontend and also user’s allowed to access IRP as well as their roles.

3.12.10.1 Allowed ranges of IP addresses

IRP Frontend includes a feature to restrict access by IP address and this feature can be turned on or off as needed. When this feature is used allowed IP addresses are maintained as a list of prefixes in CIDR notation.

figure screenshots/configuration-editor/security-settings.png

Figure 3.12.37 Configuration editor: Security

3.12.10.2 Users and user directories

Users who can access IRP are managed in User Directories. User directory is a generic name for a external user management provider for example LDAP (either POSIX or Active Directory). IRP can interface with as many user directories as needed in a specific environment.

IRP recognizes Admin and User privileges that can be granted to users. Once a user logs into the system IRP creates a profile that maintains his personal configurations like dashboards or notifications.

figure screenshots/configuration-editor/security-UsersDirs.png

Figure 3.12.38 Configuration editor: Users and Directories

3.12.10.3 LDAP and Active Directory

LDAP or AD user directories can be added, updated and removed from IRP by accessing “Configuration → Security” menu item in the User Directories tab. Each user directory takes a series of parameters specific for the protocol.

All operations with DNs (initial bind DN, group DNs, user names) are case insensitive and also strip redundant whitespace.

Refer individual protocol documentation for how to correctly configure one or another user directory.

The example below offer a generic set of parameters required to configure IRP to use Active Directory for access management.

figure screenshots/configuration-editor/security-Dir-General.png

Figure 3.12.39 Configuration editor: LDAP User Directory configuration

The general tab covers:

User directory name - the name assigned to the directory within IRP,
User directory type - one of the supported User Directories,
Enabling or disabling a user directory,
Order specifies when this user directory will be examined by IRP compared to other user directories,
User directory hostname in the form of either IP address or domain name
User directory port

figure screenshots/configuration-editor/security-Dir-Advanced.png

Figure 3.12.40 Configuration editor: Users and Directories

Advanced configuration of a user directory covers:

Initial binding user name that IRP uses to authenticate itself,
Initial bind password assigned to IRP,

Usually bind passwords are not required to access directories. If a password is required and configured via IRP Frontend, it will be scrambled in IRP configuration files. If the password is specified directly in IRP configuration it is provided in clear-text with the condition that it does not start/end with scrambled password encapsulation characters.

Timeout before failing a connection to this user directory,
TLS use,
Certificate verification and
CA certificate used to verify server’s certificate.

figure screenshots/configuration-editor/security-Dir-Bindings.png

Figure 3.12.41 Configuration editor: Users and Directories

Bindings maps User Directory attributes to IRP specific attributes, for example:

Base DN and User DN specifies the root distinguished name and user subtree

IRP recognizes BOTH short and full user identifiers. Examples below are both valid directory entries that will match user "chris" with long name "Mr. Chris Smith":

cn: ops
uniqueMember: chris

and

cn: ops
uniqueMember: cn=Mr. Chris Smith,ou=employees,ou=People,dc=ops,dc=org

Admin role groups are user directory objects that list users with Administrator privileges within IRP as compared to all users. Both short and full paths are accepted for Admin role groups.
User role, Username, Email and Common name fields map User Directory attributes to IRP user attributes,
User filters can specify any valid LDAP search criteria (without outside parenthesis).

3.12.10.4 TACACS+ directory

TACACS+ user directories can be added, updated and removed from IRP by accessing “Configuration → Security” menu item in the User Directories tab. Each user directory takes a series of parameters specific for the protocol.

The example below highlights the parameters required to configure IRP to authenticate against a TACACS+ directory/host.

figure screenshots/configuration-editor/security-Dir-TACACS.png

Figure 3.12.42 Configuration editor: TACACS+ configuration

The parameters include:

User directory name - the name assigned to the directory within IRP,
User directory type - one of the supported User Directories in our case TACACS+,
Enabling or disabling a user directory,
Order specifies when this user directory will be examined by IRP compared to other user directories,
User directory hostname in the form of either IP address or domain name
User directory port
A shared secret used by IRP and TACACS+ host to secure communications.

3.12.10.5 Internal user directory

IRP includes an Internal user directory that allows multiple user accounts to be setup. User accounts management is done by accessing “Configuration → Security” menu item.

User management for external user directories is performed outside IRP using designated tools. The Internal directory is used when external user directories are not available or their use is not desirable.

To edit a specific user’s profile, press the Edit button next to it. To add a new user, click on the “Add” button. Specific users may be given Admin privileges. In such case they can also manage all system users.

The timezone specified in the user profile will be considered while displaying system Reports and Graphs.

figure screenshots/configuration-editor/frontend-add-user.png

Figure 3.12.43 Configuration: Adding a new user account

3.12.11 Frontend configuration

figure screenshots/configuration-editor/frontend-configuration.png

Figure 3.12.44 Configuration editor: Frontend

The currency used in various cost-related reports as well as the system logo (shown in the login form and on the menu bar) can be changed on the same page.

Currencies can be added/deleted manually by using the “Manage currencies” feature.

figure screenshots/configuration-editor/frontend-manage-currencies.png

Figure 3.12.45 Configuration Editor: Manage currencies

Only a Super admin user can change the currency setting. The currency is a global system value, and cannot be adjusted per-user.

The system logo is a global setting and can be changed by the Super administrator only.

The default currency is “USD - U.S. Dollar”.

A custom logo image can be uploaded by using the Upload button. Afterward, the image can be cropped to fit the standard logo size. The new settings are applied after clicking the “Submit” button.

3.12.12 Inbound configuration

Refer Inbound optimization↑ for an overview of Inbound feature.

Flow (sFlow, NetFlow) is a prerequisite for Inbound optimization. IRP collects and uses statistics about upstream provider incoming traffic targeting inbound prefixes. This statistics should be made available by properly setting up Flow and provider specific Flow agents (refer peer.X.flow_agents↓).

SPAN does not carry details about the specific interface packets enter your network and as such it is impossible to distribute inbound traffic targeting Inbound prefixes by upstream provider.

All Inbound related features, including configuration, moderation, and reports are listed under main menu Inbound:

figure screenshots/configuration-editor/InboundBetaMenu.png

Figure 3.12.46 Main menu: Inbound

3.12.12.1 Inbound prefixes

Note that optimization of transiting traffic feature does not rely on manually configured transit prefixes. Instead filters by ASN and/or prefix are used to constrain the specific prefixes that can be improved.

Inbound optimization manipulates how much traffic targets YOUR prefixes through different providers. In order to do this you need to configure IRP and let it know what are the Inbound prefixes. You can specify a subset of your network prefixes and you can also split larger prefixes into smaller ones in order to have a more fine-grained control over the Inbound traffic. These prefixes will be announced back to your originating routers marked with the community designated to prepends count to use when announcing to individual upstream providers.

Refer Inbound prefixes full control↓ for details regarding IRP announcement of inbound prefixes.

Splitting larger prefixes into subprefixes offers IRP the opportunity to better control the granularity of traffic shaping changes enabling improvement of smaller chunks of overall traffic. At the same time original prefixes can be announced as before and they will guarantee continuity of the prefix for cases when improvements announced by IRP become unavailable for some reason (for example if smaller prefixes are filtered in some networks).

Verify during Inbound configuration that the split inbound prefixes are not filtered by your providers and propagate as expected on the Internet.

IRP Frontend includes facilities that let you define the list of YOUR prefixes covered by Inbound traffic optimization as highlighted below and includes:

prefix belonging to the network (inbound.rule.X.prefix↓),
router(s) where announcements are sent (inbound.rule.X.bgp_peer↓),

Inbound improvements must be advertised by IRP to core routers that originate the prefix. This way the improvements will propagate consistently through your network to edge routers and beyond.

prefixstatus as each prefix can be disabled in order to exclude from furhter inbound optimization (inbound.rule.X.enabled↓),
next hop used by improvements for this prefix (inbound.rule.X.next_hop↓),
an option to instruct IRP whether to fully control the prefix or to only announce improvements when there are any. Refer the section Inbound prefixes full control↓ for further details.
providers for which this inbound prefix can be optimized (inbound.rule.X.providers↓). It must be noted that if no provider is selected then the prefix is optimized for all providers. Otherwise, the prefix will be optimized only through selected providers.

figure screenshots/configuration-editor/Inbound-prefixes.png

Figure 3.12.47 Configuration editor: Inbound prefixes

Inbound prefixes can be found both via Inbound main menu or Configuration → Collectors

Inbound prefixes full control

IRP operates under the assumption that the network already originates all network’s prefixes and only a subset are configured in IRP for inbound optimization purposes. This subset includes the intentionally split prefixes that allow better bandwidth control granularity by IRP. Under these assumptions IRP announces to the network only the improvements it makes. This avoids the overhead of announcing too often without real benefit for the network.

On some networks there might be a preference to guarantee that an inbound prefix is announced to all (allowed) upstreams at the same time thus making sure that they all have identical knowledge about the route with possibly different number of prepends.

Starting IRP 3.7-3 IRP can be configured to fully control inbound prefix announcements. When full control is configured for an inbound prefix, IRP:

always announces the prefix to the network even if no improvements are made for it and
marks the prefix/route for all allowed providers accordingly.

This guarantees that even when the network itself does not originate a prefix, the inbound prefix announced by IRP fully describes the route and provider preference to all upstream providers thus ensuring that all of them can route our traffic and the corresponding prepends count will influence what providers are preferred.

To enable inbound prefix full control either individual inbound prefix explicit value should be set in Configuration → Collectors → Inbound Prefixes

figure screenshots/configuration-editor/InboundPrefixControl-prefix.png

Figure 3.12.48 Configuration editor: Control of individual inbound prefix

or default IRP behavior should be specified in Configuration → BGP and Routers

figure screenshots/configuration-editor/InboundPrefixControl-bgpd.png

Figure 3.12.49 Configuration editor: Inbound prefixes control

3.12.12.2 Inbound prepends

Inbound optimization in IRP relies on BGP priority rules that generally elect best routes based on AS PATH length. By instructing your network to add or remove prepends for specific prefixes over different providers, IRP is able to influence how your inbound traffic flows. Inbound optimization in IRP is less effective for those regions of the Internet where your providers or other networks on the Internet use other policies for electing best routes. For example, Internet Exchange routes always take precedence for peers. This makes impossible prepend based inbound traffic optimization for Internet Exchanges. Often partial providers employ similar policies and inbound traffic flowing through them cannot be optimized.

Prepends are not added directly by IRP. Instead IRP inbound improvements are marked with designated communities that are recognized and acted upon by network routers.

Base communities

Inbound optimization improvements advertise larger or smaller counts of prepends to be announced to different providers. IRP uses BGP session to communicate to routers and relies on BGP communities to communicate the improvements. Each provider is assigned a base community which represents zero prepends. Additional prepends are represented by incrementing accordingly the right-side part of the provider’s community.

IRP uses 8 prepend levels. This means that base community should be chosen to allow additional 7 values without intersecting with other community values.

figure screenshots/configuration-editor/Inbound-ProviderBaseCommunity.png

Figure 3.12.50 Configuration editor: Base communities

Inbound base communities are assigned for each individual provider Configuration → Providers and Peers

Additionally routemaps are applied so that routers depict the designated communities in IRP’s improvements and change the announcement of the inbound prefix towards that provider appropriately. Different routemaps are produced and applied on each router in your network. See a few examples below. Request assistance from Noction’s support team to prepare exact routemaps for your network configuration and router models.

3.12.12.3 Routers configuration for Inbound

Complete sample configurations for Cisco, Juniper, Brocade and Vyatta will be provided by Noction Support on request.

3.12.12.4 Inbound Commit Control

Inbound bandwidth control is part of the Commit Control feature of IRP. You need to enable Commit Control and further to enable Inbound Commit control to make use of this feature. Besides the above mentioned settings for Inbound Commit Control separate low and high limits are configured to instruct IRP when to start and when to stop improving and also how to estimate the effects of inbound improvements.

figure screenshots/configuration-editor/Inbound-CommitControl-Config.png

Figure 3.12.51 Configuration editor: Inbound commit control

Inbound commit control can be found both via Inbound main menu or Configuration → Commit Control

Commit control for inbound as can be seen above screencapture highlights the relevant configuration parameters:

whether inbound commit control is on or off
whether review and moderation feature is enabled or disabled
what is the bandwidth estimation algorithm for inbound and
high and low limits for inbound in case the 95th mode warrants independent inbound vs outbound optimization.

Refer also to peer.X.95th.mode↓, core.commit_control.inbound.enabled↓, core.commit_control.inbound.moderated↓.

Estimating inbound traffic

Internet traffic has high variability and adjacent measurements can differ significantly between them. In these conditions a leveling function helps mitigate the risk of generating excessive numbers of Inbound improvements due to higher traffic variability of some networks. This estimation can be fine tuned by the Inbound bandwidth estimation algorithm parameter that tells IRP what value to use as the basis of the calculation. See the possible base values below:

figure screenshots/configuration-editor/InboundEstimationAlgo.png

Figure 3.12.52 Configuration editor: Inbound BW estimation algorithms

Moderated Inbound Improvements

IRP includes a moderation feature that allows review and approval of Inbound improvements. If this feature is enabled Inbound improvements are not automatically announced but instead they are queued for review. If an inbound improvement has not been approved in time for a subsequent cycle of improvements IRP will review the proposal itself and will adjust as needed thus making another recommendation.

See the capture below.

figure screenshots/configuration-editor/Inbound-Moderated.png

Figure 3.12.53 Configuration editor: Inbound BW estimation algorithms

Inbound Improvements moderation highlights the number of additional prepends proposed by IRP and allows submitting or rejecting individual improvements as well as a batch of selected improvements.

3.12.12.5 Optimization of transiting traffic

Optimization of transiting traffic is setup as part of Inbound Commit Control↑.

figure screenshots/configuration-editor/Transit-CommitControl-Config.png

Figure 3.12.54 Configuration editor: Inbound optimization of transiting traffic parameters

Inbound commit control can be found both via Inbound main menu or Configuration → Commit Control

Optimization of transiting traffic configuration parameters are highlighted:

Transiting traffic toggle that enables or disables the feature (global.inbound_transit↓)
Transit Improvements Max sets the maximum number of transit improvements (core.improvements.inbound_transit.max↓)
Transit ASNs and Transit prefixes specify those segments of the Internet that IRP monitors and optimizes (bgpd.prefixlist.asn↓, bgpd.prefixlist.prefixes↓)
Transit Improvements TTL min and max set the lower and upper bounds in seconds to keep a transit improvement (core.improvements.inbound_transit.ttl.min↓, core.improvements.inbound_transit.ttl.max↓)

Note that traffic belonging to a specific prefix that fragments a large transit prefix is collected as belonging only to the specific prefix and excluded from the larger prefix traffic statistics. If the specific prefix is also filtered from IRP configuration as for example is the case of a /26 prefix, then the specific prefix will be excluded from optimization of transiting traffic and will not show up in any of the subsequent decisions or reports.

Transit Top N prefixes sets the number of transit prefixes that are collected each cycle and considered for optimization (collector.flow.export.inbound_transit.topn↓)
Match transit at egress enables or disable statistics collection for transit prefixes when packets exist the network (collector.flow.process_transit_in_outbound↓)

3.12.13 Wizards

Wizards can be accessed from Configuration → Setup Wizards or from Configuration → Frontend → Configuration Wizards.

3.12.13.1 Initial Setup

IRP initial setup requires to:

Specify Infrastructure IP addresses and list of analyzed networks
Enable and configure Flow or Span Collector to gather network statistics
Set IRP Improvement mode
Setup the management interface
Indicate interfaces that IRP uses for active probing

figure screenshots/configuration-editor/wizards-initial-setup.png

Figure 3.12.55 Configuration editor: Initial Setup

Basic Setup
- Setup Infrastructure IP addresses (explorer.infra_ips↓)
- Setup Analyzed networks (collector.ournets↓)
  
  Figure 3.12.56 Configuration editor: Setup Infrastructure IP addresses and Analyzed networks
- Configure Collector:
  - Irpflowd
    - Irpflowd enable/disable(collector.flow.enabled↓)
    - NetFlow UDP port(collector.flow.listen.nf↓)
    - sFlow UDP port(collector.flow.listen.sf↓)
    - Flow Sources(collector.flow.sources↓)
  - Irpspand
    - Irpspand enable/disable (collector.span.enabled↓)
    - Irpspand interfaces (collector.span.interfaces↓)
    - Mindelay status (collector.span.min_delay↓)
    - Mindelay probing queue slots (collector.span.min_delay.probing_queue_size↓)
      
      Figure 3.12.57 Configuration editor: Configure Collector
- Improvement mode (global.improve_mode↓)
  
  Figure 3.12.58 Configuration editor: Improvement mode
- Set the managements interface (global.master_management_interface↓)
  
  Figure 3.12.59 Configuration editor: Management Interface
- Set the probing interface (global.master_probing_interface↓)
  
  Figure 3.12.60 Configuration editor: Probing Interface
Providers Setup
- Add a router, see Add Router ↓
- Add two providers, see Add Provider ↓

3.12.13.2 Add Router

IRP communicates improvements to your edge routers. Add Router wizard guides router configuration through the following steps:

Identify the router and its AS
Router IP address to setup BGP session
BGP announcement attributes to distinguish and prioritize IRP improvements on this router

Check below the corresponding wizard steps:

Router name assigned within IRP for easy identification and
Autonomous System number of the network (bgpd.peer.X.as↓)

Figure 3.12.61 Configuration editor: Router name and AS
Router IPv4 addresses
- Local router IPv4 address (bgpd.peer.X.master_our_ip↓)
- Router IPv4 address (bgpd.peer.X.peer_ip↓)
  
  Figure 3.12.62 Configuration editor: Router IPv4 addresses
Router IPv6 addresses
- Local router IPv6 address (bgpd.peer.X.master_our_ipv6↓)
- Router IPv6 address (bgpd.peer.X.peer_ipv6↓)
- Router ID (bgpd.peer.X.slave_router_id↓)
  
  Figure 3.12.63 Configuration editor: Router IPv6 addresses
Router announcement attributes for IRP improvements
- Announced LocalPref value (bgpd.peer.X.master_localpref↓)
- Announced communities(bgpd.peer.X.master_communities↓)
- Announced MED value (bgpd.peer.X.med↓)
  
  Figure 3.12.64 Configuration editor: Router announcing

3.12.13.3 Add Provider

The wizard Add Provider covers the following:

Router - Identify the router and routing domain where the provider interconnects with your network
Provider - Specify what is a provider ASN, short name and description
IP addresses - Specify and assign provider network addresses that IRP will use
Commit Control - Optionally set provider usage thresholds to be used for Commit Control
SNMP host - Specify the SNMP host for this provider
External monitor - Indicate if an External monitor is used and designated external IP addresses used to verify this provider link status
Internal monitor - Indicate if an Internal monitor based on BGP session with provider will be used and the corresponding SNMP resource
Pre-checks - Validate given provider parameters before submitting them

Choose a Router (peer.X.bgp_peer↓)

Figure 3.12.65 Configuration editor: Choose a Router
Provider name
- Provider short name (peer.X.shortname↓)
- Provider description (peer.X.description↓)
  
  Figure 3.12.66 Configuration editor: Provider name
Provider IPv4 addresses
- Probing IPv4 address (peer.X.ipv4.master_probing↓)
- IPv4 diagnostic hop (peer.X.ipv4.diag_hop↓)
- Router next-hop address (peer.X.ipv4.next_hop↓)
- Router ASN (peer.X.ipv4.next_hop_as↓)
  
  Figure 3.12.67 Configuration editor: Provider IPv4 addresses
Provider IPv6 addresses
- Provider probing IPv6 address (peer.X.ipv6.master_probing↓)
- IPv6 diagnostic hop (peer.X.ipv6.diag_hop↓)
- Router nex-hop IPv6 address (peer.X.ipv6.next_hop↓)
- Router ASN (peer.X.ipv6.next_hop_as↓)
  
  Figure 3.12.68 Configuration editor: Provider IPv6 addresses
Provider Commit Control
- Provider 95th percentile (peer.X.95th↓)
- Provider cost per Mbps (peer.X.cost↓)
- Commit Control provider precedence (peer.X.precedence↓)
- Maximum allowed load per interface (peer.X.limit_load↓)
- Commit Control status for this provider (peer.X.cc_disable↓)
- Allow Performance/Cost Improvements within provider group members (peer.X.improve_in_group↓)
  
  Figure 3.12.69 Configuration editor: Provider Commit Control
Provider Monitoring setup IPv4
- Provider SNMP interface (peer.X.snmp.interface↓)
- Provider SNMP IPv4 address (peer.X.snmp.ip↓)
- Provider SNMP community (peer.X.snmp.community↓)
  
  Figure 3.12.70 Configuration editor: Provider Monitoring setup IPv4
Provider Monitoring setup IPv6
- Provider SNMP interface (peer.X.snmp.interface↓)
- Provider SNMP IPv6 address (peer.X.snmp.ipv6↓)
- SNMP community (peer.X.snmp.community↓)
  
  Figure 3.12.71 Configuration editor: Provider Monitoring setup IPv6
External Monitor setup IPv4
- External Monitor enable/disable (peer.X.mon.enabled↓)
- Router monitoring IPv4 address(peer.X.ipv4.mon↓)
  
  Figure 3.12.72 Configuration editor: External Monitor setup IPv4
External Monitor setup IPv6
- External Monitor enabled/disabled (peer.X.mon.enabled↓)
- Router monitoring IPv6 addresspeer.X.ipv6.mon↓)
  
  Figure 3.12.73 Configuration editor: External Monitor setup IPv6
Internal Monitor setup
- Internal Monitor enable/disable (peer.X.mon.enabled↓)
- Monitoring IPv4 address (peer.X.mon.ipv4.bgp_peer↓)
  
  Figure 3.12.74 Configuration editor: Internal Monitor setup
Provider pre-checks

Figure 3.12.75 Configuration editor: Provider pre-checks

3.12.13.4 Setup Commit Control

Step 1: Enable Commit Control

Providers that should maintain a target 95th limit are identified and the exact limits are set. Link upper limit represents 80-90% of interface capacity and will be pre-populated for you.

Commit control enables/disables the feature for a specific provider(peer.X.cc_disable↓)
95th target specifies the contracted bandwidth usage target in Mbps (peer.X.95th↓)
Link upper limit specifies the maximum allowed bandwidth on the link in Mbps (peer.X.limit_load↓). This represents ~80-90% of interface capacity and will be pre-populated for you. Change it as appropriate.

figure screenshots/configuration-editor/wizards-setup-cc-1-basic-setup.png

Figure 3.12.76 Configuration editor: Commit Control basic setup

Step 2: Provider precedence

Precedence indicates how important it is to keep the commit levels for different providers. Traffic will be unloaded to the provider with least precedence in situations when all other providers are overloaded. If two or more providers have equal precedence they will form groups.

Move the slider or use drag and drop to specify the order of providers from most important to least important.

figure screenshots/configuration-editor/wizards-setup-cc-2-precedence.png

Figure 3.12.77 Configuration editor: Commit Control provider precedence setup

Step 3: Groups configuration

Groups are formed by a set of providers (as configured in IRP) that share some common characteristics, for example a redundant link or a virtual link consisting of multiple physical links towards the same upstream provider. Depending on customer’s needs, groups can be configured to permit or forbid performance rerouting from one provider in the group towards another. If providers are grouped they will be assigned the same precedence.

Groups are optional and can be skipped.

figure screenshots/configuration-editor/wizards-setup-cc-3-groups.png

Figure 3.12.78 Configuration editor: Commit Control groups setup

Step 4: Commit Control Overview

The last step of the wizard is there to review the configuration. Last minute changes can be made to some parameters. When submitted the configuration changes will be validated and applied if correct.

Provider name (peer.X.shortname↓)
Commit control status for a specific provider (peer.X.cc_disable↓)
Configured 95th (peer.X.95th↓)
Upper limit (peer.X.limit_load↓)
Load balancing for groups
Improve in group (peer.X.improve_in_group↓)
Precedence (peer.X.precedence↓)

figure screenshots/configuration-editor/wizards-setup-cc-2-peers-overall.png

Figure 3.12.79 Configuration editor: Commit Control Provider overall

3.12.13.5 Setup Failover wizard

Step 1: Enable failover

Failover configuration has a series of mandatory global settings before enabling it.

Failover status (global.failover_role↓)
Slave IP address (global.failover_slave.ip↓)
Slave SSH port (global.failover_slave.port↓)
Failover timer in seconds (global.failover_timer_fail↓)
Failback timer in seconds (global.failover_timer_failback↓)

figure screenshots/configuration-editor/wizards-failover-enable.png

Figure 3.12.80 Configuration editor: Failover setup

Step 2: Probing IP configuration

Probing IPs must be assigned to all configured providers for both master and slave nodes (peer.X.ipv4.master_probing↓, peer.X.ipv4.slave_probing↓).

figure screenshots/configuration-editor/wizards-failover-probing.png

Figure 3.12.81 Configuration editor: Failover probing IPs for providers

Step 3: Router BGP session settings

Master and slave nodes of a failover configuration establish BGP sessions with all configured routers. The following parameters are required:

Local IP address for both master and slave (bgpd.peer.X.master_our_ip↓, bgpd.peer.X.slave_our_ip↓)
BGP session password for master and/or slave if any (bgpd.peer.X.master_password↓, bgpd.peer.X.slave_password↓)

figure screenshots/configuration-editor/wizards-failover-BGPsessions.png

Figure 3.12.82 Configuration editor: Failover BGP session settings

Step 4: BGP LocalPref and Communities

Master and slave nodes apply BGP attributes to announcements:

LocalPref for master and slave (bgpd.peer.X.master_localpref↓, bgpd.peer.X.slave_localpref↓)
Communities for master and slave (bgpd.peer.X.master_communities↓, bgpd.peer.X.slave_communities↓)

Master’s LocalPref value must be greater than slave’s LocalPref.

figure screenshots/configuration-editor/wizards-failover-localpref.png

Figure 3.12.83 Configuration editor: BGP announcement attributes

Step 6: Probing interfaces

Failover requires configuration of Management and Probing interfaces for both master and slave nodes (global.master_probing_interface↓, global.slave_probing_interface↓). Toggle between master and slave settings to specify values for both nodes.

figure screenshots/configuration-editor/wizards-failover-interfaces.png

Figure 3.12.84 Configuration editor: Probing and management and interfaces

< Previous | Next >

IRP Installation and Configuration Guide

3 Using IRP

3.1 Getting started

3.1.1 Accessing the system

3.1.2 System dashboard

3.2 Frontend structure

3.2.1 Sidebar

3.2.2 Failover status

3.2.3 Global search

3.2.4 Warning bars

3.3 Customizing the dashboard

3.3.1 Custom dashboard gadgets

3.3.1.1 Platform status

3.3.1.2 Interfaces status

3.3.1.3 List of improved prefixes

3.3.1.4 AS-Path problems

3.4 Reports

3.4.1 Platform overview

3.4.2 Platform performance

3.4.3 Loss Improvements

3.4.4 Latency Improvements

3.4.5 Probe conversion rates

3.4.6 Current improvements

3.4.7 VIP Improvements

3.4.8 Improvements to Exchanges

3.4.9 Historical records

3.4.10 Providers overall

3.4.11 Provider performance

3.4.12 Cost overview

3.4.13 Congestion and outages

3.4.14 Top volume prefixes

3.4.15 Top volume AS

3.4.16 Top problem AS

3.4.17 Prefix statistics

3.4.18 AS statistics

3.4.19 Country statistics

3.4.20 Bandwidth by peer

3.4.21 Current inbound improvements

3.4.22 Inbound Improvements history

3.4.23 Probes today

3.4.24 Monitors status

3.5 Graphs

3.5.1 Improvements by type

3.5.2 Performance improvements

3.5.3 Probed prefixes and Improvements

3.5.4 New and retry improvements

3.5.5 Improvements by probing source

3.5.6 Prefixes rerouted from provider

3.5.7 Prefixes rerouted to provider

3.5.8 Improvements by provider

3.5.9 Performance improvements by provider

3.5.10 Commit improvements by provider

3.5.11 Total and Improved traffic

3.5.12 Bandwidth usage by provider

3.5.13 Providers bandwidth usage

3.5.14 Bandwidth usage and improvements

3.5.15 Total bandwidth usage

3.5.16 Inbound traffic distribution

3.6 Routing Policies

3.7 Flowspec policies

3.8 Throttling excessive bandwidth use with Flowspec

3.9 Maintenance windows

3.10 Events

3.11 Troubleshooting

3.11.1 Traceroute

3.11.2 Looking glass

3.11.3 Whois

3.11.4 Prefix probing

3.12 Configuration editor

3.12.1 Global configuration

3.12.2 BGP and Routers configuration

3.12.3 Collector configuration

3.12.4 Core configuration

3.12.5 Commit Control configuration

3.12.6 Explorer configuration

3.12.7 Providers and Peers configuration

3.12.7.1 Providers configuration

3.12.7.2 Internet Exchanges configuration

3.12.7.3 Switch a provider from one router to another

3.12.8 SNMP hosts configuration