Tier 1 carrier performance: June, 2017 snapshot

Tier 1 carrier performance: June, 2017 snapshot

    This analysis covers widely used Tier 1 carriers performance results in the US for the month of June 2017. Rare datapoints from distant locations have not been taken into consideration.

    The presented analysis is based on more than 725 million successful probes that span the entire month. All data is aggregated per carrier on a daily basis and accounts for many thousands of successful probes. A control group (labeled C) is used as a base of comparison. The control group aggregates the average for all transit providers in a network, including Tier 1 carriers.

    average loss and latency

    Fig. 1. Average Loss and Latency in June 2017
    The numbers include a control group C (gray) to allow cross comparison.

    The values for May 2017 are included for cross comparison.

    low latency

    Fig. 2. Average Loss and Latency in May 2017
    The charts include a control group C (gray) to allow cross comparison.


    Average packet loss analysis:

    • The averages suggest that NTT, Level 3 and Zayo are the best performers for the analysed group in terms of packet loss. Cogent and XO slightly exceed the average for the control group level. Other Tier 1s have registered poorer results than the control group level;
    • NTT, GTT and Centurylink results present a stable trend in terms of packet loss for the last months, registering only insignificant changes. Telia and Hurricane bring to the table worse results than the ones for May. The decline for Telia is quite obvious.

    Average latency analysis:

    • Based on the control group level in June, 2017, lower average latency has been registered for: Centurylink, Level 3 and NTT. Other Tier 1s showed poorer results than the control group. The worst result has been reported by Hurricane;
    • Compared with May’s results, a significant change in average latency has been noted for the majority of the analysed Tier 1s. The control group has succeeded in diminishing its latency level, dividing the analysed group into two categories: the better performing and worse performing carriers. Centurylink, Level 3 and NTT have been placed in the “better performing” category. The rest of the carriers are in the “worse performing” category. We must mention, however, that XO, NTT and Zayo showed the same average latency as in the month of May. Hurricane registered the worst result in terms of latency among the analysed carriers.

    The charts below illustrate the performance of each carrier in comparison to the control group.

    packet loss and latency in April 2017

    Fig. 3. Better or worse Loss and Latency in June 2017
    The numbers are differences from average control group.

    worse latency

    Fig. 4. Better or worse Loss and Latency in May 2017
    The numbers are differences from average control group.


    In comparison with the control group for the month of June, 2017:

    • Centurylink, Level 3 and NTT showed better results in terms of Latency;
    • Level 3, NTT and Zayo showed better results in terms of Loss.

    In comparison with the data from May, 2017:

    • Tier 1s showed a radical changes in registered performances. The leading positions in terms of Loss have been kept by: Level 3, NTT and Zayo, as in terms of Latency: Centurylink, Level 3 and NTT.
    Loss

    For the Loss analysis we use a scatter plot, where average values by control group are assumed on the diagonal while the horizontal and the vertical axis highlight carrier metrics. All datapoints below the diagonal represent the better performing carriers and vice versa.

    packet loss analysis

    Fig. 5. Loss values spread on average diagonal
    Datapoints comparison with diagonal.


    Abnormally large losses are still registered for a large number of datapoints. As was mentioned in previous reports we consider excessive an average above 4.5% packet loss.
    Given the fact that Tier 1 carriers are characterized by both low loss values for some networks and abnormally high losses for other networks, the conclusion is that high loss values are not caused by the carriers themselves but rather are caused by the networks they service Or the networks they peer with. Whether the true cause is poor design, over-provisioned links or deficiencies in peering governance – this report cannot tell. What we can mention is that for many networks, whether permanently or sporadically, there is definitely an opportunity to improve things.

    network loss

    Fig. 6. Better or worse carrier loss (%)
    Average placed on the zero line


    A different representation of the above data places it around the control group (zero line) with gain values by carrier. Values are sorted and charted from left to right by increasing average loss. The chart depicts gains or worsening on a network based on the average control group’s performance – values are shown from left to right following better to worse loss values. The assumption of this analysis is that while a network’s conditions might be better or worse compared to other networks, the conditions tend to be equal across all carriers including the control group. While the carrier’s network is not the culprit causing additional loss, this analysis might be able to suggest whether those carriers peering with remote regions is deficient. Non-systemic issues with carriers will tend to cancel out with values being scattered equally above or below the zero line while systemic issues or gains will have a tendency to place a carrier consistently above or below it. The scatter plot highlights this assumption.
    More so, if we average gains or losses compared with the control group we expect the noise to cancel out.

    datapoint packet loss

    Fig. 7. Average packet loss gains/losses by carrier June 2017
    Averages determined for ALL datapoints or a cutoff at 4.5% control group applied.

    average packet loss

    Fig. 8. Average packet loss gains/losses by carrier May 2017
    Averages determined for ALL datapoints or a cutoff at 4.5% control group applied.


    A significant change in average Loss for the month of June, compared with May, has not been registered for most carriers. All Tier 1s showed roughly similar results to the previous month for averages determined for ALL datapoints and the cut off at the applied 4.5% control group. The only change has been registered by NTT which went from losers to winners.

    As in May, worst position are hold by Hurricane, Telia and Cogent.
    Theut off at the applied 4.5% control group conditioned a change in gainers structure, Zayo left the group of Tier 1s with better results. XO, contrariwise, acquired  better position visibly improved its results.

    Latency

    For Latency analysis we use a similar scatter plot to the one we used for Loss. It displays control group values on the diagonal while highlighting individual carrier measurements on the horizontal and on the vertical axis. Datapoints placed significantly and consistently below the average highlight better performing carriers while datapoints above the average highlight worse than average performance.

    clusters datapoints

    Fig. 9. Carrier latency with average group on the diagonal
    Clusters of datapoints below diagonal highlight better performance


    The results above linger around the diagonal with the following observations:

    • NTT, GTT and Level 3 are spotted on the far right of the graph that depict very long haul traffic. heir average latency is significantly below the average;
    • Zayo, Hurricane, XO,Telia and Cogent have been mostly present within the latency diapason of 85 – 180 ms;
    • Centurylink has been mostly registered within the latency diapazon of less than 150ms.

    latency by carrier

    Fig. 10. Average latency gains/losses by carrier Values averaged for the difference between carrier performance and the average group in that network.


    The differences in latency above from the control group are averaged with the expectation that better or worse performance will cancel out if the differences are caused by measurement noise.

    The results show that during the month of June, 2017 in comparison with May, 2017:

    • Hurricane drifted to the negative category, adding ~0.1 ms to RTT for each packet a network forwards through these carriers;
    • The leadership position in average RTT reduction has been obtained by Zayo with ~ 4.6 ms, followed by Centurylink ~1,4 ms, GTT, Telia and XO ~0,8 ms each.
    Appendix. Carrier Loss (highlighted)

    Loss improvement/worsening highlighting Centurylink datapoints.
    Centurylink datapoints loss in June

    Loss improvement/worsening highlighting Cogent datapoints.
    Cogent datapoints loss in June

    Loss improvement/worsening highlighting GTT datapoints.
    GTT datapoints loss in June

    Loss improvement/worsening highlighting Level 3 datapoints.
    Level3 datapoints loss in June

    Loss improvement/worsening highlighting NTT datapoints.
    NTT datapoints loss in June

    Loss improvement/worsening highlighting Telia datapoints.
    Telia datapoints loss in June

    Loss improvement/worsening highlighting XO datapoints.
    XO datapoints loss in June

    Loss improvement/worsening highlighting Zayo datapoints.
    Zayo datapoints loss in June

    Loss improvement/worsening highlighting Huricane.
    Huricane datapoints loss in June

    Appendix. Carrier Latency (highlighted)

    Latency spread chart highlighting Centurylink.
    latency Centurylink in June

    Latency spread chart highlighting Cogent.
    latency Cogent in June

    Latency spread chart highlighting GTT.
    latency GTT in June

    Latency spread chart highlighting Level 3.
    latency Level3 in June

    Latency spread chart highlighting NTT.
    latency NTT in June

    Latency spread chart highlighting Telia.
    latency Telia in June

    Latency spread chart highlighting XO.
    latency XO latency in June

    Latency spread chart highlighting Zayo.
    latency Zayo in May
    Latency spread chart highlighting Huricane.
    latency Huricane in June

     


    Disclaimer*: The data presented in this report card is intended for information purposes only and is not to be interpreted as any form of promotion or debasement for carriers herein named. Information is obtained from the Intelligent Routing Platform Lite instances, where the compulsory consent of the legal entities for collection of such information is part of the Terms and Conditions document. For privacy protection, the exact location and number of IRP Lite instances are not provided.


    Boost BGP Preformance

    Automate BGP Routing optimization with Noction IRP

    NO COMMENTS

    Leave a Reply