The National Internet Segment Reliability Research explains how the outage of a single autonomous system (AS) might affect the global connectivity of an impacted region, especially when it involves the dominant Internet Service Provider (ISP) in that country. Internet connectivity at the network level is determined by the interaction between autonomous systems. As the number of alternative routes between ASes increases, so does the fault-tolerance and stability of the Internet in that country.

The global connectivity of any Autonomous System (AS), whether an international giant with millions of service consumers or a small regional ISP, depends on the quantity and quality of its paths to Tier-1 providers. Tier-1 refers to major transnational, often transcontinental, operators that provide global connectivity between continents and countries. These operators sit at the top of the provider hierarchy and, therefore, do not pay each other for IP transit services. However, within this “elite club”, there is no obligation to maintain mutual connections with each other. Only the market can motivate such companies to establish interconnections. Is this incentive sufficient? We will address this question below in the section dedicated to IPv6 connectivity.

For many ISPs at all "tiers", losing connection to even one Tier-1 peer would likely make them unreachable from some parts of the world.

The Methodology of Internet Reliability Measurement

Examining a case when an AS experiences network degradation, we want to answer the following question: "How many ASes in the same region would lose connectivity with Tier-1 operators and their global availability along with it?"

Throughout the years, we have modelled such a situation because, at the dawn of BGP and interdomain routing design, its creators assumed that every non-transit AS would have at least two upstream providers to guarantee fault tolerance in case one of them goes down.

However, the current reality is different with more than 41% of ISPs having connections with only one IPv4 provider. In IPv6 connectivity, the situation is even worse – with over 54%. Have transit ISPs ever failed? The answer is yes, and it happens with increasing frequency. The more appropriate question is "Under what conditions would a particular ISP experience severe service degradation we would call an outage?" If such problems seem unlikely, Murphy's Law may be worth considering: "Anything that can go wrong, will".

To model such a scenario, we apply the same model for the eighth consecutive year. The following steps were taken to rate AS reliability:

• For every AS in the world, we examine all alternate paths to Tier-1 operators with the help of an AS relationship model, the core of Qrator.Radar;
• Using the Maxmind GeoIP database, we matched countries to every IP address of every AS;
• For each AS and each country of presence, we assign a weight equal to the share of the address space of this AS advertised in this country;
• For each country, we assign a weight equal to the sum of the country weights of all ASs that are present in it;
• After that, we analyze the potential impact of one's outage on other ASes as well as their respective countries;
• In the end, for each country, we identify the AS with the greatest/largest impact on other ASes in their region. 

IPv4 Reliability

Overall trend in IPv4 stability

Year by year, we observe a positive global trend towards increased reliability and overall availability. The current 2023 year is no exception.

To illustrate this thesis, we counted the average and median values of IPv4 stability for all countries throughout the entire history of reporting (the past 8 years).

We display both the average and median values to smooth situations of sharp declines in stability in countries with a small number of local ASes, which intensifies the dependency of our metrics on each of them. Such a representation of reliability is also necessary to offset cases where a 100% monopoly of an AS within a country significantly influences the assessment of the overall average stability worldwide.

It can be observed that the world continues to show a trend of increasing fault tolerance: the average reliability indicator has improved from 26.7% in 2022 to 25.7% in 2023.

IPv4 Ranking

Below is the Top 20 list of countries in terms of reliability, considering the outage of a single Autonomous System. In practice, this means that the country has good Internet connectivity, and the percentage reflects the share of Autonomous Systems that would lose global connectivity in case of failure of the most significant ISP.

Table 1. IPv4 Reliability Rating

Highlights:

• Singapore dropped 11 positions, falling from 4th to 15th place with the change of the critical AS from StarHub (AS4657) to SingNet (AS3758).
• Taiwan is back in the Top-20 of the IPv4 Reliability Rating at 17th place (took 20th place in 2021).
• After a 7-year absence, the Philippines re-entered the Rating at 20th position, having been at 24th place in 2022.
• The United States re-entered the rankings, moving from 28th to 18th place.
• Luxembourg exited the Rating, dropping from 16th to 39th place, and its critical AS changed from AS174 to AS6661.
• Indonesia lost four positions from the 17th place and exited the Rating.

The Internet Reliability Rating has consistently demonstrated a stable trend of systematic improvement in connectivity. Brazil, Germany, and the Netherlands have firmly held the top three positions for two consecutive years. This indicates that the number of connections between their Autonomous Systems is so extensive that the failure of a critical AS would not lead to the unavailability of these regions.

A notable observation of 2023 is the sharp decline in Luxembourg's ranking, which not only exited the Top 20 but also plummeted to 39th place. Its’ critical AS changed from the international Tier-1 operator Cogent (AS174) to the local Post Group Luxembourg (AS6661), which, in turn, has a significant number of local clients. In 2023, several clients of Post Group Luxembourg opted out of all other providers, having this AS as the only provider and thus becoming entirely dependent on its routing policies. Consequently, the criticality of Post Group Luxembourg outage for the country significantly increased, and due to the small size of the state, this event led to a colossal decrease in its Internet fault tolerance.

In Singapore, which lost 11 positions in 2023, there was also a change in the Autonomous System (AS) from StarHub (AS4657) to SingNet (AS3758), which has a higher level of criticality. The higher the criticality level, the more Autonomous Systems (and, in general, end-users accessing the Internet) will be left without connectivity In case of failure of this ISP, and the lower its country position is in the overall ranking.

Most likely, the increased criticality of SingNet is due to several Autonomous Systems leaving StarHub and having SingNet as the only provider, and its failure threatens them with a complete loss of connectivity.

On the other hand, looking at the largest providers in Singapore, it is noteworthy to mention a nationwide outage in the Australian subsidiary of SingTel (AS7473) – Optus (AS7474) – in November 2023. According to our data, Optus is the fourth AS in the country by the number of local clients, and users of these clients’ ASes experienced issues with internet services for almost 14 hours. Additionally, from a BGP perspective, Optus does not have backup providers but only one upstream ISP – SingTel.

The Philippines stands out among the newcomers in 2023, steadily improving its position over the last 4 years and now finally making it to the Top 20 in the Rating.

IPv6 Reliability

IPv6 adoption

It is well known that despite the similar names of the IPv4 and IPv6 protocols, their implementation is quite different. The relative novelty and substantial differences in the operation of these two protocols, of course, slow down the pace of IPv6 routing implementation on the Internet. This is confirmed by Google's IPv6 usage statistics, which shows that the adoption of this protocol worldwide is growing, but not exponentially, which is common for many Internet technologies, but linearly (the graph illustrates the percentage of all sessions using IPv6 to connect to Google servers). However, over the 8 years of our report's publication, this parameter has grown from 10 to 44.87% in 2023.

Partial Connectivity in IPv6

One interesting phenomenon in IPv6 is the partial ISP connectivity. Let's try to explain what this metric is about. To do this, let's consider the connections between major Tier-1 operators, depicting the Core of the Internet as a graph:

IPv4 Tier-1 Connectivity (left) and IPv6 Tier-1 Connectivity (right)

The dashed line on the IPv6 graph reflects the absence of links between major Tier-1 providers.

While in IPv4, all Tier-1 operators have peering relationships with each other, in IPv6, there is a lack of connectivity between several major operators such as:

• between Hurricane Electric (AS 6939) and Cogent (AS 174),
• between 2828 (XO Communications, merged by Verizon) and 3320 (DTAG),
• between 2828 (XO Communications) and 1239 (Sprint),
• between 2828 (XO Communications) and 6762 (Sparkle aka Telecom Italia).

In previous reports, we have already written about the "classic" absence of a link between 174 (Cogent) and 6939 (HE), which, due to the ongoing peering war, refused to establish a peering connection.

However, this year the link briefly appeared, giving some hope for a restoration of relations between companies that have been competing for years.

Also, some fluctuations in Verizon's connections in 2023 appeared:

• 2828 USA - 3320 Germany - the link was observed in 2019 but disappeared the same year,
• 2828 USA - 1239 USA - the link flickered and disappeared in May 2022,
• 2828 USA - 6762 Italy - the link periodically appears.

What does the absence of a link mean? If we consider only vertical relationships (provider-client), this means that traffic cannot flow between the sets of clients ("client cones") of two top-level operators because they do not exchange traffic with each other.
It is not possible to pass through a chain of more than two Tier-1 operators or a lower-tier transit operator because in this case, the Valley-Free routing principle would be violated, and route leaks would occur.

The Valley-Free principle is a widely accepted routing rule, according to which prefixes received by the considered AS from a provider or a peer can only be passed on to clients.

The absence of routes from AS in the region to at least one of the major Tier-1 operators leads to the phenomenon of partial connectivity. This may result in unavailability of favorite services for end-users.

We added information about the partial connectivity of all ASes in each country in the IPv6 Reliability Rating provided below.

Table 2. IPv6 Stability Score

Highlights:

• The Top 2 remain unchanged for the second year in a row: Brazil and Germany maintain their top positions. 
• The Netherlands gained 5 positions, securing the third spot and pushing England to the 4th position. 
• Japan, absent from the Top 20 since 2021, entered the ranking at the 5th position. 
• China landed in the 12th position after a two-year absence from the Top 20. 
• Indonesia lost 13 positions, dropping sharply from 4th to 17th place.

The Japan’s comeback to the ranking after a 2-year absence can be explained by the criticality level reduction of its most significant telecommunications operator, the National Institute of Informatics (AS2907), from 20.27% to 4.33%. The number of clients of this AS did not decrease, indicating that they started connecting backup communication channels (other providers). In case of a failure of the critical AS, their availability will not be lost, meaning more stability and a higher position in the ranking.

China reappeared in the ranking after being in the Top 20 in 2021. The country exhibits a paradox: on the one hand, the percentage of failure in case of China Telecom outage (AS4134) is very low (5.85%), but on the other hand, there is an astounding partial unavailability (almost 90%).

High partial unavailability in China is due to a large number of autonomous systems that announced prefixes belonging to the China Education and Research Network, CERNET (AS4538). These announcements were made from usually unused but APNIC-allocated Autonomous Systems, creating a large number of BGP Hijacks, as was mentioned in the Q3 2023 report in the BGP incidents section.
In terms of our global connectivity metrics, it is worth noting that the spread of these announcements towards Tier-1 operators ends in two key ASes: Hurricane Electric (AS6939) and Hong Kong Internet Exchange (AS4635).

In the case of Hong Kong IX, this implies regional connectivity due to the specifics of traffic distribution through IX and does not affect global connectivity. In the case of Hurricane Electric (AS6939), global connectivity will not be achieved since, as mentioned above, there is no peering link between AS6939 and AS174. This means that a large number of AS that announced Chinese CERNET prefixes had partial availability, according to our metric. Moreover, upon the short-term connection between AS6939 and AS174, global connectivity appeared, and many partially available ASes became globally available through AS6939, making AS6939 critical for the country (with 88% criticality).

It is important to note that emergence of a large number of rarely used ASes may add noise to the values of our metrics for China, but it does not reflect the actual situation. We just reflect the fact of the global connectivity lack for these rarely used ASes. This phenomenon is unlikely to affect regular users.

The sharp drop in Indonesia's ranking can be explained by the fact that its Autonomous System Cyberindo Aditama (AS38158) started actively beating up for clients in 2023, including local ones, which made it a more critical Autonomous System for the region, displacing last year's leader PT Mora Telematika Indonesia (AS23947).

The almost annual change of leaders indicates the dynamism of IPv6 development. Unlike IPv4, the average stability and partial connectivity for IPv6 are not improving. However, the Top 20 Rating provides evidence of improvements in the average outage rate — from 7.06% in 2022 to 5.34% in 2023.

ISPs With Only One Upstream (Stub networks) and Their Reliability

For several years in a row, we have observed the fact that a significant part of critical ASes from our rankings consists of Autonomous Systems with a large number of Stub ASes (Stub Autonomous Systems) which are essentially networks with only one upstream provider and not transit providers for anyone.

Below are the tables where critical AS from the countries in our ranking simultaneously act as major "hubs" for stub networks.

75% of our ranking for IPv4 address space have these characteristics, while for IPv6, it is less than half.

Qrator.Radar Website Updates

Every year, new functionality is added to the Qrator.Radar website. This year, we made a remarkable improvement having provided national stability ratings with monthly updates so you can access the latest data at your convenience on our website.

Key findings

• The reliability of IPv4 continues its year-by-year improvement showing growth from 26.7% in 2022 to 25.72% in 2023.
• The global adoption of IPv6 continues to linearly grow; however, the partial connectivity in IPv6 remains constant and does not show a steady increase. 
• The Philippines demonstrates a consistent increase in the overall IPv4 stability.
• The almost annual change in IPv6 leaders indicates the dynamic development of IPv6.
• The high partial IPv6 connectivity in China is associated with the local announcement of CERNET prefixes on behalf of underutilized Asian ASes.
• Up-to-date information on national stability is always available on the Qrator.Radar website.

 

In case of any questions, feel free to contact us at radar@qrator.net