Three Mistakes in a Boat (To Say Nothing of the Outage)

Yesterday, on 12.11.2018 a BGP configuration mistake happened at Mainone Cable Company (AS37282), a Nigerian ISP. It mainly hit two content providers: Google (AS15169, AS36384, AS36492, AS43515) and Cloudflare (AS13335). Leaked routes were accepted by its direct upstream, China Telecom (AS4809), further advertised in Russia to TTK (AS20485) and finally learned by NTT (AS2914) in Europe. After reaching the Tier-1 providers level leaked prefixes propagated globally, redirecting traffic to unusual Europe-Russia-China-Nigeria route.

For the last 30 years basic idea behind the Internet’ design hasn’t changed - it connects people and services with each other. However, some authorities may have a different angle on what services their citizens should be able to connect to. A regulator might require ISPs to block off selected content or IP-address space for the end-users. How is that implemented? There are many options, but the most popular one is with the help of static routes, that may be propagated locally in BGP. Mistakes in this ‘local propagation’ have happened before: most notable was the YouTube hijack back in 2008, but less famous events were continually happening all over the decade. Today we observed another one, created by Iranian ISP that affected Telegram messenger.

Dear colleagues, we are glad to inform you that our team has finished integration with IRR data sources and ROA records. It should significantly increase the quality of hijacks detection, plus improve transparency of what is happening to route objects in different registries.

Recently, several severe routing incidents were spreading globally: hijack of the 5% of an entire IPv4 address space from Brazil, route leak between Russia and Asia through Kyrgyzstan, and at last, previous Friday there was an event that could lead to an outage of a significant part of all the BGP ecosystem. Fortunately, it didn’t happen.

A few days ago several cybersecurity resources reported details of an entirely malicious traffic redirection that combined DNS, and BGP hijacking. The primary goal of this attack was to steal money from different cryptocurrency wallets and services. Moreover, it was successful, since Amazon did not detect it in time. Today, on April 26, another significant incident happened that seems to be also unnoticed by the majority of players.

The situation we observed last week was both peculiar and strange when panic for Cisco Smart Install Protocol remote code execution vulnerability (cisco-sa-20160323-smi) started circling. There was confirmed botnet activity that was wiping configuration files exploiting this vulnerability and leaving a message “Don’t mess with our elections.” Moreover, there were rumors that significant amount of ISPs and even Internet segments get down due to this malicious actions.

Last week there were several notable network incidents, which were the result of a new method for DDoS attacks amplification, using memcached database. Several DDoS mitigation providers, including Qrator Labs and Akamai, have confirmed that they were hit by this new attack kind. The new type of DDoS attack was able to break the record and reach 1.3 Tbps bandwidth. As a reaction to this new threat, Qrator.Radar team has added detection of the open-to-world memcached database in our daily scan.

We built a tool to visualize network latency measured with RIPE Atlas.

If you are looking for services such as IP-transit, MPLS channels or DDoS mitigation you can choose from a variety of products. However, it is difficult to compare offers and companies regarding actual service quality. Some organizations compare market offers, but often they look at the market share or the company’s financial condition and other business metrics that are not necessarily relevant to the quality of a service per se. Also, most of these comparisons are not available free of charge.

Fortunately, the situation is changing. Recently we have been given an opportunity to create global scale measurements with services such as PlanetLab, NLNOG RINGand, of course, RIPE Atlas. RIPE Atlas has become the biggest measurement platform, with a rich API as the primary user interface. However, an output of API requests is not always human-readable; it still requires a set of tools on top of the API, to make data easily understandable. So we decided to work on a fix.

Moscow is famous for the traffic jams, with the governments continually fighting that particular problem. Nevertheless, the beginning of 2018 was marked with the new traffic bottleneck created with the help of BGP misdirection. At 12:01 UTC 17.01.2018, AS8901 belonging to Moscow City Government started leaking prefixes between its upstreams: the Rostelecom (AS12389) and Comcor (AS8732). Redirection peaked at 70000 affected prefixes.