Authors: Ramakrishna Padmanabhan (IODA - CAIDA, UC San Diego), Alberto Dainotti (IODA - CAIDA, UC San Diego), Nima Fatemi (Kandoo), Arturo Filastò (OONI), Maria Xynou (OONI), Simone Basso (OONI)

Over the last week, Iran experienced a nation-wide Internet blackout.

Most Iranians were barred from connecting to the global Internet and the outside world, but they had access to Iran’s national Intranet: the domestic network that hosts Iranian websites and services—all under the government’s watch.

This major Internet blackout was rolled out on 16th November 2019, right after protests erupted across multiple cities in Iran. The protests (against economic mismanagement and government corruption) were sparked by the government’s abrupt announcement to increase the price of fuel (as much as 300%) and to impose a strict rationing system. According to Amnesty International, more than 100 protesters are believed to have been killed over the last week, but this figure has been disputed by Iranian authorities. Amid the protests—which began on 15th November 2019 and are ongoing—access to the Internet was reportedly shutdown.

As of 21st November 2019, Internet access is gradually being restored.

In this report, we share data on the Internet blackout in Iran. We also share technical observations for connecting to the Internet from Iran during the blackout.

Nation-wide Internet blackout

Iran’s nation-wide Internet blackout is evident through several data sources, all of which show that it started on 16th November 2019.

IODA data

The Internet Outage Detection and Analysis (IODA) project of the Center for Applied Internet Data Analysis (CAIDA) measures Internet blackouts worldwide in near real-time.

IODA data highlights three interesting aspects of Iran’s internet blackout:

1. Cellular operators in Iran were disconnected first;
2. Almost all other providers in Iran followed suit over the next 5 hours;
3. Providers appear to have used diverse mechanisms to enforce the blackout.

In order to track and confirm Internet disruptions with greater confidence, IODA uses three complementary measurement and inference methods: Routing (BGP) announcements, Active Probing, and Internet Background Radiation (IBR) traffic. The routing announcements from BGP allow us to track control-plane Internet reachability in an area, whereas Active Probing and IBR shed light on different aspects of data-plane reachability. These methods result in connectivity “liveness” signals, whose status (for each country) is always publicly visible in the IODA dashboard.

IODA data clearly shows drops in signals from Iran on 16th November 2019, as illustrated in the figure below.

Source: Internet Outage and Detection Analysis (IODA): Iran (Nov 16 to Nov 18)

Unlike other country-wide blackouts (such as those in Syria and Iraq), the signal drops did not all occur simultaneously and the magnitude of the drops across the three different signals varied. For example, the first drop occurred at around 14:00 UTC (17:30 local time in Iran), when the BGP signal dropped by 33% (with the number of globally visible /24s reducing by 15,000). However, the drops that the Active Probing and IBR signals experienced at that time are almost negligible.

Then, from around 17:00 UTC on the same day, the Active Probing and IBR time series dropped steadily until around 19:00 UTC. After 19:00 UTC on 16th November 2019, data-plane connectivity remained low until 09:00 UTC on the morning of 21st November 2019, after which the Active Probing time series began a slight trend upwards.

From the evening of 16th November 2019 to the morning of 21st November 2019, there was significantly reduced data-plane Internet connectivity, although notably, some connectivity continued to persist even during this time. Control-plane connectivity ostensibly remained at more than 50% of the pre-blackout connectivity throughout this time.

In the next section, we show that these differences in signal-drop-patterns among the three IODA data sources are likely due to both the adoption of diverse disconnection mechanisms and large differences in the timing of their execution by various Iranian Internet Service Providers (ISPs).

Network-level differences

IODA data clearly shows drops in signals for various network operators in Iran on 16th November 2019, as illustrated below (see legend on the top-right of the chart to refer to each network operator).

Source: Internet Outage and Detection Analysis (IODA) data chart generated using a script

We observe significant differences in how different Autonomous Systems (ASes) were affected by the shutdown.

First, there are timing differences—some networks lost Internet connectivity a few hours before the others. For example, three large cellular ISPs—MCCI (AS197207), IranCell (AS44244), and Rightel (AS57218)—lost connectivity at around 14:00 UTC (17:30 local time in Iran). These observations perhaps indicate that cellular operators in Iran were instructed to shut down the Internet first (before fixed-line operators were instructed).

Other ASes start losing connectivity later in the day at around 17:00 UTC (20:30 local time in Iran), but there are timing differences among these ASes too. For example, the Active Probing and IBR signals for Shatel (AS31549) drop sharply at around 17:00 UTC (20:30 local time in Iran), but these signals for ParsOnline (AS16322) drop an hour later at around 18:00 UTC (21:30 local time in Iran). These differences in timing suggest that individual operators enforced the blackout independently (as opposed to a single “kill-switch” that resulted in the simultaneous disconnection of all operators).

Second, we observe dissimilarities in how the network-level blackouts for different networks manifest in IODA’s signals. These dissimilarities may reflect the use of different approaches by operators for disconnecting their networks.

The three cellular ISPs’ blackouts are visible as drops in BGP reachability at roughly the same time (~14:00 UTC); these drops are accompanied by simultaneous drops in the IBR signal for these cellular ASes as well. These results suggest that cellular operators perhaps executed the blackout in their networks by blocking control-plane reachability.

Other large Iranian ASes’ IODA signals exhibit differences compared to the cellular ASes’ signals, and these differences are not limited to the timings at which drops occur. For example, ITC (AS12880) has no drop in the BGP signal on 16th November 2019, but the Active Probing and IBR signals drop to nearly 0 at around 18:00 UTC (21:30 local time in Iran) on that day.

On the other hand, Iran Telecom Co (AS58224) experiences not one, but three different drops in the BGP signal (16:30, 18:30, 20:00 UTC), only two of which are accompanied by drops in the other signals. The Active Probing and IBR signals for ParsOnline (AS16322) drop to nearly 0 at 18:00 UTC on 16th November 2019, but surprisingly, at around 20:30 UTC on the same day, there is an increase in the IBR signal. These results show that there can be disparities in the effects of the blackout across networks.

Signs of recovery

At around 10:00 UTC on 21st November 2019, we observe a spike in Active Probing measurements.

Source: Internet Outage and Detection Analysis (IODA): Iran, Recovery of internet outage by network

At around 10:50 UTC on 23rd November 2019, we can see another uptick in the Active Probing IODA signal, showing that significantly more networks were now reachable from the outside world.

Source: Internet Outage and Detection Analysis (IODA) data chart generated using a script

To put it into perspective, below is a chart that shows the IODA signals for Active Probing before the outages and during the latest recovery stage.

Source: Internet Outage and Detection Analysis (IODA) data chart generated using a script

This was also further confirmed with direct measurements from local vantage points which showed that connectivity was being restored on many more networks. We saw an increase in the amount of OONI measurements coming from Iran in the morning of 23rd November 2019 (see sections below).

Other data sources

Iran’s Internet blackout is also confirmed by several other data sources, such as Google traffic data, Tor Metrics (statistics on the use of Tor software, which is used for online privacy, anonymity, and censorship circumvention), and Oracle’s Internet Intelligence, as well as by NetBlocks and Cloudflare reports.

Google traffic data, illustrated below, shows an abrupt disruption of Google search traffic originating from Iran at around 13:00 UTC on 16th November 2019.

Source: Google Transparency Reports: Traffic and disruptions to Google

As demonstrated by the above chart, the Internet blackout in Iran appears to be ongoing since barely any Google search traffic has originated from the country since 16th November 2019. Similarly to IODA data, Google traffic data shows an uptick on 21st November 2019, with more traffic gradually being restored from 22nd November 2019 onwards.

Tor Metrics below illustrate that the number of Tor users in Iran dropped abruptly on 16th November 2019.

Source: Tor Metrics: Directly connecting users from Iran

Similarly, we observe a sudden drop in the use of Tor bridges (which are used to circumvent the blocking of the Tor network) in Iran on 16th November 2019 as well.

Source: Tor Metrics: Bridge users in Iran

Oracle’s Internet Intelligence measures Internet outages worldwide by tracking the drops in the routed BGP prefixes, completed traceroutes, and DNS queries. These metrics suggest the presence of an Internet disruption when they are significantly reduced from baseline values of normalized counts of successful traceroutes, volumes of DNS queries, and absolute counts of globally routed BGP prefixes.

The following chart, taken from Oracle’s Internet Intelligence Map, suggests the presence of an Internet blackout in Iran (which they also confirmed in their relevant publication a few days ago) since there was a significant drop in completed traceroutes and routed BGP prefixes.

Source: Oracle Internet Intelligence Map: Iran

It’s worth highlighting, however, that DNS requests increased recently, showing that DNS traffic is able to reach the Internet.

Throughout the course of Iran’s Internet blackout, we observe some amount of traffic originating from some networks in Iran. The fact that IODA traffic, for example, hasn’t dropped completely suggests that Iran is not experiencing a total Internet blackout, as there are still some connections to the Internet. This is also confirmed through OONI measurements which were collected from Iran over the last week, despite the Internet blackout.

The following chart shows the number of OONI measurements collected from 49 different networks in Iran between 14th November 2019 to 23rd November 2019.

Source: OONI measurements: Iran (from OONI’s fast-path pipeline)

As OONI Probe requires Internet connectivity in order to perform tests (and submit test results for publication), the collected OONI measurements from Iran over the last week demonstrate that the Internet blackout was not total, since some people were still able to connect to the Internet on some networks.

Throughout the Internet blackout (from 16th November 2019 to 23rd November 2019), OONI Probe was run on the following networks:

• AS12880 (DCI-AS - Information Technology Company (ITC))
• AS16322 (PARSONLINE - Pars Online PJS)
• AS197207 (MCCI-AS - Mobile Communication Company of Iran PLC)
• AS25124 (DATAK - DATAK Internet Engineering Inc)
• AS29068 (UT-AS - University of Tehran Informatics Center)
• AS39074 (IR-SEPANTA-ISP - Sepanta Communication Development Co. Ltd)
• AS39501 (NGSAS - Parvaresh Dadeha Co. Private Joint Stock)
• AS42337 (RESPINA-AS - Respina Networks & Beyond PJSC)
• AS43754 (ASIATECH - Asiatech Data Transmission company)
• AS44244 (IRANCELL-AS - Iran Cell Service and Communication Company)
• AS44889 (AZMA-AS - Farhang Azma Communications Company LTD)
• AS48159 (TIC-AS - Telecommunication Infrastructure Company)
• AS48431 (MAXNET-AS - Bozorg Net-e Aria)
• AS49100 (IR-THR-PTE - Pishgaman Toseeh Ertebatat Company (Private Joint Stock))
• AS50810 (MOBINNET-AS - Mobin Net Communication Company (Private Joint Stock))
• AS56402 (DADEHGOSTAR-AS - Dadeh Gostar Asr Novin P.J.S. Co.)
• AS57218 (RIGHTEL - “Rightel Communication Service Company PJS”)
• AS57457 (ASAN-AS - Asan Pardakht Persian Private Stock)
• AS57497 (FARASOSAMANEHPASARGAD - Faraso Samaneh Pasargad Ltd.)
• AS58224 (TCI - Iran Telecommunication Company PJS)
• AS60631 (PARVASYSTEM - Pars Parva System Co. Ltd.)

Not everyone in Iran was disconnected from the Internet during the blackout. We were told that some hosting providers, banks, businesses, and journalists were able to maintain access to the Internet. Meanwhile, most people in Iran were limited to using Iran’s national Intranet during the Internet blackout.

Technical observations from manual testing

To better understand how the Internet blackout was technically implemented in Iran, we ran a series of tests locally.

We performed a TCP traceroute to port 443 on a host located in Europe (mtr -n -T 37.218.xx.xx -P 443 –report) from a vantage point inside of Iran, connected to the MCCI network (5.210.xx.xx).

By observing the network traffic data from both sides, we can see that a RST packet is injected at both ends of the connection.

Packet trace from host in Europe:

 5274 122.556004  5.210.x.x → 37.218.x.x TCP 54 58823 → 443 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
 5359 123.806532  5.210.x.x → 37.218.x.x TCP 54 46267 → 443 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0
 5406 124.600632  5.210.x.x → 37.218.x.x TCP 54 40903 → 443 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

Packet trace from host on MCCI (Iran):

   53  10.987705 192.168.x.x → 37.218.x.x TCP 74 55849 → 443 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=216567 TSecr=0 WS=128
   71  12.013533 192.168.x.x → 37.218.x.x TCP 74 [TCP Retransmission] 55849 → 443 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=216824 TSecr=0 WS=128
  131  14.029151 192.168.x.x → 37.218.x.x TCP 74 [TCP Retransmission] 55849 → 443 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=217328 TSecr=0 WS=128
  301  18.190035 192.168.x.x → 37.218.x.x TCP 74 [TCP Retransmission] 55849 → 443 [SYN] Seq=0 Win=29200 Len=0 MSS=1460 SACK_PERM=1 TSval=218368 TSecr=0 WS=128
  469  23.198129 37.218.x.x → 192.168.x.x TCP 60 443 → 55849 [RST, ACK] Seq=1 Ack=1 Win=0 Len=0

It is interesting to observe that it takes many seconds to receive the RST packet on the MCCI network, making it plausible that the network is buffering packets for a large amount of time.

Further research and investigation into this pattern and behaviour is left for future work.

Iran’s Intranet

Back in 2011, a senior Iranian official announced government plans to launch a national Intranet, the “National Information Network (NIN)”, which would host Iranian websites and services, and promote Islamic values.

Research on Iran’s network infrastructure, published in 2012, revealed the presence of a “hidden Internet”, with private IP addresses allocated on the country’s national network. Telecommunications entities were found to allow private addresses to route domestically (whether intentionally or unintentionally), creating a hidden network only reachable within Iran. This research also found that records, such as DNS queries, suggest that servers are assigned both domestic IP addresses and global ones.

While it was initially reported that Iran’s Intranet (called “SHOMA” in Persian) would be launched in 2012, it was rolled out in August 2016. Many Iranian Internet users, however, have reportedly stated that they either were not aware of Iran’s Intranet, or that they didn’t take it seriously. The main arguments that the Iranian government has highlighted to get Iranians to use the Intranet and host their websites there is that it is supposedly faster and more secure. As part of our recent discussions with Iranian Internet users, they have claimed that Iran’s Intranet cannot work properly without the Internet, and that every day electronic devices become more vulnerable as a result of not receiving updates.

But amid Iran’s nation-wide Internet blackout, for most Iranians it has only been possible to access websites and services hosted on Iran’s Intranet.

Connecting to the Internet from Iran

Through manual testing, we were able to determine that it could theoretically be possible to use DNS tunneling to get traffic to leave Iran.

Below we share the DNS query from the MCCI mobile network to whoami.akamai.net, showing the address of the resolver.

# dig whoami.akamai.net
;; QUESTION SECTION:
;whoami.akamai.net.		IN	A
;; ANSWER SECTION:
whoami.akamai.net.	94	IN	A	185.5.159.155
;; Query time: 76 msec
;; WHEN: Wed Nov 20 09:50:08 EST 2019

When performing an A lookup for a random subdomain (therefore bypassing any potential local DNS cache), we can see that the query succeeds.

# dig dvcdjtje7z.1.2.3.4.xip.io
;; QUESTION SECTION:
;dvcdjtje7z.1.2.3.4.xip.io.	IN	A
;; ANSWER SECTION:
dvcdjtje7z.1.2.3.4.xip.io. 300	IN	A	1.2.3.4
;; Query time: 305 msec
;; WHEN: Wed Nov 20 09:17:06 EST 2019

We repeated similar experiments on the resolvers of a fixed line ADSL network.

# dig dassdnjasde7z.1.2.3.4.xip.io @217.218.155.155
;; QUESTION SECTION:
;dassdnjasde7z.1.2.3.4.xip.io.	IN	A
;; ANSWER SECTION:
dassdnjasde7z.1.2.3.4.xip.io. 300 IN	A	1.2.3.4
;; AUTHORITY SECTION:
xip.io.			86400	IN	NS	ns-1.xip.io.
xip.io.			86400	IN	NS	ns-2.xip.io.
;; ADDITIONAL SECTION:
ns-1.xip.io.		86400	IN	A	166.78.161.251
ns-2.xip.io.		86400	IN	A	192.237.180.202
;; SERVER: 217.218.155.155#53(217.218.155.155)
;; WHEN: Wed Nov 20 10:34:01 EST 2019

As we can see, requests to the secondary DNS resolver 217.218.155.155 are let through.

This means that it’s possible to get the upstream recursive resolvers of the ISP to perform DNS queries on our behalf. This channel could theoretically be used to transfer some data to the Internet at a very low throughput and with high overhead. Tools like iodine could be used.

When querying the primary DNS resolver 5.200.200.200, the result is a SERVFAIL.

# dig dassdnjasde7z.1.2.3.4.xip.io @5.200.200.200
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 37763
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;dassdnjasde7z.1.2.3.4.xip.io.	IN	A
;; Query time: 142 msec
;; SERVER: 5.200.200.200#53(5.200.200.200)
;; WHEN: Wed Nov 20 10:33:40 EST 2019
;; MSG SIZE  rcvd: 57

It is, however, interesting to note that queries for records which are likely cached, work as expected.

#  dig yahoo.com @5.200.200.200
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51852
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;yahoo.com.			IN	A
;; ANSWER SECTION:
yahoo.com.		1087	IN	A	98.137.246.7
yahoo.com.		1087	IN	A	72.30.35.9
yahoo.com.		1087	IN	A	98.138.219.232
yahoo.com.		1087	IN	A	98.137.246.8
yahoo.com.		1087	IN	A	98.138.219.231
yahoo.com.		1087	IN	A	72.30.35.10
;; Query time: 52 msec
;; SERVER: 5.200.200.200#53(5.200.200.200)
;; WHEN: Wed Nov 20 10:52:51 EST 2019
;; MSG SIZE  rcvd: 134

It is possible for Iranian citizens to acquire virtual private servers (VPS) inside the country. Through local testing, we were able to determine that these VPS hosts have connectivity with both users inside the country, but also to the Internet. This makes it possible to use these servers to setup a local proxy inside of the country and to use that proxy to tunnel traffic to another proxy outside Iran. Information about this method has also been circulated in Iranian Telegram forums.

However, locals using VPS in Iran to connect to the global Internet reported that they may have experienced throttling. We do not have measurement data available to confirm this, but it could be an area for further research. If VPS traffic is in fact being throttled, it would not be the first time that Iran has throttled Internet traffic.

Conclusion

As protests erupted in Iran, the Internet was shut down on a nation-wide level over the last week.

Iran’s nation-wide Internet blackout is confirmed by several data sources, such as IODA, Google traffic data, and Oracle’s Internet Intelligence data. These data sources show that the Internet blackout in Iran started on 16th November 2019 and has been ongoing. As of 21st November 2019 (and more drastically from 23rd November 2019), Internet connectivity is being restored in Iran.

IODA data, which provides publicly-accessible network-level granularity, shows that Iranian cellular operators were disconnected first on 16th November 2019 (followed by almost all other operators over the next 5 hours), and that ISPs appear to have used diverse mechanisms to enforce the blackout. In an orthogonal study, by analyzing packet captures from the MCCI (AS197207) network, we found that a RST packet is injected at both ends of the connection.

During the blackout, most Iranians were barred from connecting to the global Internet, but they still had access to Iran’s national intranet: the domestic network hosting Iranian websites and services. Yet, OONI measurements (which require Internet connectivity) were collected from multiple networks in Iran between 16th November 2019 to 23rd November 2019, showing that the internet blackout was not total.

To explore whether and how connectivity to the Internet could be possible from Iran during the blackout, we performed manual testing locally. We found that DNS tunneling could possible be a low bandwidth solution to get network traffic to leave Iran. We also found that it was possible to connect to the Internet by using virtual private servers (VPS) to setup a local proxy in Iran and use that proxy to tunnel traffic to another proxy outside Iran.