Jekyll2023-08-17T11:27:04-06:00https://www.breakpointingbad.com/feed.xmlHome of Breakpointing Badders
Port Shadows via Network Alchemy:(CVE-2021-3773)2021-09-08T00:00:00-06:002021-09-08T00:00:00-06:00https://www.breakpointingbad.com/2021/09/08/Port-Shadows-via-Network-Alchemy<div style="width:100%; margin: auto;" align="justify">
<p><img src="/assets/ChrysopoeaofCleopatra1-dark.png" alt="alt text" /></p>
<p>Credit: <a href="https://upload.wikimedia.org/wikipedia/commons/3/32/Chrysopoea_of_Cleopatra_1.png">Wikipedia</a></p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Introduction</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>OpenVPN’s use of Netfilter makes it susceptible to several attacks that can
cause denial-of-service, deanonymization of clients, or redirection of a victim
client connection to an attacker controlled server. Netfilter is a module
within the Linux kernel that implements stateless and stateful firewall
mechanisms and network address translation (NAT). Netfilter’s design includes
hooks that are called at various points in the networking code to execute,
e.g., user-defined firewall rules and NAT code. The NAT portion of Netfilter
is designed in such a way that if a machine behind the NAT uses the same source
port as an application listening on the same port as the NAT (i.e., the NAT is
acting both as a NAT-router and a server), then Netfilter translates and routes
received packets intended for the NAT’s own listening port to a host behind the
NAT using the same port. This shadowing behavior is not specified in any relevant
request for comments (rfc768, rfc793, rfc4787, rfc5382, rfc7857, or any of their
successors). The remainder of this disclosure uses the term “port shadow(ing)”
when discussing this attack primitive.</p>
<p>Port shadowing’s root cause originates from Netfilter’s lack of coordination
with the Linux socket infrastructure to determine whether a port in a listening
state (or any particular state) on the NAT creates ambiguity with a machine
using the same port behind the NAT. Port shadowing has interesting
implications for applications, such as OpenVPN, that rely on Netfilter for NAT.
A malicious OpenVPN client can use port shadowing to deanonymize victim machines
connected to the same OpenVPN server or escalate privileges from an OpenVPN
client to a man-in-the-middle (c2mitm) between another client (the victim) and
the OpenVPN server to which both the attacker and victim are connected. The
c2mitm variation of the attack can be combined with a recently disclosed,
server-side attack against OpenVPN to inject DNS responses, reset, or even
hijack TCP connections of the victim, even when that connection is tunneled
through the OpenVPN server.</p>
<p>Version information is at the bottom of this note. While we have not
yet tested other versions of OpenVPN, any version of Wireguard or strongSwan,
or any other VPN server implementations, we believe our attack applies to any
version of any VPN that relies on Linux’s Netfilter for NAT. We have<br />
successfully tested the full FreeBSD-13 with natd and OpenVPN. Additionally,
we have tested against FreeBSD-13 with IPFW, PF, and IPF and found that, while
they are not succesptable to c2mitm, the port shadow still applies. There is no
specific implemetation bug with Netfilter, nor presumably FreeBSD’s NAT
implementations, that makes port shadowing possible, it is simply an
implementation artifact that is apropos of following the RFCs that relate to
NAT.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Background</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>The port shadow attack primitive is not specific to OpenVPN and is
fundamentally related to network address translation. OpenVPN is tightly
coupled with the concept of NAT and depends on Netfilter for NAT support,
hence, the following discussion about OpenVPN focuses on its role as a NAT
since our attack arises from NAT related implementation details. Cryptography
is not described in any detail since our attack does not target any cryptographic
components. The following background subsections provide information on relevant
threat models, OpenVPN, Netfilter, and the port shadow primitive.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Threat Models</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>The two threat models most relevant to this disclosure are <em>adjacent</em> and
<em>in-path</em> attackers. Building a port shadow assumes a logically
adjacent attacker and the DNS injection, the coup de grace covered at the
end of this post, requires an in-path attacker.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Adjacent</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>MITRE defines a logically adjacent attacker as follows under the attack vector
description <a href="https://www.first.org/cvss/v3.1/specification-document">MITRE</a>:</p>
<blockquote>
<p>The vulnerable component is bound to the network stack, but the attack is
limited at the protocol level to a logically adjacent topology. This can mean
an attack must be launched from the same shared physical (e.g., Bluetooth or
IEEE 802.11) or logical (e.g., local IP subnet) network, or from within a
secure or otherwise limited administrative domain (e.g., MPLS, secure VPN to an
administrative network zone). One example of an Adjacent attack would be an ARP
(IPv4) or neighbor discovery (IPv6) flood leading to a denial of service on the
local LAN segment (e.g., CVE‑2013‑6014).</p>
</blockquote>
<pre><code class="language-ascii">
|OpenVPN Server(NAT/router)|
/ \
. .
/ \
|routerA| |routerV|
| |
. .
/ \
|attacker| |victim|
</code></pre>
<p>It is important to note that logically adjacent and network adjacent are two
different threat models. Our attacks do not make any assupmtions about the
attacker being network adjacent. In the context of our attacks, for practical
purposes logically adjacent simply means the attacker is connected as a client
to the same VPN server as the victim.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">In-path</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>The in-path attacker is a machine that routes packets between two hosts. For
this disclosure, the attacker is between the victim and the OpenVPN server.
Readers seeking a formal definition of in-path threats or
interesting in-path attacks are referred to Marczak et al.’s work on the <a href="https://www.usenix.org/system/files/conference/foci15/foci15-paper-marczak.pdf">Great
Cannon</a>.</p>
<pre><code class="language-ascii">
|OpenVPN Server(NAT/router)|
|
.
|attacker|
|
.
|
|victim|
</code></pre>
<p>An attacker can leverage the port shadow primitive to escalate from logically adjacent
to in-path against OpenVPN.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">OpenVPN</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>OpenVPN uses Netfilter to perform NAT for clients connected to the server. A non-exhaustive
list of VPN use-cases is:</p>
<ol>
<li>securing network traffic when using public WiFi at hotels, coffee shops or airports</li>
<li>hiding illegally torrented content</li>
<li>bypassing geo blocked content filters</li>
<li>bypassing restrictive firewalls</li>
<li>masking your IP address</li>
</ol>
<p>Regardless of the specific use-case, a client must first establish a VPN tunnel
between herself and the OpenVPN server. During connection establishment, the
victim sends an initial UDP or TCP packet to a listening port on the OpenVPN
server (typically 1194, but is configurable and known by the client).
The subsequent packets exchanged establish a secure channel using SSL/TLS and
various configuration parameters, including the client’s virtual IP address,
are pushed to the client.</p>
<pre><code class="language-ascii">
(C)lient OpenVPN (S)erver
|-- {src=C:Cport, dst=S:1194} ->| 1. First packet Client sends to OpenVPN server
|<- { Exchange parameters over SSL/TLS } ->| 2. Server and Client establish tunnel parameters
</code></pre>
<p>Fig. 1. Diagram showing the first packet the client sends to the server
and the subsequent packets exchanged to establish the client’s tunnel
parameters.</p>
<p>Once the client has a virtual-IP address that the OpenVPN server associates
with her and her routes are configured to send all originating packets through
the tunnel, the client may exchange packets between other globally routable IP
addresses and they will assume the traffic originated from the OpenVPN server
instead of the client. Also, any in-path machines between the
OpenVPN client and server will only see encrypted traffic.</p>
<pre><code class="language-ascii">
|----------| |------------------|
| Client A |~~~~~| In-path Machine |
|----------| |------------------|
| |------------|
. ~~~~~~~~~~~~| Web Server |
| |------------|
|----------| |----------------|
| Client B |~~~~~| OpenVPN Server |
|----------| |----------------|
</code></pre>
<p>Fig. 2. An example Topology of an OpenVPN Server and two VPN clients. There is also an in-path router
between Client A and the OpenVPN server. If Client A (or B or Both) request data from Web Server,
Web Server will think the requests came from OpenVPN server, regardless of which client issued
the request. In-path Machine will not know if Client A requests a page from Web Server.</p>
<p>A recently disclosed attack by Tolley et al.
<a href="https://breakpointingbad.com/papers/Blind-in-path-attacks-VPN-USENIX21.pdf">Tolley2021</a>
demonstrates how an attacker can leak connection information or even hijack
connections between an OpenVPN client and an end-server. Their attacks require
that the attacker be in-path between the OpenVPN client and the OpenVPN
server. While in-path capabilities are typically reserved for nation-states, ISP, or
similarly powerful adversaries, using <em>network alchemy</em>, we are able to
deanonymize an OpenVPN client or transmute a basic OpenVPN client into an in-path
attacker.</p>
<p>We define network alchemy as the creation of states of network connections that
should not exist, using a combination of TTL limiting, packet spoofing, RFC
ambiguities, and other low-level network primitives.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Conntrack & NAT</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Conntrack stores state about connections it sees in a single, global
hash table called
<a href="https://elixir.bootlin.com/linux/latest/C/ident/nf_conntrack_hash"><code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code></a>.</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">hlist_nulls_head</span> <span class="o">*</span><span class="n">nf_conntrack_hash</span> <span class="n">__read_mostly</span><span class="p">;</span>
<span class="n">EXPORT_SYMBOL_GPL</span><span class="p">(</span><span class="n">nf_conntrack_hash</span><span class="p">);</span>
</code></pre></div> </div>
<p>Fig. 3. <code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> table declaration.</p>
<p>The table is shared by all protocols and packets, regardless of
network namespaces or sockets. The table is initialized at module load
time and has a maximum size that is a function of the amount of system
memory. The size is also partially controlled by the system variable
<code class="language-plaintext highlighter-rouge">net.ipv4.netfilter.nf_conntrack_max</code>. The fact that the table is
global and does not coordinate with the socket infrastructure are
the primary factors contributing to port-shadow behavior.</p>
<p>Individual connections are represented as entries in <code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> by the
<a href="https://elixir.bootlin.com/linux/latest/source/include/net/netfilter/nf_conntrack.h#L58"><code class="language-plaintext highlighter-rouge">nf_conn</code></a>
struct.</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">struct</span> <span class="n">nf_conn</span> <span class="p">{</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="n">u32</span> <span class="n">timeout</span><span class="p">;</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="cm">/* XXX should I move this to the tail ? - Y.K */</span>
<span class="cm">/* These are my tuples; original and reply */</span>
<span class="k">struct</span> <span class="n">nf_conntrack_tuple_hash</span> <span class="n">tuplehash</span><span class="p">[</span><span class="n">IP_CT_DIR_MAX</span><span class="p">];</span>
<span class="cm">/* Have we seen traffic both ways yet? (bitset) */</span>
<span class="kt">unsigned</span> <span class="kt">long</span> <span class="n">status</span><span class="p">;</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
</code></pre></div> </div>
<p>Fig. 4. <code class="language-plaintext highlighter-rouge">nf_conn</code> struct including the fields related to this disclosure.</p>
<p>The metadata inside each <code class="language-plaintext highlighter-rouge">nf_conn</code> entry includes a <code class="language-plaintext highlighter-rouge">timeout</code> value, a
two-entry array of <code class="language-plaintext highlighter-rouge">nf_conntrack_tuple_hash</code> structs (one for the
<code class="language-plaintext highlighter-rouge">ORIGINAL</code> direction and one for the <code class="language-plaintext highlighter-rouge">REPLY</code> direction), and a status
bitfield. Each <code class="language-plaintext highlighter-rouge">nf_conntrack_tuple_hash</code> contains an
<code class="language-plaintext highlighter-rouge">nf_conntrack_tuple</code> and each <code class="language-plaintext highlighter-rouge">nf_conntrack_tuple</code> contains its own
copy of the packet’s direction, source and destination ports and IP
addresses, and protocol number.</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/* Connections have two entries in the hash table: one for each way */</span>
<span class="k">struct</span> <span class="n">nf_conntrack_tuple_hash</span> <span class="p">{</span>
<span class="k">struct</span> <span class="n">hlist_nulls_node</span> <span class="n">hnnode</span><span class="p">;</span>
<span class="k">struct</span> <span class="n">nf_conntrack_tuple</span> <span class="n">tuple</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div> </div>
<p>Fig. 5. <code class="language-plaintext highlighter-rouge">nf_conntrack_tuple_hash</code> declaration.</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cm">/* The manipulable part of the tuple. */</span>
<span class="k">struct</span> <span class="n">nf_conntrack_man</span> <span class="p">{</span>
<span class="k">union</span> <span class="n">nf_inet_addr</span> <span class="n">u3</span><span class="p">;</span>
<span class="k">union</span> <span class="n">nf_conntrack_man_proto</span> <span class="n">u</span><span class="p">;</span>
<span class="cm">/* Layer 3 protocol */</span>
<span class="n">u_int16_t</span> <span class="n">l3num</span><span class="p">;</span>
<span class="p">};</span>
<span class="cm">/* This contains the information to distinguish a connection. */</span>
<span class="k">struct</span> <span class="n">nf_conntrack_tuple</span> <span class="p">{</span>
<span class="k">struct</span> <span class="n">nf_conntrack_man</span> <span class="n">src</span><span class="p">;</span>
<span class="cm">/* These are the parts of the tuple which are fixed. */</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="k">union</span> <span class="n">nf_inet_addr</span> <span class="n">u3</span><span class="p">;</span>
<span class="k">union</span> <span class="p">{</span>
<span class="cm">/* Add other protocols here. */</span>
<span class="n">__be16</span> <span class="n">all</span><span class="p">;</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="n">__be16</span> <span class="n">port</span><span class="p">;</span>
<span class="p">}</span> <span class="n">tcp</span><span class="p">;</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="n">__be16</span> <span class="n">port</span><span class="p">;</span>
<span class="p">}</span> <span class="n">udp</span><span class="p">;</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="n">u_int8_t</span> <span class="n">type</span><span class="p">,</span> <span class="n">code</span><span class="p">;</span>
<span class="p">}</span> <span class="n">icmp</span><span class="p">;</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="n">__be16</span> <span class="n">port</span><span class="p">;</span>
<span class="p">}</span> <span class="n">dccp</span><span class="p">;</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="n">__be16</span> <span class="n">port</span><span class="p">;</span>
<span class="p">}</span> <span class="n">sctp</span><span class="p">;</span>
<span class="k">struct</span> <span class="p">{</span>
<span class="n">__be16</span> <span class="n">key</span><span class="p">;</span>
<span class="p">}</span> <span class="n">gre</span><span class="p">;</span>
<span class="p">}</span> <span class="n">u</span><span class="p">;</span>
<span class="cm">/* The protocol. */</span>
<span class="n">u_int8_t</span> <span class="n">protonum</span><span class="p">;</span>
<span class="cm">/* The direction (for tuplehash) */</span>
<span class="n">u_int8_t</span> <span class="n">dir</span><span class="p">;</span>
<span class="p">}</span> <span class="n">dst</span><span class="p">;</span>
<span class="p">};</span>
</code></pre></div> </div>
<p>Fig. 6. <code class="language-plaintext highlighter-rouge">nf_conntrack_tuple</code> and <code class="language-plaintext highlighter-rouge">nf_conntrack_man</code> declarations. Notice that only
source portion of the tuple is manipulatable.</p>
<p>When Conntrack processes a packet, it checks to see whether it is in
<code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code>. If the packet is part of an existing connection,
Conntrack pulls the associated <code class="language-plaintext highlighter-rouge">nf_conn</code> entry for the remainder of
processing, otherwise, a new <code class="language-plaintext highlighter-rouge">nf_conn</code> entry is created and inserted
into <code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code>. If the <code class="language-plaintext highlighter-rouge">nf_conn</code> entry is new, processing
includes remapping colliding ports and is performed by
<a href="https://elixir.bootlin.com/linux/v5.12/source/net/netfilter/nf_nat_core.c#L506"><code class="language-plaintext highlighter-rouge">get_unique_tuple</code></a>
in Netfilter’s NAT module. As we demonstrate in the <em>Attack
Methodology</em> section, Netfitler’s port remapping (and, in special cases, lack thereof) make our
attacks possible (port shadowing leverages this
behavior). As the packet passes through Netfitler’s NAT hook, its source IP address
and port may be overwritten, depending on whether or not ports
collide.</p>
<p>The <code class="language-plaintext highlighter-rouge">direction</code> field represents whether the packet is from the
connection originator or replier (i.e., <code class="language-plaintext highlighter-rouge">ORIGINAL</code> and <code class="language-plaintext highlighter-rouge">REPLY</code>). Conntrack
infers the connection’s direction based on whether an <code class="language-plaintext highlighter-rouge">nf_conn</code> entry exists
for the given source and destination IP addresses, protocol number, and port
numbers in the case of UDP and TCP. When Conntrack sees a packet for which no
<code class="language-plaintext highlighter-rouge">nf_conn</code> entry exists, it assumes that packet is the originator of the
connection and assigns <code class="language-plaintext highlighter-rouge">nf_conn.tuplehash[ORIGINAL]</code> using the fields of the
packet being processed. This idea of connection direction is based on the
notion of “start of session”, and Conntrack’s use of <code class="language-plaintext highlighter-rouge">ORIGINAL</code> and <code class="language-plaintext highlighter-rouge">REPLY</code>
directions are consistent with section 2.5. of
<a href="https://datatracker.ietf.org/doc/html/rfc2663#section-2.5">rfc2663</a>.</p>
<blockquote>
<p>2.5. Start of session for TCP, UDP and others</p>
<p>The first packet of every TCP session tries to establish a session
and contains connection startup information. The first packet of a
TCP session may be recognized by the presence of SYN bit and absence
of ACK bit in the TCP flags. All TCP packets, with the exception of
the first packet, must have the ACK bit set.</p>
<p>However, there is no deterministic way of recognizing the start of a
UDP based session or any non-TCP session. A heuristic approach would
be to assume the first packet with hitherto non-existent session
parameters (as defined in section 2.3) as constituting the start of
new session.</p>
</blockquote>
<p>As stated previously, Conntrack stores two copies of the source and destination
IP addresses and ports, the protocol number, and the direction in each
<code class="language-plaintext highlighter-rouge">nf_conn</code> entry in the <code class="language-plaintext highlighter-rouge">tuplehash</code> array. In the case where the machine running
Conntrack is also the actual sender of packets in a session, the
<code class="language-plaintext highlighter-rouge">ORIGINAL</code> direction’s source IP address is the machine running
Conntrack, the source port is whatever Conntrack’s sending
application chose, the destination IP address is some remote host’s IP
address, and the destination port is whatever port the remote host is using to communicate
with the Conntrack host. In the <code class="language-plaintext highlighter-rouge">REPLY</code> direction, the source IP and
destination IP are switched and the destination is the Conntrack
machine’s IP address (i.e., the source in the <code class="language-plaintext highlighter-rouge">ORIGINAL</code> direction). This
detail is important because with OpenVPN, from Conntrack’s
perspective, the <code class="language-plaintext highlighter-rouge">ORIGINAL</code> direction for tunnel initializations <em>should</em> always
the OpenVPN client’s IP address. According to Netfilter, maintaining two entries
is an optimization for faster lookups and translations. Fig. 7 below provides a
generic example of what the <code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> and some <code class="language-plaintext highlighter-rouge">nf_conn</code> entry might
look like.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H .-> nf_conn[.] -> {
timeout = T;
tuplehash = {
[ORIGINAL] .-> {src={u3=Conntrack Machine, udp.port=sport}, dst={u3=Remote Server, udp.port=dport}},
[REPLY] .-> {src={u3=Remote Server, udp.port=dport}, dst={u3=Conntrack Machine, udp.port=sport}}
status = S;
};
}
}
</code></pre>
<p>Fig. 7. <code class="language-plaintext highlighter-rouge">nf_conn</code> entry structure for entries in the
<code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> table where the NAT is the actually end-point in a
connection session.</p>
<p>The following case the machine running Conntrack acts as a NAT. When a
host behind the NAT (with a private IP) sends a packet out the NAT,
the <code class="language-plaintext highlighter-rouge">nf_conntrack_tuple[ORIGINAL]</code> direction’s source IP address is
the private IP address of the client sending the packet while the
source of the <code class="language-plaintext highlighter-rouge">REPLY</code> direction is the NAT’s public IP
address. Assuming the port of this packet does not collide with other ports NAT is
translating, port numbers are swapped without being manipulated. This behavior is related to the c2mitm attack’s use of
the port shadowing primitive. Fig. 7 provides a graphical representation
of this situation.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H .-> nf_conn[.] -> {
timeout = T;
tuplehash = {
[ORIGINAL] .-> {src={u3=Machine Behind NAT, udp.port=sport}, dst={u3=Remote Server, udp.port=dport}},
[REPLY] .-> {src={u3=Remote Server, udp.port=dport}, dst={u3=Conntrack Machine, udp.port=sport}}
};
status = S;
}
}
</code></pre>
<p>Fig. 7. <code class="language-plaintext highlighter-rouge">nf_conn</code> entry structure for NAT entries in the
<code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> table.</p>
<p>If there are port collisions, then NAT will remap the ports, as depicted in the following figure.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = T;
tuplehash = {
[ORIGINAL] .-> {src={u3=Remote Server, udp.port=Rport}, dst={u3=Conntrack Machine, udp.port=1194}},
[REPLY] .-> {src={u3=Conntrack Machine, udp.port=1194}, dst={u3=Remote Server, udp.port=Rport}}
};
status = S;
}
H2 .-> nf_conn[.] -> {
timeout = T;
tuplehash = {
[ORIGINAL] .-> {src={u3=Machine Behind NAT, udp.port=1194}, dst={u3=Remote Server, udp.port=Rport}},
[REPLY] .-> {src={u3=Remote Server, udp.port=NAT(Rport)}, dst={u3=Conntrack Machine, udp.port=1194}}
};
status = S;
}
}
</code></pre>
<p>Fig. 8. Depicts <code class="language-plaintext highlighter-rouge">nf_conn</code> structure for NAT entries in the
<code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> table when there is a port collision.</p>
<p>Fig. 8., assume that <code class="language-plaintext highlighter-rouge">H1</code> was in the table before <code class="language-plaintext highlighter-rouge">H2</code>. Notice that in the
<code class="language-plaintext highlighter-rouge">REPLY</code> direction of the <code class="language-plaintext highlighter-rouge">H2</code> entry that <code class="language-plaintext highlighter-rouge">src.udp.port</code> is a modified version
of Rport, denoted by a function <code class="language-plaintext highlighter-rouge">NAT</code> that remaps that port. This is a
simplified example of what happens when Netfilter’s NAT module resolves ports
that collide in this way. The collision happens because <code class="language-plaintext highlighter-rouge">H1</code> in the <code class="language-plaintext highlighter-rouge">ORIGINAL</code>
direction and H2 in the <code class="language-plaintext highlighter-rouge">REPLY</code> direction would hash to the same entry and
hence generate a non-unique tuple which cannot happen if NATing needs to
function correctly. We found that a malicious machine connected to the OpenVPN
server can shadow the OpenVPN server’s listening port (typically 1194) by
sending packets with the same source port as the OpenVPN server’s listening port to
arbitrary destination IP address and ports. Because these entries
are consulted for NAT before most packet processing <em>and</em> because they are used
to reroute packets, any packets matching an entry of this type bypasses any
listening sockets (such as the one OpenVPN uses) it was intended for. We will show how an attacker
can build a port shadow to deanonymize another OpenVPN client
or perform a c2mitm attack against a victim in the <em>Attack Methodology</em> section.</p>
<p>One challenge the attacker faces is uncertainty about when a victim will connect
to the OpenVPN server.</p>
<p><code class="language-plaintext highlighter-rouge">nf_conn</code> entries are kept in the table until their <code class="language-plaintext highlighter-rouge">timeout</code> value
expires. Conntrack’s garbage collector runs periodically and evicts
entries whose timeout has expired or if the table is filled beyond
95\%. In the latter case, Conntrack can evict entries before their
timeout has expired, but only if the entry’s <code class="language-plaintext highlighter-rouge">status</code> does not have
the <code class="language-plaintext highlighter-rouge">ASSURED</code> bit set. As packets are exchanged and as Netfilter
processes them, the timeout value and status get updated. An attacker
can take advantage of this to place entries in the <code class="language-plaintext highlighter-rouge">ASSURED</code> state,
and periodically send packets through Conntrack to ensure the timeout
value never expires which is necessary for the c2mitm attack.</p>
<p>Placing an entry in the <code class="language-plaintext highlighter-rouge">ASSURED</code> state is protocol dependent and
Conntrack requires that each protocol module maintain its state
according to that protocol’s requirements. There is no RFC to
dictate how this should or must be done, so the module authors make a
best effort to determine when the <code class="language-plaintext highlighter-rouge">status</code> bits should be updated. In
the case of UDP, the <code class="language-plaintext highlighter-rouge">ASSURED</code> bit is set whenever packets are
exchanged in both directions.</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/* Returns verdict for packet, and may modify conntracktype */</span>
<span class="kt">int</span> <span class="nf">nf_conntrack_udp_packet</span><span class="p">(</span><span class="k">struct</span> <span class="n">nf_conn</span> <span class="o">*</span><span class="n">ct</span><span class="p">,</span>
<span class="k">struct</span> <span class="n">sk_buff</span> <span class="o">*</span><span class="n">skb</span><span class="p">,</span>
<span class="kt">unsigned</span> <span class="kt">int</span> <span class="n">dataoff</span><span class="p">,</span>
<span class="k">enum</span> <span class="n">ip_conntrack_info</span> <span class="n">ctinfo</span><span class="p">,</span>
<span class="k">const</span> <span class="k">struct</span> <span class="n">nf_hook_state</span> <span class="o">*</span><span class="n">state</span><span class="p">)</span>
<span class="p">{</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="cm">/* If we've seen traffic both ways, this is some kind of UDP
* stream. Set Assured.
*/</span>
<span class="k">if</span> <span class="p">(</span><span class="n">test_bit</span><span class="p">(</span><span class="n">IPS_SEEN_REPLY_BIT</span><span class="p">,</span> <span class="o">&</span><span class="n">ct</span><span class="o">-></span><span class="n">status</span><span class="p">))</span> <span class="p">{</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="cm">/* Also, more likely to be important, and not a probe */</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="n">test_and_set_bit</span><span class="p">(</span><span class="n">IPS_ASSURED_BIT</span><span class="p">,</span> <span class="o">&</span><span class="n">ct</span><span class="o">-></span><span class="n">status</span><span class="p">))</span>
<span class="n">nf_conntrack_event_cache</span><span class="p">(</span><span class="n">IPCT_ASSURED</span><span class="p">,</span> <span class="n">ct</span><span class="p">);</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="k">return</span> <span class="n">NF_ACCEPT</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div> </div>
<p>Fig. 9. ASSURED bit handling for UDP packets in Conntrack.</p>
<p>For TCP, Conntrack maintains a shadow TCP state machine of a TCP
connection between two hosts whose packet’s the machine running
Conntrack routes. This is accomplished using the
<a href="https://elixir.bootlin.com/linux/latest/C/ident/tcp_conntracks"><code class="language-plaintext highlighter-rouge">tcp_conntrack</code></a>
state transition table, a function to determine whether the incoming
packet should be routed based on Conntrack policy
(<code class="language-plaintext highlighter-rouge">nf_conntrack_tcp_packet</code>), and the <code class="language-plaintext highlighter-rouge">nf_conn.status</code> field. The
<a href="https://elixir.bootlin.com/linux/latest/C/ident/nf_conntrack_tcp_packet"><code class="language-plaintext highlighter-rouge">nf_conntrack_tcp_packet</code></a>
function contains a code-path that sets the <code class="language-plaintext highlighter-rouge">ASSURED</code> bit to 1.</p>
<div class="language-c highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cm">/* Returns verdict for packet, or -1 for invalid. */</span>
<span class="kt">int</span> <span class="nf">nf_conntrack_tcp_packet</span><span class="p">(</span><span class="k">struct</span> <span class="n">nf_conn</span> <span class="o">*</span><span class="n">ct</span><span class="p">,</span>
<span class="k">struct</span> <span class="n">sk_buff</span> <span class="o">*</span><span class="n">skb</span><span class="p">,</span>
<span class="kt">unsigned</span> <span class="kt">int</span> <span class="n">dataoff</span><span class="p">,</span>
<span class="k">enum</span> <span class="n">ip_conntrack_info</span> <span class="n">ctinfo</span><span class="p">,</span>
<span class="k">const</span> <span class="k">struct</span> <span class="n">nf_hook_state</span> <span class="o">*</span><span class="n">state</span><span class="p">)</span>
<span class="p">{</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">}</span> <span class="k">else</span> <span class="nf">if</span> <span class="p">(</span><span class="o">!</span><span class="n">test_bit</span><span class="p">(</span><span class="n">IPS_ASSURED_BIT</span><span class="p">,</span> <span class="o">&</span><span class="n">ct</span><span class="o">-></span><span class="n">status</span><span class="p">)</span>
<span class="o">&&</span> <span class="p">(</span><span class="n">old_state</span> <span class="o">==</span> <span class="n">TCP_CONNTRACK_SYN_RECV</span>
<span class="o">||</span> <span class="n">old_state</span> <span class="o">==</span> <span class="n">TCP_CONNTRACK_ESTABLISHED</span><span class="p">)</span>
<span class="o">&&</span> <span class="n">new_state</span> <span class="o">==</span> <span class="n">TCP_CONNTRACK_ESTABLISHED</span><span class="p">)</span> <span class="p">{</span>
<span class="cm">/* Set ASSURED if we see valid ack in ESTABLISHED
after SYN_RECV or a valid answer for a picked up
connection. */</span>
<span class="n">set_bit</span><span class="p">(</span><span class="n">IPS_ASSURED_BIT</span><span class="p">,</span> <span class="o">&</span><span class="n">ct</span><span class="o">-></span><span class="n">status</span><span class="p">);</span>
<span class="n">nf_conntrack_event_cache</span><span class="p">(</span><span class="n">IPCT_ASSURED</span><span class="p">,</span> <span class="n">ct</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="p">.</span>
<span class="k">return</span> <span class="n">NF_ACCEPT</span><span class="p">;</span>
<span class="err">}</span>
</code></pre></div> </div>
<p>Fig. 10. ASSURED bit handling for UDP packets in Conntrack.</p>
<p>Using this information, an attacker can keep her entries in the
Conntrack table long enough to perform the c2mitm attack.</p>
<p>In addition to modifying the <code class="language-plaintext highlighter-rouge">status</code> bits of the <code class="language-plaintext highlighter-rouge">nf_conn</code> entry, Netfilter’s
TCP state machine makes forwarding decisions based on the TCP header and
Netfilter’s view of the TCP state between the two TCP end-points. This has
implications for which TCP packets can cross the NAT-external/ NAT-internal
address realms. TCP simultaneous open in particular causes security issues for
OpenVPN clients because an attacker can leverage Netfilter’s behavior when
processing SYNs to trick the victim into connecting to some TCP service through
the VPN server, back to the client. In the <em>Attack Methodology</em> section, we
demonstrate how an attacker can combine c2mitm and DNS injection with
simultaneous open to strip the OpenVPN’s tunnel encryption and establish a TCP
session with the victim.</p>
<p>In summary, we have described Netfilter’s stateful Connection tracking
and network address translation implementation (Conntrack and NAT) as
they relate to port shadowing. We described how entries are created,
stored, used and destroyed by Conntrack. Next, we provide a
description of how the attacker builds the port shadowing exploit
primitive.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Port Shadowing Exploit Primitive</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>We identified an exploit primitive, port shadowing, that an attacker can use to
either deanonymize a client connected to an OpenVPN server or perform a
privilege escalation from client to man-in-the-middle. The attacks are
related to each other and differ mainly in the context of their use of the
exploit primitive as well as whether a client connects to the OpenVPN server
before or after the attacker.</p>
<p>The attacker builds the exploit primitive in two steps. The process is
the same for both attacks.</p>
<ol>
<li>Connect to the OpenVPN server.</li>
<li>Send packets to a victim’s candidate, public IP address using the
OpenVPN server’s listening port as the source port to all
destination ports in the IANA ephemeral port range.</li>
</ol>
<p>After step (2.) the OpenVPN server’s <code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> contains
16383 entries. Each entry is capable of routing packets matching the
tuples in <code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> back to the attacker’s OpenVPN tunnel
interface. We describe in the <em>Attack Methodology</em> section how an
attacker uses this to deanonymize the victim or become a man-in-the-middle
between the victim and the OpenVPN server.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Attack Methodology</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>We now describe our testing environment, the deanonymization and c2mitm
attacks, and a third attack that demonstrates that network alchemy can be
applied to TCP in addition to UDP connections.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Testing Environment</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>We tested the attacks in a virtual environment and on the real Internet. Both environments
ran OpenVPN server version 2.4.4 on an Ubuntu 18.04 headless server running
Linux kernel version 4.15.0-142. We packaged our virtual environment as a vagrant
script and include the required software dependencies and code to reproduce our
attack (to the best of our abilities though some assembly may be required). Our
live attack was performed against a server we setup and configured ourselves. We
deployed our OpenVPN server on a machine with 2048 MB RAM, 1 core, and 55 GB storage
using <a href="https://www.vultr.com/">vultr</a> cloud services.
We manually installed OpenVPN 2.4.4 on the Ubuntu 18.04 vultr instance using
<a href="https://www.digitalocean.com/community/tutorials/how-to-set-up-an-openvpn-server-on-ubuntu-18-04">Digital Oceans instructions</a>.
Our use of our own server for testing was to eliminate any interference our
attacks may create with production clients using commercial or consumer VPN
providers.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">OpenVPN Client Deanonymization</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Assumptions</p>
<p>The deanonymization attack makes the following assumption:</p>
<ol>
<li>The Victim (V) connects to the OpenVPN server (S) before the Attacker (A)
carries out any attempt to see if they are connected. This is somewhat
obvious, but we specify it as an assumption to make the exact nature of the
attack clear. In this example, we assume the VPN uses UDP, however, these
attacks are also possible using TCP.</li>
</ol>
<pre><code class="language-ascii">nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=V, udp.port=Vport}}
};
status = {ASSURED};
},
}
</code></pre>
<p>Fig. 11. S’s Conntrack table with V’s legitimate tunnel entry.</p>
<ol>
<li>A has two abilities: their own credentials to connect to the same OpenVPN server as
V, and the ability to spoof packets on the Internet. A may or may not use two different
computers for the attack.</li>
</ol>
<p>Execution</p>
<p>The deanonymization attack is executed in four steps as follows:</p>
<ol>
<li>
<p>A connects to S</p>
<pre><code class="language-ascii"> nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=V, udp.port=Vport}}
};
status = {ASSURED};
},
H2 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}}
};
status = {ASSURED};
},
}
</code></pre>
<p>Fig. 12. S’s Conntrack table with V and A’s legitimate tunnel
entries.</p>
</li>
<li>
<p>A sends UDP packets from herself to a candidate IP address of V. Each
packet has the source port equal to whatever the OpenVPN server’s listening
port is (assume 1194), a unique destination port in the IANA ephemeral port
range, and is TTL limited such that the UDP packets do not reach the candidate
IP.</p>
<p>The attacker must cycle through the victim’s ephemeral port space only once.
This requires 16383 packets per candidate victim IP (assuming the operating
system uses the IANA dynamic port range, some operating systems the victim may
be using have a larger range). The attacker can check candidate victim IPs in
parallel and send at any reasonable speed, and note also that the attacker does
not need to check every one of the roughly 4 billion possible IPv4 addresses if
they have a profile of the victim, such as what city they connect to the VPN
from. We spoof responses in subsequent steps, which imposes a 180 second
timeout (for the conntrack entry to be garbage collected) before we can check
again if this particular candidate victim IP has connected.</p>
<pre><code class="language-ascii"> nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=V, udp.port=Vport}}
};
status = {ASSURED};
},
H2 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}}
};
status = {ASSURED};
},
.
.
.
Hn .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=tun0A, port=1194}, dst={u3=V, udp.port=Vport}},
[REPLY] .-> {src={u3=V, port=NAT(Vport)}, dst={u3=S, udp.port=1194}}
};
status = {ASSURED};
},
}
</code></pre>
<p>Fig. 13. S’s Conntrack table with V and A’s legitimate tunnel entry
and A’s entry whose destination port in the <code class="language-plaintext highlighter-rouge">ORIGINAL</code> direction
collides with V’s legitimate Conntrack entry’s source port in the
Conntrack table. The entries are denoted by entries H1, H2, and Hn,
respectively.</p>
</li>
<li>
<p>From a seperate machine, A spoofs UDP responses for each of the UDP
packets sent in (2.) from the candidate IP to S.</p>
</li>
<li>
<p>A’s machine connected to S collects the responses sent in (3.)
routed back to her by S.</p>
</li>
</ol>
<p>If A receives responses for every ephemeral port sent out in step
(2.), then she knows the candidate IP is not connected to the same
OpenVPN server (S) as her. If A observes <strong>one</strong> missing ephemeral
port response, then she knows the candidate IP is V and is connected to
the same OpenVPN server as her (she can repeat missing probes as needed to account for packet loss). This is because if V is connected to
the same OpenVPN server (S) as her, then S’s Conntrack table will have
an <code class="language-plaintext highlighter-rouge">nf_conn</code> entry with S’s IP and OpenVPN port (1194) and V’s public
IP address and ephemeral port.
This entry makes it impossible for packets sent from A:1194 to V:Vport to
be differentiated from the original tuple. Conntrack/NAT must select
a new destination port for the packet sent in step (2.) to maintain
transparency and operational requirements specified in the RFCs.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">OpenVPN Client-to-Man-in-the-Middle (c2mitm)</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Assumptions</p>
<p>The c2mitm attack makes the following assumptions:</p>
<ol>
<li>
<p>The attacker (A) knows the candidate, public IP address for a potential
Victim (V).</p>
</li>
<li>
<p>V does not connect to the OpenVPN server (S) until after (A) builds the
exploit primitive.</p>
</li>
</ol>
<p>Assumption (1.) is viable in two situations. The first is if A has performed
the deanonymization attack in the past, and enumerated one or more victims that
includes V. The second is if V is at the same coffee shop or hotel as A. The
hotel or coffee shop is likely running its own NAT to share internet resources
with all the customers using the network. A can easily discover the public IP
address of the establishment. We imagine (2.) holding because, as stated above,
A can hold <code class="language-plaintext highlighter-rouge">nf_conn</code> entries in the OpenVPN server’s table indefinitely.</p>
<p>Execution</p>
<p>The c2mitm attack is executed in three stages:</p>
<ol>
<li>
<p>A connects to S</p>
<pre><code class="language-ascii"> time
| (A)ttacker OpenVPN (S)erver (V)ictim
| |---{src=A:Aport, dst=S:1194}--->| 1.1.
| |<--{src=S:1194, dst=A:Aport}----| 1.2.
| tun0 |=============TUNNEL=============| 1.3.
V
</code></pre>
<p>Fig 14. High-level space-time network diagram of A establishing
OpenVPN tunnel to S.</p>
<pre><code class="language-ascii"> nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}}
};
status = {ASSURED};
}
}
</code></pre>
<p>Fig 15. S’s <code class="language-plaintext highlighter-rouge">nf_conntrack_hash</code> table after A establishes her VPN
tunnel.</p>
</li>
<li>
<p>A sends UDP packets to a victim’s public IP address with
S’s OpenVPN server listening port as A’s source port, creating
a Conntrack entry that S uses for routing packets back to A with. A sends
a packet for each ephemeral port of V.</p>
<p>The c2mitm attack requires that the attacker constant cycle through
the ephemeral space to keep entries fresh until
the victim attempts to connect to the VPN server. The conntrack entries may
expire, so the attacker can either spoof replies to the VPN server with
matching UDP parameters or refresh at a faster rate. Spoofing will place the
entries in the ASSURED state where they cannot be evicted before 180 seconds,
but adds the requirement that the attacker has the ability to spoof packets on
the Internet and doubles the number of packets per port for the initial cycle. Since
spoofing is already a requirement for the c2mitm attack it makes sense to do
so, and somewhat reduces the necessary rate (about 182 packets per second,
assuming the IANA dynamic port range). The attacker may also keep entries
alive by cycling through the ephemeral port space every 30 seconds with a
packet for each port (546 packets/second) and not spoof in this step, if they so desire. If the
victim’s OS complies with <a href="https://datatracker.ietf.org/doc/html/rfc6056#section-3.2">rfc6056, section 3.2</a>, then this is
the worst case and the packet rate to keep conntrack entries fresh for all
possible victim ephemeral ports is only about 717 packets per second. Remember
that these are empty packets, half of which are encrypted and they are split
across tens of thousands of flows.</p>
<pre><code class="language-ascii"> time
| (A)ttacker OpenVPN (S)erver (V)ictim
| |-------------------{src=A:Aport, dst=S:1194}------------------->| | 1.1.
| |<------------------{src=S:1194, dst=A:Aport}--------------------| | 1.2.
| tun0 |===========================TUNNEL===============================| | 1.3.
| |---{src=A:Aport, dst=S:1194, <src=tun0A:1194, dst=V:Vport1>}--->| | 2.1.
| | |---{src=S:1194, dst=V:Vport1}-X | 2.2.
: : : :
| |---{src=A:Aport, dst=S:1194, <src=tun0A:1194, dst=V:VportN>}--->| | 2.3.
| | |---{src=S:1194, dst=V:VportN}-X | 2.4.
V
</code></pre>
<p>Fig 16. Space-time digram depicting execution step (2.) for c2mitm.</p>
<pre><code class="language-ascii"> nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}}
};
status = {ASSURED};
},
H2 .-> nf_conn[.] -> {
timeout = 29; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=tun0A, udp.port=1194}, dst={u3=V, udp.port=Vport1}},
[REPLY] .-> {src={u3=V, udp.port=Vport1}, dst={u3=S, udp.port=1194}}
};
status = {UNREPLIED};
}
:
:
Hk .-> nf_conn[.] -> {
timeout = 29; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=tun0A, udp.port=1194}, dst={u3=V, udp.port=VportN}},
[REPLY] .-> {src={u3=V, udp.port=VportN}, dst={u3=S, udp.port=1194}}
};
status = {UNREPLIED};
}
}
</code></pre>
<p>Fig 17. S’s Conntrack table after executing step (2.) for c2mitm.</p>
</li>
<li>
<p>V sends an OpenVPN connection request to S.</p>
</li>
</ol>
<pre><code class="language-ascii">time
| (A)ttacker OpenVPN (S)erver (V)ictim
| |-------------------{src=A:Aport, dst=S:1194}------------------>| | 1.1.
| |<------------------{src=S:1194, dst=A:Aport}-------------------| | 1.2.
| tun0 |===========================TUNNEL==============================| | 1.3.
| |---{src=A:Aport, dst=S:1194, <src=tun0A:1194, dst=V:Vport>}--->| | 2.1.
| | |---{src=S:1194, dst=V:Vport}-x | 2.2.
| | |<--{src=V:Vport, dst=S:1194}---| 3.1.
| |<--{src=V:1194, dst=A:Aport, <src=V:Vport, dst=tun0A:1194>}----| | 3.2.
| |-------------------{src=A:Vport, dst=S:1194}------------------>| | 3.3.
| |<------------------{src=S:1194, dst=A:Vport}-------------------| | 3.4.
| |---{src=S:1194, dst=A:Vport, <src=tun0A:1194, dst=V:Vport>}--->| | 3.5.
| | |---{src=S:1194, dst=V:Vport}-->| 3.6.
| | |<--{src=V:Vport, dst=S:1194}---| 3.7.
| |<--{src=V:1194, dst=A:Aport, <src=V:Vport, dst=tun0A:1194>}----| | 3.8.
| |---{src=S:1194, dst=A:Vport, <src=tun0A:1194, dst=V:Vport>}--->| | 3.9.
| | |---{src=S:1194, dst=V:Vport}-->| 3.10.
V
</code></pre>
<p>Fig. 18. Network space-time diagram of the c2mitm attack to
completion.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}}
};
status = {ASSURED};
},
H2 .-> nf_conn[.] -> {
timeout = 29; // seconds
tuplehash = {
[ORIGINAL] .-> {src=u3{=tun0A, udp.port=1194}, dst={u3=V, udp.port=Vport}},
[REPLY] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}} ******
};
status = {ASSURED};
}
}
</code></pre>
<p>Fig. 19. Shows the server, S’s Conntrack table just after V sends a connection
request to S and before A has relayed V’s connection request. This is at steps
3.1 and 3.2 of the Fig. 19 above. The asterisks indicate a state of network
connection that should not exist, because V will attempt to initiate a
connection but match a NAT rule as a REPLY.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}}
};
status = {ASSURED};
},
H2 .-> nf_conn[.] -> {
timeout = 29; // seconds
tuplehash = {
[ORIGINAL] .-> {src=u3{=tun0A, udp.port=1194}, dst={u3=V, udp.port=Vport}},
[REPLY] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}}
};
status = {ASSURED};
},
H3 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src=u{3=A, udp.port=Vport}, dst={u3=S, udp.port=1194}},
[REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Vport}}
};
status = {ASSURED};
},
}
</code></pre>
<p>Fig. 20. Shows S’s Conntrack table just after V sends a connection
request to S and after A has relayed the V’s connection request,
successfully become a man-in-the-middle between V and S (steps
3.4-3.10).</p>
<p>As we have shown, both the deanonymization and c2mitm attacks are
possible because of implementation details in Conntrack’s NATing
functionality. The difference in attacks is dependent on the order in
which a victim connects to the same OpenVPN server as the attacker.
Each case is harmful in its own right. The deanonymization attack is harmful
because an often cited use case for VPN software is anonymity. The c2mitm
attack places the attacker in-path for the victim’s connection to the VPN
server, which could lead to DNS or TCP hijacking, traffic analysis, and other
attacks that normally would be outside the reach of an off-path attacker who is
just another VPN client.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Closing the loop</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Not only is the victim deanonymized, but the attacker is also positioned in the
network in-path. From this position, the attacker can leverage a recently
disclosed attack (<a href="https://breakpointingbad.com/papers/Blind-in-path-attacks-VPN-USENIX21.pdf">Tolley2021</a>
) against OpenVPN servers. This attack assumes an in-path
attacker. The attacker spoofs packet to the OpenVPN server from some other
globally routable IP address such as a website or DNS server. The in-path
attacker can use the DNS redirect primitive to send the victim to an attacker
controlled server. If the server the victim was attempting to access is
plaintext HTTP or the attacker has a compromised SSL/TLS certificate, the
attacker has carte blanche over the victim.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">DNS Injection & SYN_SENT2</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Although the attacks we have described above enable the attacker to deanonymize
the victim and redirect their web connections to an attacker-controlled server,
demonstrating the c2mitm attack with TCP shows that the problem is not UDP
specific and may open up other possible attacks that involve other services and
ports.</p>
<p>Assumptions</p>
<ol>
<li>Attacker is in-path</li>
<li>Attacker can spoof packets from a NAT-external DNS server to the OpenVPN server.</li>
</ol>
<p>Execution</p>
<p>The attack is executed in two steps:</p>
<ol>
<li>Send TTL limited TCP SYN packets to the victim’s public IP address through the VPN.
<pre><code class="language-ascii"> time
| (A)ttacker OpenVPN (S)erver (V)ictim
| |----{src=A:80, dst=V:pA}-->|---{src=S:80, dst=V:pA}-x | 1.1. TCP SYN
| |----{src=A:80, dst=V:pB}-->|---{src=S:80, dst=V:pB}-x | 1.2. TCP SYN
. . . .
| |----{src=A:80, dst=V:pN}-->|---{src=S:80, dst=V:pN}-x | 1.3. TCP SYN
V
</code></pre>
<p>Fig. 21. Space-time diagram of A sending TTL limited SYN packets to V on all ephemeral ports.</p>
<pre><code class="language-ascii"> nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}},
[REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
H2 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}},
[REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
.
.
Hn .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}},
[REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
}
</code></pre>
<p>Fig. 21. S’s Conntrack table after all of A’s <code class="language-plaintext highlighter-rouge">N</code> SYN packets traverse Conntrack.</p>
</li>
<li>DNS inject the URL with the OpenVPN server’s IP address</li>
<li>Victim initates 3-way handshake to OpenVPN server</li>
</ol>
<pre><code class="language-ascii">time
| (A)ttacker OpenVPN (S)erver (V)ictim
| |<----{src=V:pA, dst=S:80, tcp.flags=SYN}----|<--{src=V:pA, dst=S:80, tcp.flags=SYN}-----| 1.1.
V
</code></pre>
<p>Fig. 22. Space-time diagram of V sending a SYN for as part of the TCP three-way handshake after a successful
DNS injection by the attacker.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}},
[REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT2}
status = {REPLIED};
},
H2 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}},
[REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
.
.
Hn .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}},
[REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
}
</code></pre>
<p>Fig. 23. S’s Conntrack table after V’s SYN packet.</p>
<pre><code class="language-ascii">time
| (A)ttacker OpenVPN (S)erver (V)ictim
| |<----{src=V:pA, dst=S:80, tcp.flags=SYN}----|<--{src=V:pA, dst=S:80, tcp.flags=SYN}-----| 1.1.
| |---{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->|--{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->| 1.1.
V
</code></pre>
<p>Fig. 24. Space-time diagram of A sending a SYN/ACK as part of the TCP three-way handshake.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}},
[REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_RECV}
status = {REPLIED};
},
H2 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}},
[REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
.
.
Hn .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}},
[REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
}
</code></pre>
<p>Fig. 25. S’s Conntrack table after A’s SYN/ACK packet.</p>
<pre><code class="language-ascii">time
| (A)ttacker OpenVPN (S)erver (V)ictim
| |<----{src=V:pA, dst=S:80, tcp.flags=SYN}----|<--{src=V:pA, dst=S:80, tcp.flags=SYN}-----| 1.1.
| |---{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->|--{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->| 1.1.
| |<----{src=V:pA, dst=S:80, tcp.flags=ACK}----|<--{src=V:pA, dst=S:80, tcp.flags=ACK}-----| 1.1.
V
</code></pre>
<p>Fig. 26. Space-time diagram of V sending the final ACK, completing the TCP three-way handshake.</p>
<pre><code class="language-ascii">nf_conntrack_hash {
H1 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}},
[REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_RECV}
status = {UNASSURED, REPLIED};
},
H2 .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}},
[REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
.
.
Hn .-> nf_conn[.] -> {
timeout = 179; // seconds
tuplehash = {
[ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}},
[REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}}
};
protocol.tcp.state={SYN_SENT}
status = {UNASSURED, UNREPLIED};
},
}
</code></pre>
<p>Fig. 27. S’s Conntrack table after V’s final ACK packet. Notice that the
matching conntrack entry for the TCP connection (H1), is still in the UNASSURED
and SYN_RECV state. This is because, from Netfitler’s perspective, A has not
sent the final ACK to complete the simultaneous open. This also means the
connection cannot be in the ASSURED state because that connection is
technically not established from Netfilter’s perspective.</p>
<p>In addition to using the DNS redirect primitive to send the victim to some
NAT-external server, the attacker can even redirect the victim back to the
in-path machine through which the victim’s traffic are already being routed.
In this ouroboros configuration the attacker uses a similar methodology
for c2mitm, except this time, she sends TTL limited TCP SYN packets through the
VPN server to the victim with whatever port the client thinks the server is
using, e.g., port 80 or 443. Next, when the victim’s (forged) DNS query returns
the VPN server’s IP address, the victim sends a SYN packet to the VPN server.
Because the attacker primed the Conntrack table with SYN packets to each of the
victim’s ephemeral ports, the table is full of <code class="language-plaintext highlighter-rouge">nf_conn</code> entries in the
<code class="language-plaintext highlighter-rouge">SYN_SENT</code> state. When the victim’s SYN packet is translated using the matching
<code class="language-plaintext highlighter-rouge">nf_conn</code> entry, that entry is updated to be in the <code class="language-plaintext highlighter-rouge">SYN_SENT2</code> state and the
victim’s SYN packet is sent to the attacker. Next, the attacker sends an
SYN/ACK to the victim, who finally responds back to the attacker with the final
ACK. From the victim’s perspective, she is initiating a 3-way handshake, but
from the OpenVPN server (NAT)’s perspective, a simultaneous open is happening.
In this way, the attacker tricks the victim into connecting back to the in-path
router. Note that similar tricks can be used to connect to services running on a
victim behind a NAT or perform various port scans against such a victim.
Simultaneous open, described in <a href="https://datatracker.ietf.org/doc/html/rfc793#section-3.4">rfc794, section 3.4</a>, is a less
common connection establishment procedure where two machines initiate a connection between eachother by both exchanging SYNs, SYN/ACKs,
and ACKs similar to how the three-way handshake works.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Potential Mitigations</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>There are two obvious ways to mitigate these issues, to some extent:</p>
<ol>
<li>The server could add firewall rules to prevent the port the VPN service is listening on from being used as a source port by clients.</li>
<li>The NAT functionality could be changed (e.g., as the VPN server is sending
decrypted packets over the virtual interface, or by changing the conntrack
module of Netfilter upstream) so that any port not designated as “Dynamic
and/or Private” by IANA is translated into such a port.</li>
</ol>
<p>Either of these changes would prevent the specific attacks presented above.
However, one can envision other attacks that do not involve specific ports,
such as attacks on BitTorrent users.</p>
<p>A comprehensive fix to this vulnerability would entail somehow modifying the
notions of garbage collection, connection direction, connection status, and
simultaneous open in NAT implementations to be more consistent with the
security and privacy requirements of VPNs.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">NAT Security Considerations in RFCs</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>The RFCs mentioned throughout this disclosure make various references to
security and privacy. However, the level of security provided by a NAT based on
network configuration is not clearly defined. Furthermore, the NAT-router +
server situation, as in the OpenVPN case, gives rise to this particular attack.
This host configuration is not explicitly addressed in the RFCs, though some do
state that the NAT-external IPs and ports should be managed “appropriately” using
firewall rules. The lack of end-to-end security is known and stated in the RFCs
as is the fact that NAT brakes end-to-end model of IP and is simply a
work-around to prolonging IPv4’s lifetime. The following are some specific examples
of security considerations related to our attack.</p>
<p><a href="https://datatracker.ietf.org/doc/html/rfc2663#section-9.0">rfc 2663, section 9</a>, states:</p>
<blockquote>
<p>Many people view traditional NAT router as a one-way (session)
traffic filter, restricting sessions from external hosts into their
machines.</p>
</blockquote>
<p>However, our port shadowing examples, particularly our abuse of TCP simultaneous open,
demonstrate that an attacker can bypass one-way traffic filtering in specification
conditions.</p>
<p><a href="https://datatracker.ietf.org/doc/html/rfc2993#section-10">rfc 2993, section 10</a></p>
<blockquote>
<p>Determine need for public toward private connections, variability
of destinations on the private side, and potential for simultaneous
use of public side port numbers. NAPTs increase administration if
these apply.</p>
</blockquote>
<p>Port shadowing demonstrates why this point is critical to preserving client security and
privacy.</p>
<p><a href="https://datatracker.ietf.org/doc/html/rfc4787#section-13">rfc 4787, section 13</a> states:</p>
<blockquote>
<p>This document recommends that the NAT filters be specific to the
external IP address only (see REQ-8) and not to the external IP
address and UDP port. It can be argued that this is less secure than
using the IP and port. Devices that wish to filter on IP and port do
still comply with these requirements.</p>
</blockquote>
<p>However, port shadowing clearly requires some port-dependent action be taken
by the NAT to mitigate unintentionally forwarding packets to an attacker.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Related RFCs</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>NAT related RFCs:</p>
<ol>
<li><a href="https://datatracker.ietf.org/doc/html/rfc1631">rfc1631</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc2663">rfc2663</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc2993">rfc2993</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc3027">rfc3027</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc4787">rfc4787</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc5382">rfc5382</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc6056">rfc6056</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc6146">rfc6146</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc6888">rfc6888</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc7857">rfc7857</a></li>
</ol>
<p>Protocol RFCs:</p>
<ol>
<li><a href="https://datatracker.ietf.org/doc/html/rfc768">rfc768</a></li>
<li><a href="https://datatracker.ietf.org/doc/html/rfc793">rfc793</a></li>
</ol>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Additional Notes</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>While this write-up covers Netfilter and Linux in detail, we have also tested
our attacks on FreeBSD 13 using natd, ipfw, ipf, and pf. We found that all the
implementations are susepctible to port shadowing, however, only natd appears to
succeptable to a c2mitm using OpenVPN the same was as Netfilter on Linux.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">CPE Information</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>The following CPE information is the software setup for our tests.</p>
<ol>
<li>part=”a”, vendor=”Netfilter”, product=”Netfilter”</li>
<li>part=”o”, vendor=”Linux”, product=”Linux”, version=”4.15.0-142”</li>
<li>part=”a”, vendor=”OpenVPN”, product=”OpenVPN Server”, version=”2.4.4”,</li>
<li>part=”o”, vendor=”FreeBSD”, product=”FreeBSD-13”</li>
<li>part=”a”, vendor=”FreeBSD”, product=”natd”</li>
</ol>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Acknowledgements</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>We would like to thank our colleagues, William J. Tolley, Beau Kujath, and Jeffrey Knockel for
their input and feedback during the analysis and attack execution phases of
this work.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Funding Information</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>This project received funding from the following sources:</p>
<p><img class="image image--xl" src="/assets/darknsf.png" /></p>
<p>This material is based upon work supported by the U.S. National Science
Foundation under Grant Nos. 1801613 and 2007741 . Any opinions, findings and
conclusions or recommendations expressed in this material do not necessarily
reflect the views of the National Science Foundation.</p>
</div>Benjamin Mixon-Baca and Jedidiah R. Crandallvpn-research@breakpointingbad.comCredit: Wikipedia Introduction OpenVPN’s use of Netfilter makes it susceptible to several attacks that can cause denial-of-service, deanonymization of clients, or redirection of a victim client connection to an attacker controlled server. Netfilter is a module within the Linux kernel that implements stateless and stateful firewall mechanisms and network address translation (NAT). Netfilter’s design includes hooks that are called at various points in the networking code to execute, e.g., user-defined firewall rules and NAT code. The NAT portion of Netfilter is designed in such a way that if a machine behind the NAT uses the same source port as an application listening on the same port as the NAT (i.e., the NAT is acting both as a NAT-router and a server), then Netfilter translates and routes received packets intended for the NAT’s own listening port to a host behind the NAT using the same port. This shadowing behavior is not specified in any relevant request for comments (rfc768, rfc793, rfc4787, rfc5382, rfc7857, or any of their successors). The remainder of this disclosure uses the term “port shadow(ing)” when discussing this attack primitive. Port shadowing’s root cause originates from Netfilter’s lack of coordination with the Linux socket infrastructure to determine whether a port in a listening state (or any particular state) on the NAT creates ambiguity with a machine using the same port behind the NAT. Port shadowing has interesting implications for applications, such as OpenVPN, that rely on Netfilter for NAT. A malicious OpenVPN client can use port shadowing to deanonymize victim machines connected to the same OpenVPN server or escalate privileges from an OpenVPN client to a man-in-the-middle (c2mitm) between another client (the victim) and the OpenVPN server to which both the attacker and victim are connected. The c2mitm variation of the attack can be combined with a recently disclosed, server-side attack against OpenVPN to inject DNS responses, reset, or even hijack TCP connections of the victim, even when that connection is tunneled through the OpenVPN server. Version information is at the bottom of this note. While we have not yet tested other versions of OpenVPN, any version of Wireguard or strongSwan, or any other VPN server implementations, we believe our attack applies to any version of any VPN that relies on Linux’s Netfilter for NAT. We have successfully tested the full FreeBSD-13 with natd and OpenVPN. Additionally, we have tested against FreeBSD-13 with IPFW, PF, and IPF and found that, while they are not succesptable to c2mitm, the port shadow still applies. There is no specific implemetation bug with Netfilter, nor presumably FreeBSD’s NAT implementations, that makes port shadowing possible, it is simply an implementation artifact that is apropos of following the RFCs that relate to NAT. Background The port shadow attack primitive is not specific to OpenVPN and is fundamentally related to network address translation. OpenVPN is tightly coupled with the concept of NAT and depends on Netfilter for NAT support, hence, the following discussion about OpenVPN focuses on its role as a NAT since our attack arises from NAT related implementation details. Cryptography is not described in any detail since our attack does not target any cryptographic components. The following background subsections provide information on relevant threat models, OpenVPN, Netfilter, and the port shadow primitive. Threat Models The two threat models most relevant to this disclosure are adjacent and in-path attackers. Building a port shadow assumes a logically adjacent attacker and the DNS injection, the coup de grace covered at the end of this post, requires an in-path attacker. Adjacent MITRE defines a logically adjacent attacker as follows under the attack vector description MITRE: The vulnerable component is bound to the network stack, but the attack is limited at the protocol level to a logically adjacent topology. This can mean an attack must be launched from the same shared physical (e.g., Bluetooth or IEEE 802.11) or logical (e.g., local IP subnet) network, or from within a secure or otherwise limited administrative domain (e.g., MPLS, secure VPN to an administrative network zone). One example of an Adjacent attack would be an ARP (IPv4) or neighbor discovery (IPv6) flood leading to a denial of service on the local LAN segment (e.g., CVE‑2013‑6014). |OpenVPN Server(NAT/router)| / \ . . / \ |routerA| |routerV| | | . . / \ |attacker| |victim| It is important to note that logically adjacent and network adjacent are two different threat models. Our attacks do not make any assupmtions about the attacker being network adjacent. In the context of our attacks, for practical purposes logically adjacent simply means the attacker is connected as a client to the same VPN server as the victim. In-path The in-path attacker is a machine that routes packets between two hosts. For this disclosure, the attacker is between the victim and the OpenVPN server. Readers seeking a formal definition of in-path threats or interesting in-path attacks are referred to Marczak et al.’s work on the Great Cannon. |OpenVPN Server(NAT/router)| | . |attacker| | . | |victim| An attacker can leverage the port shadow primitive to escalate from logically adjacent to in-path against OpenVPN. OpenVPN OpenVPN uses Netfilter to perform NAT for clients connected to the server. A non-exhaustive list of VPN use-cases is: securing network traffic when using public WiFi at hotels, coffee shops or airports hiding illegally torrented content bypassing geo blocked content filters bypassing restrictive firewalls masking your IP address Regardless of the specific use-case, a client must first establish a VPN tunnel between herself and the OpenVPN server. During connection establishment, the victim sends an initial UDP or TCP packet to a listening port on the OpenVPN server (typically 1194, but is configurable and known by the client). The subsequent packets exchanged establish a secure channel using SSL/TLS and various configuration parameters, including the client’s virtual IP address, are pushed to the client. (C)lient OpenVPN (S)erver |-- {src=C:Cport, dst=S:1194} ->| 1. First packet Client sends to OpenVPN server |<- { Exchange parameters over SSL/TLS } ->| 2. Server and Client establish tunnel parameters Fig. 1. Diagram showing the first packet the client sends to the server and the subsequent packets exchanged to establish the client’s tunnel parameters. Once the client has a virtual-IP address that the OpenVPN server associates with her and her routes are configured to send all originating packets through the tunnel, the client may exchange packets between other globally routable IP addresses and they will assume the traffic originated from the OpenVPN server instead of the client. Also, any in-path machines between the OpenVPN client and server will only see encrypted traffic. |----------| |------------------| | Client A |~~~~~| In-path Machine | |----------| |------------------| | |------------| . ~~~~~~~~~~~~| Web Server | | |------------| |----------| |----------------| | Client B |~~~~~| OpenVPN Server | |----------| |----------------| Fig. 2. An example Topology of an OpenVPN Server and two VPN clients. There is also an in-path router between Client A and the OpenVPN server. If Client A (or B or Both) request data from Web Server, Web Server will think the requests came from OpenVPN server, regardless of which client issued the request. In-path Machine will not know if Client A requests a page from Web Server. A recently disclosed attack by Tolley et al. Tolley2021 demonstrates how an attacker can leak connection information or even hijack connections between an OpenVPN client and an end-server. Their attacks require that the attacker be in-path between the OpenVPN client and the OpenVPN server. While in-path capabilities are typically reserved for nation-states, ISP, or similarly powerful adversaries, using network alchemy, we are able to deanonymize an OpenVPN client or transmute a basic OpenVPN client into an in-path attacker. We define network alchemy as the creation of states of network connections that should not exist, using a combination of TTL limiting, packet spoofing, RFC ambiguities, and other low-level network primitives. Conntrack & NAT Conntrack stores state about connections it sees in a single, global hash table called nf_conntrack_hash. struct hlist_nulls_head *nf_conntrack_hash __read_mostly; EXPORT_SYMBOL_GPL(nf_conntrack_hash); Fig. 3. nf_conntrack_hash table declaration. The table is shared by all protocols and packets, regardless of network namespaces or sockets. The table is initialized at module load time and has a maximum size that is a function of the amount of system memory. The size is also partially controlled by the system variable net.ipv4.netfilter.nf_conntrack_max. The fact that the table is global and does not coordinate with the socket infrastructure are the primary factors contributing to port-shadow behavior. Individual connections are represented as entries in nf_conntrack_hash by the nf_conn struct. struct nf_conn { . . . u32 timeout; . . . /* XXX should I move this to the tail ? - Y.K */ /* These are my tuples; original and reply */ struct nf_conntrack_tuple_hash tuplehash[IP_CT_DIR_MAX]; /* Have we seen traffic both ways yet? (bitset) */ unsigned long status; . . . Fig. 4. nf_conn struct including the fields related to this disclosure. The metadata inside each nf_conn entry includes a timeout value, a two-entry array of nf_conntrack_tuple_hash structs (one for the ORIGINAL direction and one for the REPLY direction), and a status bitfield. Each nf_conntrack_tuple_hash contains an nf_conntrack_tuple and each nf_conntrack_tuple contains its own copy of the packet’s direction, source and destination ports and IP addresses, and protocol number. /* Connections have two entries in the hash table: one for each way */ struct nf_conntrack_tuple_hash { struct hlist_nulls_node hnnode; struct nf_conntrack_tuple tuple; }; Fig. 5. nf_conntrack_tuple_hash declaration. /* The manipulable part of the tuple. */ struct nf_conntrack_man { union nf_inet_addr u3; union nf_conntrack_man_proto u; /* Layer 3 protocol */ u_int16_t l3num; }; /* This contains the information to distinguish a connection. */ struct nf_conntrack_tuple { struct nf_conntrack_man src; /* These are the parts of the tuple which are fixed. */ struct { union nf_inet_addr u3; union { /* Add other protocols here. */ __be16 all; struct { __be16 port; } tcp; struct { __be16 port; } udp; struct { u_int8_t type, code; } icmp; struct { __be16 port; } dccp; struct { __be16 port; } sctp; struct { __be16 key; } gre; } u; /* The protocol. */ u_int8_t protonum; /* The direction (for tuplehash) */ u_int8_t dir; } dst; }; Fig. 6. nf_conntrack_tuple and nf_conntrack_man declarations. Notice that only source portion of the tuple is manipulatable. When Conntrack processes a packet, it checks to see whether it is in nf_conntrack_hash. If the packet is part of an existing connection, Conntrack pulls the associated nf_conn entry for the remainder of processing, otherwise, a new nf_conn entry is created and inserted into nf_conntrack_hash. If the nf_conn entry is new, processing includes remapping colliding ports and is performed by get_unique_tuple in Netfilter’s NAT module. As we demonstrate in the Attack Methodology section, Netfitler’s port remapping (and, in special cases, lack thereof) make our attacks possible (port shadowing leverages this behavior). As the packet passes through Netfitler’s NAT hook, its source IP address and port may be overwritten, depending on whether or not ports collide. The direction field represents whether the packet is from the connection originator or replier (i.e., ORIGINAL and REPLY). Conntrack infers the connection’s direction based on whether an nf_conn entry exists for the given source and destination IP addresses, protocol number, and port numbers in the case of UDP and TCP. When Conntrack sees a packet for which no nf_conn entry exists, it assumes that packet is the originator of the connection and assigns nf_conn.tuplehash[ORIGINAL] using the fields of the packet being processed. This idea of connection direction is based on the notion of “start of session”, and Conntrack’s use of ORIGINAL and REPLY directions are consistent with section 2.5. of rfc2663. 2.5. Start of session for TCP, UDP and others The first packet of every TCP session tries to establish a session and contains connection startup information. The first packet of a TCP session may be recognized by the presence of SYN bit and absence of ACK bit in the TCP flags. All TCP packets, with the exception of the first packet, must have the ACK bit set. However, there is no deterministic way of recognizing the start of a UDP based session or any non-TCP session. A heuristic approach would be to assume the first packet with hitherto non-existent session parameters (as defined in section 2.3) as constituting the start of new session. As stated previously, Conntrack stores two copies of the source and destination IP addresses and ports, the protocol number, and the direction in each nf_conn entry in the tuplehash array. In the case where the machine running Conntrack is also the actual sender of packets in a session, the ORIGINAL direction’s source IP address is the machine running Conntrack, the source port is whatever Conntrack’s sending application chose, the destination IP address is some remote host’s IP address, and the destination port is whatever port the remote host is using to communicate with the Conntrack host. In the REPLY direction, the source IP and destination IP are switched and the destination is the Conntrack machine’s IP address (i.e., the source in the ORIGINAL direction). This detail is important because with OpenVPN, from Conntrack’s perspective, the ORIGINAL direction for tunnel initializations should always the OpenVPN client’s IP address. According to Netfilter, maintaining two entries is an optimization for faster lookups and translations. Fig. 7 below provides a generic example of what the nf_conntrack_hash and some nf_conn entry might look like. nf_conntrack_hash { H .-> nf_conn[.] -> { timeout = T; tuplehash = { [ORIGINAL] .-> {src={u3=Conntrack Machine, udp.port=sport}, dst={u3=Remote Server, udp.port=dport}}, [REPLY] .-> {src={u3=Remote Server, udp.port=dport}, dst={u3=Conntrack Machine, udp.port=sport}} status = S; }; } } Fig. 7. nf_conn entry structure for entries in the nf_conntrack_hash table where the NAT is the actually end-point in a connection session. The following case the machine running Conntrack acts as a NAT. When a host behind the NAT (with a private IP) sends a packet out the NAT, the nf_conntrack_tuple[ORIGINAL] direction’s source IP address is the private IP address of the client sending the packet while the source of the REPLY direction is the NAT’s public IP address. Assuming the port of this packet does not collide with other ports NAT is translating, port numbers are swapped without being manipulated. This behavior is related to the c2mitm attack’s use of the port shadowing primitive. Fig. 7 provides a graphical representation of this situation. nf_conntrack_hash { H .-> nf_conn[.] -> { timeout = T; tuplehash = { [ORIGINAL] .-> {src={u3=Machine Behind NAT, udp.port=sport}, dst={u3=Remote Server, udp.port=dport}}, [REPLY] .-> {src={u3=Remote Server, udp.port=dport}, dst={u3=Conntrack Machine, udp.port=sport}} }; status = S; } } Fig. 7. nf_conn entry structure for NAT entries in the nf_conntrack_hash table. If there are port collisions, then NAT will remap the ports, as depicted in the following figure. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = T; tuplehash = { [ORIGINAL] .-> {src={u3=Remote Server, udp.port=Rport}, dst={u3=Conntrack Machine, udp.port=1194}}, [REPLY] .-> {src={u3=Conntrack Machine, udp.port=1194}, dst={u3=Remote Server, udp.port=Rport}} }; status = S; } H2 .-> nf_conn[.] -> { timeout = T; tuplehash = { [ORIGINAL] .-> {src={u3=Machine Behind NAT, udp.port=1194}, dst={u3=Remote Server, udp.port=Rport}}, [REPLY] .-> {src={u3=Remote Server, udp.port=NAT(Rport)}, dst={u3=Conntrack Machine, udp.port=1194}} }; status = S; } } Fig. 8. Depicts nf_conn structure for NAT entries in the nf_conntrack_hash table when there is a port collision. Fig. 8., assume that H1 was in the table before H2. Notice that in the REPLY direction of the H2 entry that src.udp.port is a modified version of Rport, denoted by a function NAT that remaps that port. This is a simplified example of what happens when Netfilter’s NAT module resolves ports that collide in this way. The collision happens because H1 in the ORIGINAL direction and H2 in the REPLY direction would hash to the same entry and hence generate a non-unique tuple which cannot happen if NATing needs to function correctly. We found that a malicious machine connected to the OpenVPN server can shadow the OpenVPN server’s listening port (typically 1194) by sending packets with the same source port as the OpenVPN server’s listening port to arbitrary destination IP address and ports. Because these entries are consulted for NAT before most packet processing and because they are used to reroute packets, any packets matching an entry of this type bypasses any listening sockets (such as the one OpenVPN uses) it was intended for. We will show how an attacker can build a port shadow to deanonymize another OpenVPN client or perform a c2mitm attack against a victim in the Attack Methodology section. One challenge the attacker faces is uncertainty about when a victim will connect to the OpenVPN server. nf_conn entries are kept in the table until their timeout value expires. Conntrack’s garbage collector runs periodically and evicts entries whose timeout has expired or if the table is filled beyond 95\%. In the latter case, Conntrack can evict entries before their timeout has expired, but only if the entry’s status does not have the ASSURED bit set. As packets are exchanged and as Netfilter processes them, the timeout value and status get updated. An attacker can take advantage of this to place entries in the ASSURED state, and periodically send packets through Conntrack to ensure the timeout value never expires which is necessary for the c2mitm attack. Placing an entry in the ASSURED state is protocol dependent and Conntrack requires that each protocol module maintain its state according to that protocol’s requirements. There is no RFC to dictate how this should or must be done, so the module authors make a best effort to determine when the status bits should be updated. In the case of UDP, the ASSURED bit is set whenever packets are exchanged in both directions. /* Returns verdict for packet, and may modify conntracktype */ int nf_conntrack_udp_packet(struct nf_conn *ct, struct sk_buff *skb, unsigned int dataoff, enum ip_conntrack_info ctinfo, const struct nf_hook_state *state) { . . . /* If we've seen traffic both ways, this is some kind of UDP * stream. Set Assured. */ if (test_bit(IPS_SEEN_REPLY_BIT, &ct->status)) { . . . /* Also, more likely to be important, and not a probe */ if (!test_and_set_bit(IPS_ASSURED_BIT, &ct->status)) nf_conntrack_event_cache(IPCT_ASSURED, ct); . . . return NF_ACCEPT; } Fig. 9. ASSURED bit handling for UDP packets in Conntrack. For TCP, Conntrack maintains a shadow TCP state machine of a TCP connection between two hosts whose packet’s the machine running Conntrack routes. This is accomplished using the tcp_conntrack state transition table, a function to determine whether the incoming packet should be routed based on Conntrack policy (nf_conntrack_tcp_packet), and the nf_conn.status field. The nf_conntrack_tcp_packet function contains a code-path that sets the ASSURED bit to 1. /* Returns verdict for packet, or -1 for invalid. */ int nf_conntrack_tcp_packet(struct nf_conn *ct, struct sk_buff *skb, unsigned int dataoff, enum ip_conntrack_info ctinfo, const struct nf_hook_state *state) { . . . } else if (!test_bit(IPS_ASSURED_BIT, &ct->status) && (old_state == TCP_CONNTRACK_SYN_RECV || old_state == TCP_CONNTRACK_ESTABLISHED) && new_state == TCP_CONNTRACK_ESTABLISHED) { /* Set ASSURED if we see valid ack in ESTABLISHED after SYN_RECV or a valid answer for a picked up connection. */ set_bit(IPS_ASSURED_BIT, &ct->status); nf_conntrack_event_cache(IPCT_ASSURED, ct); } . . . return NF_ACCEPT; } Fig. 10. ASSURED bit handling for UDP packets in Conntrack. Using this information, an attacker can keep her entries in the Conntrack table long enough to perform the c2mitm attack. In addition to modifying the status bits of the nf_conn entry, Netfilter’s TCP state machine makes forwarding decisions based on the TCP header and Netfilter’s view of the TCP state between the two TCP end-points. This has implications for which TCP packets can cross the NAT-external/ NAT-internal address realms. TCP simultaneous open in particular causes security issues for OpenVPN clients because an attacker can leverage Netfilter’s behavior when processing SYNs to trick the victim into connecting to some TCP service through the VPN server, back to the client. In the Attack Methodology section, we demonstrate how an attacker can combine c2mitm and DNS injection with simultaneous open to strip the OpenVPN’s tunnel encryption and establish a TCP session with the victim. In summary, we have described Netfilter’s stateful Connection tracking and network address translation implementation (Conntrack and NAT) as they relate to port shadowing. We described how entries are created, stored, used and destroyed by Conntrack. Next, we provide a description of how the attacker builds the port shadowing exploit primitive. Port Shadowing Exploit Primitive We identified an exploit primitive, port shadowing, that an attacker can use to either deanonymize a client connected to an OpenVPN server or perform a privilege escalation from client to man-in-the-middle. The attacks are related to each other and differ mainly in the context of their use of the exploit primitive as well as whether a client connects to the OpenVPN server before or after the attacker. The attacker builds the exploit primitive in two steps. The process is the same for both attacks. Connect to the OpenVPN server. Send packets to a victim’s candidate, public IP address using the OpenVPN server’s listening port as the source port to all destination ports in the IANA ephemeral port range. After step (2.) the OpenVPN server’s nf_conntrack_hash contains 16383 entries. Each entry is capable of routing packets matching the tuples in nf_conntrack_hash back to the attacker’s OpenVPN tunnel interface. We describe in the Attack Methodology section how an attacker uses this to deanonymize the victim or become a man-in-the-middle between the victim and the OpenVPN server. Attack Methodology We now describe our testing environment, the deanonymization and c2mitm attacks, and a third attack that demonstrates that network alchemy can be applied to TCP in addition to UDP connections. Testing Environment We tested the attacks in a virtual environment and on the real Internet. Both environments ran OpenVPN server version 2.4.4 on an Ubuntu 18.04 headless server running Linux kernel version 4.15.0-142. We packaged our virtual environment as a vagrant script and include the required software dependencies and code to reproduce our attack (to the best of our abilities though some assembly may be required). Our live attack was performed against a server we setup and configured ourselves. We deployed our OpenVPN server on a machine with 2048 MB RAM, 1 core, and 55 GB storage using vultr cloud services. We manually installed OpenVPN 2.4.4 on the Ubuntu 18.04 vultr instance using Digital Oceans instructions. Our use of our own server for testing was to eliminate any interference our attacks may create with production clients using commercial or consumer VPN providers. OpenVPN Client Deanonymization Assumptions The deanonymization attack makes the following assumption: The Victim (V) connects to the OpenVPN server (S) before the Attacker (A) carries out any attempt to see if they are connected. This is somewhat obvious, but we specify it as an assumption to make the exact nature of the attack clear. In this example, we assume the VPN uses UDP, however, these attacks are also possible using TCP. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=V, udp.port=Vport}} }; status = {ASSURED}; }, } Fig. 11. S’s Conntrack table with V’s legitimate tunnel entry. A has two abilities: their own credentials to connect to the same OpenVPN server as V, and the ability to spoof packets on the Internet. A may or may not use two different computers for the attack. Execution The deanonymization attack is executed in four steps as follows: A connects to S nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=V, udp.port=Vport}} }; status = {ASSURED}; }, H2 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}} }; status = {ASSURED}; }, } Fig. 12. S’s Conntrack table with V and A’s legitimate tunnel entries. A sends UDP packets from herself to a candidate IP address of V. Each packet has the source port equal to whatever the OpenVPN server’s listening port is (assume 1194), a unique destination port in the IANA ephemeral port range, and is TTL limited such that the UDP packets do not reach the candidate IP. The attacker must cycle through the victim’s ephemeral port space only once. This requires 16383 packets per candidate victim IP (assuming the operating system uses the IANA dynamic port range, some operating systems the victim may be using have a larger range). The attacker can check candidate victim IPs in parallel and send at any reasonable speed, and note also that the attacker does not need to check every one of the roughly 4 billion possible IPv4 addresses if they have a profile of the victim, such as what city they connect to the VPN from. We spoof responses in subsequent steps, which imposes a 180 second timeout (for the conntrack entry to be garbage collected) before we can check again if this particular candidate victim IP has connected. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=V, udp.port=Vport}} }; status = {ASSURED}; }, H2 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}} }; status = {ASSURED}; }, . . . Hn .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=tun0A, port=1194}, dst={u3=V, udp.port=Vport}}, [REPLY] .-> {src={u3=V, port=NAT(Vport)}, dst={u3=S, udp.port=1194}} }; status = {ASSURED}; }, } Fig. 13. S’s Conntrack table with V and A’s legitimate tunnel entry and A’s entry whose destination port in the ORIGINAL direction collides with V’s legitimate Conntrack entry’s source port in the Conntrack table. The entries are denoted by entries H1, H2, and Hn, respectively. From a seperate machine, A spoofs UDP responses for each of the UDP packets sent in (2.) from the candidate IP to S. A’s machine connected to S collects the responses sent in (3.) routed back to her by S. If A receives responses for every ephemeral port sent out in step (2.), then she knows the candidate IP is not connected to the same OpenVPN server (S) as her. If A observes one missing ephemeral port response, then she knows the candidate IP is V and is connected to the same OpenVPN server as her (she can repeat missing probes as needed to account for packet loss). This is because if V is connected to the same OpenVPN server (S) as her, then S’s Conntrack table will have an nf_conn entry with S’s IP and OpenVPN port (1194) and V’s public IP address and ephemeral port. This entry makes it impossible for packets sent from A:1194 to V:Vport to be differentiated from the original tuple. Conntrack/NAT must select a new destination port for the packet sent in step (2.) to maintain transparency and operational requirements specified in the RFCs. OpenVPN Client-to-Man-in-the-Middle (c2mitm) Assumptions The c2mitm attack makes the following assumptions: The attacker (A) knows the candidate, public IP address for a potential Victim (V). V does not connect to the OpenVPN server (S) until after (A) builds the exploit primitive. Assumption (1.) is viable in two situations. The first is if A has performed the deanonymization attack in the past, and enumerated one or more victims that includes V. The second is if V is at the same coffee shop or hotel as A. The hotel or coffee shop is likely running its own NAT to share internet resources with all the customers using the network. A can easily discover the public IP address of the establishment. We imagine (2.) holding because, as stated above, A can hold nf_conn entries in the OpenVPN server’s table indefinitely. Execution The c2mitm attack is executed in three stages: A connects to S time | (A)ttacker OpenVPN (S)erver (V)ictim | |---{src=A:Aport, dst=S:1194}--->| 1.1. | |<--{src=S:1194, dst=A:Aport}----| 1.2. | tun0 |=============TUNNEL=============| 1.3. V Fig 14. High-level space-time network diagram of A establishing OpenVPN tunnel to S. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}} }; status = {ASSURED}; } } Fig 15. S’s nf_conntrack_hash table after A establishes her VPN tunnel. A sends UDP packets to a victim’s public IP address with S’s OpenVPN server listening port as A’s source port, creating a Conntrack entry that S uses for routing packets back to A with. A sends a packet for each ephemeral port of V. The c2mitm attack requires that the attacker constant cycle through the ephemeral space to keep entries fresh until the victim attempts to connect to the VPN server. The conntrack entries may expire, so the attacker can either spoof replies to the VPN server with matching UDP parameters or refresh at a faster rate. Spoofing will place the entries in the ASSURED state where they cannot be evicted before 180 seconds, but adds the requirement that the attacker has the ability to spoof packets on the Internet and doubles the number of packets per port for the initial cycle. Since spoofing is already a requirement for the c2mitm attack it makes sense to do so, and somewhat reduces the necessary rate (about 182 packets per second, assuming the IANA dynamic port range). The attacker may also keep entries alive by cycling through the ephemeral port space every 30 seconds with a packet for each port (546 packets/second) and not spoof in this step, if they so desire. If the victim’s OS complies with rfc6056, section 3.2, then this is the worst case and the packet rate to keep conntrack entries fresh for all possible victim ephemeral ports is only about 717 packets per second. Remember that these are empty packets, half of which are encrypted and they are split across tens of thousands of flows. time | (A)ttacker OpenVPN (S)erver (V)ictim | |-------------------{src=A:Aport, dst=S:1194}------------------->| | 1.1. | |<------------------{src=S:1194, dst=A:Aport}--------------------| | 1.2. | tun0 |===========================TUNNEL===============================| | 1.3. | |---{src=A:Aport, dst=S:1194, <src=tun0A:1194, dst=V:Vport1>}--->| | 2.1. | | |---{src=S:1194, dst=V:Vport1}-X | 2.2. : : : : | |---{src=A:Aport, dst=S:1194, <src=tun0A:1194, dst=V:VportN>}--->| | 2.3. | | |---{src=S:1194, dst=V:VportN}-X | 2.4. V Fig 16. Space-time digram depicting execution step (2.) for c2mitm. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}} }; status = {ASSURED}; }, H2 .-> nf_conn[.] -> { timeout = 29; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=tun0A, udp.port=1194}, dst={u3=V, udp.port=Vport1}}, [REPLY] .-> {src={u3=V, udp.port=Vport1}, dst={u3=S, udp.port=1194}} }; status = {UNREPLIED}; } : : Hk .-> nf_conn[.] -> { timeout = 29; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=tun0A, udp.port=1194}, dst={u3=V, udp.port=VportN}}, [REPLY] .-> {src={u3=V, udp.port=VportN}, dst={u3=S, udp.port=1194}} }; status = {UNREPLIED}; } } Fig 17. S’s Conntrack table after executing step (2.) for c2mitm. V sends an OpenVPN connection request to S. time | (A)ttacker OpenVPN (S)erver (V)ictim | |-------------------{src=A:Aport, dst=S:1194}------------------>| | 1.1. | |<------------------{src=S:1194, dst=A:Aport}-------------------| | 1.2. | tun0 |===========================TUNNEL==============================| | 1.3. | |---{src=A:Aport, dst=S:1194, <src=tun0A:1194, dst=V:Vport>}--->| | 2.1. | | |---{src=S:1194, dst=V:Vport}-x | 2.2. | | |<--{src=V:Vport, dst=S:1194}---| 3.1. | |<--{src=V:1194, dst=A:Aport, <src=V:Vport, dst=tun0A:1194>}----| | 3.2. | |-------------------{src=A:Vport, dst=S:1194}------------------>| | 3.3. | |<------------------{src=S:1194, dst=A:Vport}-------------------| | 3.4. | |---{src=S:1194, dst=A:Vport, <src=tun0A:1194, dst=V:Vport>}--->| | 3.5. | | |---{src=S:1194, dst=V:Vport}-->| 3.6. | | |<--{src=V:Vport, dst=S:1194}---| 3.7. | |<--{src=V:1194, dst=A:Aport, <src=V:Vport, dst=tun0A:1194>}----| | 3.8. | |---{src=S:1194, dst=A:Vport, <src=tun0A:1194, dst=V:Vport>}--->| | 3.9. | | |---{src=S:1194, dst=V:Vport}-->| 3.10. V Fig. 18. Network space-time diagram of the c2mitm attack to completion. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}} }; status = {ASSURED}; }, H2 .-> nf_conn[.] -> { timeout = 29; // seconds tuplehash = { [ORIGINAL] .-> {src=u3{=tun0A, udp.port=1194}, dst={u3=V, udp.port=Vport}}, [REPLY] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}} ****** }; status = {ASSURED}; } } Fig. 19. Shows the server, S’s Conntrack table just after V sends a connection request to S and before A has relayed V’s connection request. This is at steps 3.1 and 3.2 of the Fig. 19 above. The asterisks indicate a state of network connection that should not exist, because V will attempt to initiate a connection but match a NAT rule as a REPLY. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, udp.port=Aport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Aport}} }; status = {ASSURED}; }, H2 .-> nf_conn[.] -> { timeout = 29; // seconds tuplehash = { [ORIGINAL] .-> {src=u3{=tun0A, udp.port=1194}, dst={u3=V, udp.port=Vport}}, [REPLY] .-> {src={u3=V, udp.port=Vport}, dst={u3=S, udp.port=1194}} }; status = {ASSURED}; }, H3 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src=u{3=A, udp.port=Vport}, dst={u3=S, udp.port=1194}}, [REPLY] .-> {src={u3=S, udp.port=1194}, dst={u3=A, udp.port=Vport}} }; status = {ASSURED}; }, } Fig. 20. Shows S’s Conntrack table just after V sends a connection request to S and after A has relayed the V’s connection request, successfully become a man-in-the-middle between V and S (steps 3.4-3.10). As we have shown, both the deanonymization and c2mitm attacks are possible because of implementation details in Conntrack’s NATing functionality. The difference in attacks is dependent on the order in which a victim connects to the same OpenVPN server as the attacker. Each case is harmful in its own right. The deanonymization attack is harmful because an often cited use case for VPN software is anonymity. The c2mitm attack places the attacker in-path for the victim’s connection to the VPN server, which could lead to DNS or TCP hijacking, traffic analysis, and other attacks that normally would be outside the reach of an off-path attacker who is just another VPN client. Closing the loop Not only is the victim deanonymized, but the attacker is also positioned in the network in-path. From this position, the attacker can leverage a recently disclosed attack (Tolley2021 ) against OpenVPN servers. This attack assumes an in-path attacker. The attacker spoofs packet to the OpenVPN server from some other globally routable IP address such as a website or DNS server. The in-path attacker can use the DNS redirect primitive to send the victim to an attacker controlled server. If the server the victim was attempting to access is plaintext HTTP or the attacker has a compromised SSL/TLS certificate, the attacker has carte blanche over the victim. DNS Injection & SYN_SENT2 Although the attacks we have described above enable the attacker to deanonymize the victim and redirect their web connections to an attacker-controlled server, demonstrating the c2mitm attack with TCP shows that the problem is not UDP specific and may open up other possible attacks that involve other services and ports. Assumptions Attacker is in-path Attacker can spoof packets from a NAT-external DNS server to the OpenVPN server. Execution The attack is executed in two steps: Send TTL limited TCP SYN packets to the victim’s public IP address through the VPN. time | (A)ttacker OpenVPN (S)erver (V)ictim | |----{src=A:80, dst=V:pA}-->|---{src=S:80, dst=V:pA}-x | 1.1. TCP SYN | |----{src=A:80, dst=V:pB}-->|---{src=S:80, dst=V:pB}-x | 1.2. TCP SYN . . . . | |----{src=A:80, dst=V:pN}-->|---{src=S:80, dst=V:pN}-x | 1.3. TCP SYN V Fig. 21. Space-time diagram of A sending TTL limited SYN packets to V on all ephemeral ports. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}}, [REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, H2 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}}, [REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, . . Hn .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}}, [REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, } Fig. 21. S’s Conntrack table after all of A’s N SYN packets traverse Conntrack. DNS inject the URL with the OpenVPN server’s IP address Victim initates 3-way handshake to OpenVPN server time | (A)ttacker OpenVPN (S)erver (V)ictim | |<----{src=V:pA, dst=S:80, tcp.flags=SYN}----|<--{src=V:pA, dst=S:80, tcp.flags=SYN}-----| 1.1. V Fig. 22. Space-time diagram of V sending a SYN for as part of the TCP three-way handshake after a successful DNS injection by the attacker. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}}, [REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT2} status = {REPLIED}; }, H2 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}}, [REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, . . Hn .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}}, [REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, } Fig. 23. S’s Conntrack table after V’s SYN packet. time | (A)ttacker OpenVPN (S)erver (V)ictim | |<----{src=V:pA, dst=S:80, tcp.flags=SYN}----|<--{src=V:pA, dst=S:80, tcp.flags=SYN}-----| 1.1. | |---{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->|--{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->| 1.1. V Fig. 24. Space-time diagram of A sending a SYN/ACK as part of the TCP three-way handshake. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}}, [REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_RECV} status = {REPLIED}; }, H2 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}}, [REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, . . Hn .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}}, [REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, } Fig. 25. S’s Conntrack table after A’s SYN/ACK packet. time | (A)ttacker OpenVPN (S)erver (V)ictim | |<----{src=V:pA, dst=S:80, tcp.flags=SYN}----|<--{src=V:pA, dst=S:80, tcp.flags=SYN}-----| 1.1. | |---{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->|--{src=V:pA, dst=S:80, tcp.flags=SYN/ACK}->| 1.1. | |<----{src=V:pA, dst=S:80, tcp.flags=ACK}----|<--{src=V:pA, dst=S:80, tcp.flags=ACK}-----| 1.1. V Fig. 26. Space-time diagram of V sending the final ACK, completing the TCP three-way handshake. nf_conntrack_hash { H1 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pA}}, [REPLY] .-> {src={u3=V, tcp.port=pA}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_RECV} status = {UNASSURED, REPLIED}; }, H2 .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pB}}, [REPLY] .-> {src={u3=V, tcp.port=pB}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, . . Hn .-> nf_conn[.] -> { timeout = 179; // seconds tuplehash = { [ORIGINAL] .-> {src={u3=A, tcp.port=80}, dst={u3=V, tcp.port=pN}}, [REPLY] .-> {src={u3=V, tcp.port=pN}, dst={u3=S, tcp.port=80}} }; protocol.tcp.state={SYN_SENT} status = {UNASSURED, UNREPLIED}; }, } Fig. 27. S’s Conntrack table after V’s final ACK packet. Notice that the matching conntrack entry for the TCP connection (H1), is still in the UNASSURED and SYN_RECV state. This is because, from Netfitler’s perspective, A has not sent the final ACK to complete the simultaneous open. This also means the connection cannot be in the ASSURED state because that connection is technically not established from Netfilter’s perspective. In addition to using the DNS redirect primitive to send the victim to some NAT-external server, the attacker can even redirect the victim back to the in-path machine through which the victim’s traffic are already being routed. In this ouroboros configuration the attacker uses a similar methodology for c2mitm, except this time, she sends TTL limited TCP SYN packets through the VPN server to the victim with whatever port the client thinks the server is using, e.g., port 80 or 443. Next, when the victim’s (forged) DNS query returns the VPN server’s IP address, the victim sends a SYN packet to the VPN server. Because the attacker primed the Conntrack table with SYN packets to each of the victim’s ephemeral ports, the table is full of nf_conn entries in the SYN_SENT state. When the victim’s SYN packet is translated using the matching nf_conn entry, that entry is updated to be in the SYN_SENT2 state and the victim’s SYN packet is sent to the attacker. Next, the attacker sends an SYN/ACK to the victim, who finally responds back to the attacker with the final ACK. From the victim’s perspective, she is initiating a 3-way handshake, but from the OpenVPN server (NAT)’s perspective, a simultaneous open is happening. In this way, the attacker tricks the victim into connecting back to the in-path router. Note that similar tricks can be used to connect to services running on a victim behind a NAT or perform various port scans against such a victim. Simultaneous open, described in rfc794, section 3.4, is a less common connection establishment procedure where two machines initiate a connection between eachother by both exchanging SYNs, SYN/ACKs, and ACKs similar to how the three-way handshake works. Potential Mitigations There are two obvious ways to mitigate these issues, to some extent: The server could add firewall rules to prevent the port the VPN service is listening on from being used as a source port by clients. The NAT functionality could be changed (e.g., as the VPN server is sending decrypted packets over the virtual interface, or by changing the conntrack module of Netfilter upstream) so that any port not designated as “Dynamic and/or Private” by IANA is translated into such a port. Either of these changes would prevent the specific attacks presented above. However, one can envision other attacks that do not involve specific ports, such as attacks on BitTorrent users. A comprehensive fix to this vulnerability would entail somehow modifying the notions of garbage collection, connection direction, connection status, and simultaneous open in NAT implementations to be more consistent with the security and privacy requirements of VPNs. NAT Security Considerations in RFCs The RFCs mentioned throughout this disclosure make various references to security and privacy. However, the level of security provided by a NAT based on network configuration is not clearly defined. Furthermore, the NAT-router + server situation, as in the OpenVPN case, gives rise to this particular attack. This host configuration is not explicitly addressed in the RFCs, though some do state that the NAT-external IPs and ports should be managed “appropriately” using firewall rules. The lack of end-to-end security is known and stated in the RFCs as is the fact that NAT brakes end-to-end model of IP and is simply a work-around to prolonging IPv4’s lifetime. The following are some specific examples of security considerations related to our attack. rfc 2663, section 9, states: Many people view traditional NAT router as a one-way (session) traffic filter, restricting sessions from external hosts into their machines. However, our port shadowing examples, particularly our abuse of TCP simultaneous open, demonstrate that an attacker can bypass one-way traffic filtering in specification conditions. rfc 2993, section 10 Determine need for public toward private connections, variability of destinations on the private side, and potential for simultaneous use of public side port numbers. NAPTs increase administration if these apply. Port shadowing demonstrates why this point is critical to preserving client security and privacy. rfc 4787, section 13 states: This document recommends that the NAT filters be specific to the external IP address only (see REQ-8) and not to the external IP address and UDP port. It can be argued that this is less secure than using the IP and port. Devices that wish to filter on IP and port do still comply with these requirements. However, port shadowing clearly requires some port-dependent action be taken by the NAT to mitigate unintentionally forwarding packets to an attacker. Related RFCs NAT related RFCs: rfc1631 rfc2663 rfc2993 rfc3027 rfc4787 rfc5382 rfc6056 rfc6146 rfc6888 rfc7857 Protocol RFCs: rfc768 rfc793 Additional Notes While this write-up covers Netfilter and Linux in detail, we have also tested our attacks on FreeBSD 13 using natd, ipfw, ipf, and pf. We found that all the implementations are susepctible to port shadowing, however, only natd appears to succeptable to a c2mitm using OpenVPN the same was as Netfilter on Linux. CPE Information The following CPE information is the software setup for our tests. part=”a”, vendor=”Netfilter”, product=”Netfilter” part=”o”, vendor=”Linux”, product=”Linux”, version=”4.15.0-142” part=”a”, vendor=”OpenVPN”, product=”OpenVPN Server”, version=”2.4.4”, part=”o”, vendor=”FreeBSD”, product=”FreeBSD-13” part=”a”, vendor=”FreeBSD”, product=”natd” Acknowledgements We would like to thank our colleagues, William J. Tolley, Beau Kujath, and Jeffrey Knockel for their input and feedback during the analysis and attack execution phases of this work. Funding Information This project received funding from the following sources: This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. 1801613 and 2007741 . Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation.Blind In/On-Path Attacks and Applications to VPNs2021-07-04T00:00:00-06:002021-07-04T00:00:00-06:00https://www.breakpointingbad.com/2021/07/04/Blind-In-Path-Attacks<p><a href="javascript:;" class="button button--success button--xl">Abstract</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Protecting network protocols within an encrypted tunnel, using technologies such as Virtual Private Networks (VPNs), is increasingly important to millions of users needing solutions to evade censorship or protect their traffic against in/on-path observers/attackers. In this paper, we present a series of attacks from two threat models: an attacker that can inject spoofed packets into the network stack of a VPN client (called client-side), and an attacker that can spoof packets on the Internet and send them to a VPN server (called server-side). In both cases, we assume that the attacker is in/on-path, and can count encrypted bytes or packets over time. In both threat models, we demonstrate attacks to infer the existence of, interfere with, or inject data into TCP connections forwarded through the encrypted VPN tunnel. In the server-side threat model, we also demonstrate an attack to hijack tunneled DNS queries and completely remove the protections of the VPN tunnel. For the attacks presented in this paper, we (1) assess their feasibility in terms of packet rates and timing; (2) test their applicability against a broad range of VPN technologies, types, and vendors; and (3) consider practical issues with respect to real-world attacks. We followed an ethical disclosure process for all attacks presented in this paper. Client-side attacks were addressed with two CVEs and partially mitigated by a series of updates from some operating system and VPN client vendors. Server-side attacks have not been addressed and are still feasible with all operating systems and VPN servers that we tested.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Open Access Media</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Breakpointing Bad is committed to providing all of our research materials, source code, and virtual environments free of any barriers that would limit access or reproducibility. We provide links to a preprint of our paper (set to appear in USENIX Security ‘21) and a public repository of our source material below.</p>
<p><a href="https://breakpointingbad.com/papers/Blind-in-path-attacks-VPN-USENIX21.pdf">Blind In/On-Path Attacks and Applications to VPNs - PDF</a></p>
<p><a href="https://git.breakpointingbad.com/Breakpointing-Bad-Public/vpn-attacks">Public Repository with Source Code, Demos, and Virtual Environment Scripts</a></p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Previous Posts</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p><a href="https://breakpointingbad.com/2020/05/25/Vintage-Protocol-Nonsense.html">Vintage Protocol Nonsense: Annoying the TCP Stack to Uncover Tunneled VPN Connections</a></p>
<p><a href="https://breakpointingbad.com/2020/08/12/VPN-FAQ.html">Blind In/On-Path Attack Disclosure FAQ</a></p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Funding Information</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>This project received funding from the following sources:</p>
<p><img class="image image--xl" src="/assets/darkotf.png" /></p>
<p>OTF's <em>Information Controls Fellowship Program</em> (ICFP) supports examination into how governments in countries, regions, or areas of OTF's core focus are restricting the free flow of information, impeding access to the open internet, and implementing censorship mechanisms, thereby threatening the ability of global citizens to exercise basic human rights and democracy. The program supports fellows to work within host organizations that are established centers of expertise by offering competitively paid fellowships for three, six, nine, or twelve months in duration.</p>
<p><img class="image image--xl" src="/assets/darknsf.png" /></p>
<p>This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. 1518878, 1518523, and 1801613. Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation.</p>
<p><img class="image image--xl" src="/assets/ministry.png" /></p>
<p>This material is based upon work supported by the Ministry of Science and Innovation (Spain) (PID2019-111429RB-C22). Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the Ministry of Science and Innovation.</p>
</div>William J. Tolley, Beau Kujath, Mohammad Taha Khan, Narseo Vallina-Rodriguez, and Jedidiah R. Crandallvpn-research@breakpointingbad.comAbstract Protecting network protocols within an encrypted tunnel, using technologies such as Virtual Private Networks (VPNs), is increasingly important to millions of users needing solutions to evade censorship or protect their traffic against in/on-path observers/attackers. In this paper, we present a series of attacks from two threat models: an attacker that can inject spoofed packets into the network stack of a VPN client (called client-side), and an attacker that can spoof packets on the Internet and send them to a VPN server (called server-side). In both cases, we assume that the attacker is in/on-path, and can count encrypted bytes or packets over time. In both threat models, we demonstrate attacks to infer the existence of, interfere with, or inject data into TCP connections forwarded through the encrypted VPN tunnel. In the server-side threat model, we also demonstrate an attack to hijack tunneled DNS queries and completely remove the protections of the VPN tunnel. For the attacks presented in this paper, we (1) assess their feasibility in terms of packet rates and timing; (2) test their applicability against a broad range of VPN technologies, types, and vendors; and (3) consider practical issues with respect to real-world attacks. We followed an ethical disclosure process for all attacks presented in this paper. Client-side attacks were addressed with two CVEs and partially mitigated by a series of updates from some operating system and VPN client vendors. Server-side attacks have not been addressed and are still feasible with all operating systems and VPN servers that we tested. Open Access Media Breakpointing Bad is committed to providing all of our research materials, source code, and virtual environments free of any barriers that would limit access or reproducibility. We provide links to a preprint of our paper (set to appear in USENIX Security ‘21) and a public repository of our source material below. Blind In/On-Path Attacks and Applications to VPNs - PDF Public Repository with Source Code, Demos, and Virtual Environment Scripts Previous Posts Vintage Protocol Nonsense: Annoying the TCP Stack to Uncover Tunneled VPN Connections Blind In/On-Path Attack Disclosure FAQ Funding Information This project received funding from the following sources: OTF's Information Controls Fellowship Program (ICFP) supports examination into how governments in countries, regions, or areas of OTF's core focus are restricting the free flow of information, impeding access to the open internet, and implementing censorship mechanisms, thereby threatening the ability of global citizens to exercise basic human rights and democracy. The program supports fellows to work within host organizations that are established centers of expertise by offering competitively paid fellowships for three, six, nine, or twelve months in duration. This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. 1518878, 1518523, and 1801613. Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the Ministry of Science and Innovation (Spain) (PID2019-111429RB-C22). Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the Ministry of Science and Innovation.Blind In/On-Path Attack Disclosure FAQ2020-08-12T00:00:00-06:002020-08-12T00:00:00-06:00https://www.breakpointingbad.com/2020/08/12/VPN-FAQ<div style="width:100%; margin: auto;" align="justify">
<p>In November of last year we reported a vulnerability that allowed to a network adjacent attacker to make inferences about active connections inside VPN-tunneled connections and inject data to reset or hijack these connections. This vulnerability was assigned <a href="https://seclists.org/oss-sec/2019/q4/122">CVE-2019-14899</a> and affected Apple, Android, and many Linux and BSD systems. You can read about this in detail in our <a href="https://breakpointingbad.com/2020/05/25/Vintage-Protocol-Nonsense.html">post</a> from May. This post describes a new attack which takes advantage of the insight gained from the previous attack, but expands the threat to include any in-path router, such as your internet service provider. We have included a <a href="#faq">section</a> with the most commonly asked questions and will update this post as we receive more correspondence.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Background</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>In the previous attack, a network adjacent attacker can see the traffic as it passes from the VPN client to the VPN server (and vice versa), and although it is encrypted, they can learn information about the TCP headers by making some good educated guesses. This attack first required the attacker to learn the private, virtual IP that has been assigned to the client by spoofing packets to the client across the private address space. When the attacker guesses correctly, they will see a response from the client (an encrypted RST packet), otherwise, the client will not respond.</p>
<p>Once the attacker knows the virtual IP address, they then can test a connection to an arbitrary website by spoofing packets to the client with the destination address of the virtual interface and the source address of the website. The process is similar to the first step and the attacker will see a response from the client if there is a connection, and no response if the attacker has guessed incorrectly. After the attacker has established that there is an active connection, they can continue to probe the client and learn enough to find an in-window sequence and acknowledgment number. They can then use this to inject arbitrary data into the connection.</p>
<p>This attack works because without some kind of source address validation, such as reverse path filtering, the kernel will process packets received on an incorrect interface as legitimate. This attack can be mitigated by enabling reverse path filtering or adding an iptables/nftables rule to drop packets that are received on an incorrect interface. Some vendors and developers, such as WireGuard, Private Internet Acess, ProtonVPN, and Mullvad have patched their software to protect clients from this attack.</p>
<p>For this new kind of attack, however, we show how one can attack “the other end of the tunnel” from the perspective of an in-path router and assume that reverse path filtering has been enabled. In this scenario the attacker is still able to perform the same attack, as well as inject malicious DNS responses. We disclosed this vulnerability to a number of mailing lists, including OSS private, Linux Kernel Security, OpenVPN, WireGuard, Android, Apple, and Microsoft. The most important aspect of this attack is that it expands the vulnerable client systems to include any operating system and VPN software, including Windows and policy-based VPNs.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Threat Model</a></p>
<div style="width:100%; margin: auto;" align="justify" id="virtlab">
<p>The previous attack considered the threat of using a VPN on public WiFi to add an additional layer of security, but this attack includes any machine between the VPN client and the VPN server with the ability to spoof packets. We have created a virtual lab to test this vulnerability, as shown in the figure below. In this scenario, the attacker is spoofing packets from Router 1. The packets are being sent to the VPN server with the source address of the website at the other end of the connection. In a real world scenario, this threat would be the ISP being used by the victim or any machine the packets traverse on their way to the VPN server.</p>
<p><a href="/assets/virtlab.jpg"><img src="/assets/virtlab.jpg" alt="Threat Model" /></a></p>
</div>
<p> </p>
<div style="width:100%; margin: auto;" align="justify" id="faq">
<p><a href="javascript:;" class="button button--success button--xl">FAQ</a></p>
<p>In this section we will provide common questions received from developers and kernel maintainers in hopes to clear up any misunderstandings. If you have any questions, feel free to email us at <a href="mailto:vpn-research@breakpointingbad.com">vpn-research@breakpointingbad.com</a>.</p>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Q. Doesn’t enabling reverse path filtering or martian filtering prevent this attack?</a></p>
<p><a href="javascript:;"><em>Asked by Anthony Liguori and Colm MacCárthaigh</em></a></p>
<p>The reverse path filtering settings of the VPN client or server can be anything and the attack will still work. We have reverse path filtering <em>enabled</em> on both client and server in our experiments.</p>
<p>Martian filtering probably is not relevant because the attacker never sends to or from non-routable Internet addresses (except when we use local IPs as Internet addresses in our virtual network). The spoofed packets have the source IP of the web server and the destination IP of the VPN server.</p>
<p>The issue here is that even with reverse path filtering enabled on the VPN client and the gateway, we can perform this attack from any router that is in-path between the client and the VPN server. The mitigation that was previously offered was for the client to enable reverse path filtering, or some other form of source address validation, but this illustrates that this problem requires more than a client-side mitigation.</p>
<p>The routers on the internet do not have source address validation, so this attack also makes policy-based VPNs (with the configuration suggested by Noel Kuntze in the conversation from last December) and operating systems that enforce the strong host model, including Windows, vulnerable. This isn’t the case with the vulnerability as reported in November.</p>
<p>The reverse path filtering we’re talking about is the RFC 3704 kind, but specifically we’re talking about the hosts (VPN server and VPN client). So it has to do with which interfaces they allow packets to come in on. The packets we spoof for the attack enter the VPN server on the same Interface as legitimate packets, and the source IP address is identical to legitimate traffic. So if reverse path filtering is set to strict mode on the VPN server, the incoming spoofed packet will be checked to make sure that any replies to it would be routed out the same interface it’s coming in. Since it’s the Internet-facing interface in all cases (spoofed or legitimate, coming or going) reverse path filtering ala RFC 3704 never comes into play. Then the VPN server NATs the packets into the VPN tunnel and the VPN client receives them through the tunnel on the same virtual interface where legitimate packets come from. So strict reverse path filtering on the client doesn’t make a difference.</p>
<p>Strict reverse path filtering can be carried out by edge networks but if you get deep enough into the Internet then asymmetrical routing makes it impossible (as far as we know, none of us have had the opportunity to play with non-edge networks on the Internet before). Referring to our <a href="#virtlab">setup</a>, there’s an opportunity for router 2 to block the spoofed packets using the reasoning that router 1 would never route packets from the website, but that would be some kind of special case (like the website and VPN server are both customers of the same ISP, or router 1 and router 2 are owned by different ISPs that collaborate to validate source IP addresses in some way).</p>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">Doesn’t BCP38 prevent this attack? Is this an issue of a misconfigured network?</a></p>
<p><a href="javascript:;"><em>Asked by Colm MacCárthaigh</em></a></p>
<p>BCP38 also seems like it isn’t a major factor. For our own research (which is largely focused on journalists and activists in Asia) we’re specifically interested in attacks that the network infrastructure could carry out on users, the kinds of attacks where it is sometimes assumed that VPNs will protect against them. So if a router is carrying out the attack an upstream router would need to apply the BCP38 filtering, which we would expect to see only for smaller ISPs that only get their Internet through one gateway and don’t route any traffic for others. And any router that was limited by such filtering could easily spoof the packets from another part of the Internet where spoofing is possible. Put another way, we expect that most nations and most major ISPs would have the ability to spoof packets.</p>
<p>Just to be clear, in our new attacks the VPN client never sees spoofed packets anywhere else other than coming through the VPN tunnel on the same virtual interface as legitimate packets. And, as for the VPN server, spoofed packets come in the same interface and have IP and TCP headers identical to legitimate packets (except for whatever field we’re guessing, such as the ephemeral source port).</p>
<p>We mostly have experience with edge networks and would be curious to find out more about how BCP38 is deployed in practice on the parts of the Internet where asymmetrical routing comes into play. We recently moved from the University of New Mexico to Arizona State University. While at UNM we had a VLAN that took us all the way to the Albuqueruque Gigapop, and we never had any trouble spoofing any return IP address we wanted to from there. As far as we know there is source address validation on a national level in some places (e.g., China does it on a national level for IPv6), but mostly networks reason about reverse routes only in local ways. Here are a couple of papers where we used source IP spoofing:</p>
<p><a href="https://ensa.fi/papers/Ensafi2014c.pdf">Detecting Intentional Packet Drops on theInternet via TCP/IP Side Channels</a></p>
<p><a href="https://ieeexplore.ieee.org/document/7218441">Original SYN: Finding Machines Hidden Behind Firewalls</a></p>
<p>The second one also has an interesting result that only 23.9% of the networks we measured stopped us from spoofing packets with source IPs on the same /24 as the destination. In general, we’ve never had major problems with any kind of filtering which prevented us for being able to spoof source IP addresses. UMich has the same basic network setup we had at UNM, and we plan to have the same thing at ASU very soon. CAIDA’s results show that about a quarter of the world’s ISPs don’t even do the simple kind of BCP38 filtering…</p>
<p><a href="https://www.caida.org/publications/papers/2019/network_hygiene_incentives_regulation/">Network Hygiene, Incentives, and Regulation - Caida</a></p>
<p>Maybe to make it concrete, could you give us an example of how the spoofed packets would be filtered if, e.g., the VPN client was in Ecuador using an Ecuadorian ISP, the VPN server were in the U.S. using a U.S. ISP, and the website were in Brazil using a Brazilian ISP, and an Ecuadorian ISP was the attacker?</p>
<p> </p>
<p><a href="javascript:;" class="button button--outline-success button--pill">How is this related to VPNs? How is it different than TCP attacks on the unencrypted internet?</a></p>
<p><a href="javascript:;"><em>Asked by Jason Donenfeld and Colm MacCárthaigh</em></a></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>*This example was provided by Jason*
Client accesses web pages by sending TCP packets:
0. vpn client makes a packet: [inner]
1. vpn client encrypts and sends to AP: [outer {encrypted inner}]
2. AP does nat: [outer' {encrypted inner}]
3. AP to router 1: [outer' {encrypted inner}]
4. router 1 to router 2: [outer' {encrypted inner}]
5. router 2 to vpn server: [outer' {encrypted inner}]
6. vpn server decrypts and does nat: [inner']
7. vpn server to router 2: [inner']
8. router 2 to router 3: [inner']
9. router 3 to website: [inner']
10. website replies: [inner2]
11. website to router 3: [inner2]
12. router 3 to router 2: [inner2]
13. router 2 to vpn server: [inner2]
14. vpn server does nat and encrypts: [outer2 {encrypted inner2'}]
15. vpn server to router 2: [outer2 {encrypted inner2'}]
16. router 2 to router 1: [outer2 {encrypted inner2'}]
17. router 1 to AP: [outer2 {encrypted inner2'}]
18. AP does nat: [outer2' {encrypted inner2'}]
19. AP to vpn client: [outer2' {encrypted inner2'}]
20. vpn client does NAT and decrypts: [inner2']
</code></pre></div> </div>
<p>We perform attacks from router 1, which never sees any unencrypted traffic between the VPN server and the website/DNS server. It might be instructive to separate the vulnerability into two separate parts:</p>
<ol>
<li>
<p>It’s possible to infer the 4-tuple of a connection being NATed by the VPN server, because only packets with the correct 4-tuple get NATed and tunneled. So if we assume we know what server and port the client is connecting to (e.g., a DNS server on port 53), and we know the TCP connection will appear to come from the VPN server, then we know three of the four parts of the 4-tuple (and we assume we know the protocol, UDP in the case of DNS if people prefer to think in terms of 5-tuples). The part of the tuple we don’t know is the source port used by the VPN server for the connection (which may or may not match the source port the client chose, doesn’t matter for the attack). But if we try every possible source port only the correct one will get NATed by the VPN server and sent through the tunnel. For our attack that spoofs a DNS reply, to find the right source port we put size markers and fill the packets from /dev/urandom so the VPN server can’t compress them. Then we see what size come through the VPN tunnel (we can see the size of encrypted packets at router 1) and keep sorting into buckets and guessing until we’re sure we have the correct source port. Again, all packets we spoof without the correct source port are dropped by the VPN server and never NATed. We’re basically taking advantage of the fact that there’s a conntrack entry, and using that like a very simple side channel.</p>
</li>
<li>
<p>It might be more clear to think of this as a separate vulnerability… Once we know the source port the VPN server is NATing, then any packets we want to spoof to the client can be sent directly to the server. The server will NAT them, encrypt them, and send them to the client through the VPN tunnel. From the client’s perspective the spoofed packets will be identical to legitimate packets for the tunneled connection. For spoofing replies to a DNS query from the client, we assume that we start when the DNS query is sent by the client (we can fingerprint it by size as it traverses the VPN tunnel from client to VPN server), and we also assume that no legitimate reply is coming from the DNS server because we have triggered per-IP rate limiting to the VPN server’s IP, by basically DoSing the DNS server with spoofed packets from that IP. So if we can get the right source port in, e.g., 2 seconds, and the DNS resolver on the client waits for 5 or 10 seconds (0 or 1 DNS retransmissions with 5 second lookup timeout), then we have 3-8 seconds to spoof replies with guessed TXIDs. We spoof them to the VPN server from the IP of the DNS server, and the VPN server NATs them. In practice we can get the TXID right and inject a malicious DNS reply ~10% of the time, but we’re working on improving that. For TCP, we’ve confirmed that we can get the client to respond to in-window spoofed packets, but haven’t built that out into a full TCP hijacking/spoofing attack yet.</p>
</li>
</ol>
<p>In some ways, this can be seen as a vulnerability in conntrack and the related NAT functionality in netfilter, but really it’s only performing NAT the way it’s supposed to and the attack would probably work with the VPN server running, e.g., a BSD with something other than netfilter. Conntrack could start reasoning about things in TCP headers or DNS records—like sequence numbers or TXIDs—to make NAT decisions instead of only for its own state machine, but that could lead to other side channels. (Note: we haven’t completely studied the netfilter source code, but we assume it just NATs packets when the 5-tuple is correct because that’s what we’ve observed.)</p>
<p>So, your statement about where we inject packets and your model are correct, but our attack separates guessing the source port from guessing the TXID (or sequence number or whatever), so that we’re searching two 2^16 spaces instead of a 2^32 space (in the case of DNS and TXIDs, as an example). One way that VPNs are involved is that we use the existence of a tunneled packet to infer something about the conntrack state on the VPN server. That’s how we’re able to separate the source port from other fields we have to figure out for correct spoofing. Another way that VPNs are involved is that the most common mitigation for CVE-2019-14899 was to add a firewall rule or something similar to replace what reverse path filtering would have done, which is to stop spoofed packets from coming from non-VPN interfaces. But in this case both the VPN server and VPN client see the packets in the same format and on the same interface as legitimate packets.</p>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Funding Information</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>This project received funding from the following sources:</p>
<p><img class="image image--xl" src="/assets/darknsf.png" /></p>
<p>This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. 1518878, 1518523, and 1801613. Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation.</p>
</div>
</div>Jedidiah R. Crandall, Beau Kujath, and William J. Tolleyvpn-research@breakpointingbad.comIn November of last year we reported a vulnerability that allowed to a network adjacent attacker to make inferences about active connections inside VPN-tunneled connections and inject data to reset or hijack these connections. This vulnerability was assigned CVE-2019-14899 and affected Apple, Android, and many Linux and BSD systems. You can read about this in detail in our post from May. This post describes a new attack which takes advantage of the insight gained from the previous attack, but expands the threat to include any in-path router, such as your internet service provider. We have included a section with the most commonly asked questions and will update this post as we receive more correspondence. Background In the previous attack, a network adjacent attacker can see the traffic as it passes from the VPN client to the VPN server (and vice versa), and although it is encrypted, they can learn information about the TCP headers by making some good educated guesses. This attack first required the attacker to learn the private, virtual IP that has been assigned to the client by spoofing packets to the client across the private address space. When the attacker guesses correctly, they will see a response from the client (an encrypted RST packet), otherwise, the client will not respond. Once the attacker knows the virtual IP address, they then can test a connection to an arbitrary website by spoofing packets to the client with the destination address of the virtual interface and the source address of the website. The process is similar to the first step and the attacker will see a response from the client if there is a connection, and no response if the attacker has guessed incorrectly. After the attacker has established that there is an active connection, they can continue to probe the client and learn enough to find an in-window sequence and acknowledgment number. They can then use this to inject arbitrary data into the connection. This attack works because without some kind of source address validation, such as reverse path filtering, the kernel will process packets received on an incorrect interface as legitimate. This attack can be mitigated by enabling reverse path filtering or adding an iptables/nftables rule to drop packets that are received on an incorrect interface. Some vendors and developers, such as WireGuard, Private Internet Acess, ProtonVPN, and Mullvad have patched their software to protect clients from this attack. For this new kind of attack, however, we show how one can attack “the other end of the tunnel” from the perspective of an in-path router and assume that reverse path filtering has been enabled. In this scenario the attacker is still able to perform the same attack, as well as inject malicious DNS responses. We disclosed this vulnerability to a number of mailing lists, including OSS private, Linux Kernel Security, OpenVPN, WireGuard, Android, Apple, and Microsoft. The most important aspect of this attack is that it expands the vulnerable client systems to include any operating system and VPN software, including Windows and policy-based VPNs. Threat Model The previous attack considered the threat of using a VPN on public WiFi to add an additional layer of security, but this attack includes any machine between the VPN client and the VPN server with the ability to spoof packets. We have created a virtual lab to test this vulnerability, as shown in the figure below. In this scenario, the attacker is spoofing packets from Router 1. The packets are being sent to the VPN server with the source address of the website at the other end of the connection. In a real world scenario, this threat would be the ISP being used by the victim or any machine the packets traverse on their way to the VPN server. FAQ In this section we will provide common questions received from developers and kernel maintainers in hopes to clear up any misunderstandings. If you have any questions, feel free to email us at vpn-research@breakpointingbad.com. Q. Doesn’t enabling reverse path filtering or martian filtering prevent this attack? Asked by Anthony Liguori and Colm MacCárthaigh The reverse path filtering settings of the VPN client or server can be anything and the attack will still work. We have reverse path filtering enabled on both client and server in our experiments. Martian filtering probably is not relevant because the attacker never sends to or from non-routable Internet addresses (except when we use local IPs as Internet addresses in our virtual network). The spoofed packets have the source IP of the web server and the destination IP of the VPN server. The issue here is that even with reverse path filtering enabled on the VPN client and the gateway, we can perform this attack from any router that is in-path between the client and the VPN server. The mitigation that was previously offered was for the client to enable reverse path filtering, or some other form of source address validation, but this illustrates that this problem requires more than a client-side mitigation. The routers on the internet do not have source address validation, so this attack also makes policy-based VPNs (with the configuration suggested by Noel Kuntze in the conversation from last December) and operating systems that enforce the strong host model, including Windows, vulnerable. This isn’t the case with the vulnerability as reported in November. The reverse path filtering we’re talking about is the RFC 3704 kind, but specifically we’re talking about the hosts (VPN server and VPN client). So it has to do with which interfaces they allow packets to come in on. The packets we spoof for the attack enter the VPN server on the same Interface as legitimate packets, and the source IP address is identical to legitimate traffic. So if reverse path filtering is set to strict mode on the VPN server, the incoming spoofed packet will be checked to make sure that any replies to it would be routed out the same interface it’s coming in. Since it’s the Internet-facing interface in all cases (spoofed or legitimate, coming or going) reverse path filtering ala RFC 3704 never comes into play. Then the VPN server NATs the packets into the VPN tunnel and the VPN client receives them through the tunnel on the same virtual interface where legitimate packets come from. So strict reverse path filtering on the client doesn’t make a difference. Strict reverse path filtering can be carried out by edge networks but if you get deep enough into the Internet then asymmetrical routing makes it impossible (as far as we know, none of us have had the opportunity to play with non-edge networks on the Internet before). Referring to our setup, there’s an opportunity for router 2 to block the spoofed packets using the reasoning that router 1 would never route packets from the website, but that would be some kind of special case (like the website and VPN server are both customers of the same ISP, or router 1 and router 2 are owned by different ISPs that collaborate to validate source IP addresses in some way). Doesn’t BCP38 prevent this attack? Is this an issue of a misconfigured network? Asked by Colm MacCárthaigh BCP38 also seems like it isn’t a major factor. For our own research (which is largely focused on journalists and activists in Asia) we’re specifically interested in attacks that the network infrastructure could carry out on users, the kinds of attacks where it is sometimes assumed that VPNs will protect against them. So if a router is carrying out the attack an upstream router would need to apply the BCP38 filtering, which we would expect to see only for smaller ISPs that only get their Internet through one gateway and don’t route any traffic for others. And any router that was limited by such filtering could easily spoof the packets from another part of the Internet where spoofing is possible. Put another way, we expect that most nations and most major ISPs would have the ability to spoof packets. Just to be clear, in our new attacks the VPN client never sees spoofed packets anywhere else other than coming through the VPN tunnel on the same virtual interface as legitimate packets. And, as for the VPN server, spoofed packets come in the same interface and have IP and TCP headers identical to legitimate packets (except for whatever field we’re guessing, such as the ephemeral source port). We mostly have experience with edge networks and would be curious to find out more about how BCP38 is deployed in practice on the parts of the Internet where asymmetrical routing comes into play. We recently moved from the University of New Mexico to Arizona State University. While at UNM we had a VLAN that took us all the way to the Albuqueruque Gigapop, and we never had any trouble spoofing any return IP address we wanted to from there. As far as we know there is source address validation on a national level in some places (e.g., China does it on a national level for IPv6), but mostly networks reason about reverse routes only in local ways. Here are a couple of papers where we used source IP spoofing: Detecting Intentional Packet Drops on theInternet via TCP/IP Side Channels Original SYN: Finding Machines Hidden Behind Firewalls The second one also has an interesting result that only 23.9% of the networks we measured stopped us from spoofing packets with source IPs on the same /24 as the destination. In general, we’ve never had major problems with any kind of filtering which prevented us for being able to spoof source IP addresses. UMich has the same basic network setup we had at UNM, and we plan to have the same thing at ASU very soon. CAIDA’s results show that about a quarter of the world’s ISPs don’t even do the simple kind of BCP38 filtering… Network Hygiene, Incentives, and Regulation - Caida Maybe to make it concrete, could you give us an example of how the spoofed packets would be filtered if, e.g., the VPN client was in Ecuador using an Ecuadorian ISP, the VPN server were in the U.S. using a U.S. ISP, and the website were in Brazil using a Brazilian ISP, and an Ecuadorian ISP was the attacker? How is this related to VPNs? How is it different than TCP attacks on the unencrypted internet? Asked by Jason Donenfeld and Colm MacCárthaigh *This example was provided by Jason* Client accesses web pages by sending TCP packets: 0. vpn client makes a packet: [inner] 1. vpn client encrypts and sends to AP: [outer {encrypted inner}] 2. AP does nat: [outer' {encrypted inner}] 3. AP to router 1: [outer' {encrypted inner}] 4. router 1 to router 2: [outer' {encrypted inner}] 5. router 2 to vpn server: [outer' {encrypted inner}] 6. vpn server decrypts and does nat: [inner'] 7. vpn server to router 2: [inner'] 8. router 2 to router 3: [inner'] 9. router 3 to website: [inner'] 10. website replies: [inner2] 11. website to router 3: [inner2] 12. router 3 to router 2: [inner2] 13. router 2 to vpn server: [inner2] 14. vpn server does nat and encrypts: [outer2 {encrypted inner2'}] 15. vpn server to router 2: [outer2 {encrypted inner2'}] 16. router 2 to router 1: [outer2 {encrypted inner2'}] 17. router 1 to AP: [outer2 {encrypted inner2'}] 18. AP does nat: [outer2' {encrypted inner2'}] 19. AP to vpn client: [outer2' {encrypted inner2'}] 20. vpn client does NAT and decrypts: [inner2'] We perform attacks from router 1, which never sees any unencrypted traffic between the VPN server and the website/DNS server. It might be instructive to separate the vulnerability into two separate parts: It’s possible to infer the 4-tuple of a connection being NATed by the VPN server, because only packets with the correct 4-tuple get NATed and tunneled. So if we assume we know what server and port the client is connecting to (e.g., a DNS server on port 53), and we know the TCP connection will appear to come from the VPN server, then we know three of the four parts of the 4-tuple (and we assume we know the protocol, UDP in the case of DNS if people prefer to think in terms of 5-tuples). The part of the tuple we don’t know is the source port used by the VPN server for the connection (which may or may not match the source port the client chose, doesn’t matter for the attack). But if we try every possible source port only the correct one will get NATed by the VPN server and sent through the tunnel. For our attack that spoofs a DNS reply, to find the right source port we put size markers and fill the packets from /dev/urandom so the VPN server can’t compress them. Then we see what size come through the VPN tunnel (we can see the size of encrypted packets at router 1) and keep sorting into buckets and guessing until we’re sure we have the correct source port. Again, all packets we spoof without the correct source port are dropped by the VPN server and never NATed. We’re basically taking advantage of the fact that there’s a conntrack entry, and using that like a very simple side channel. It might be more clear to think of this as a separate vulnerability… Once we know the source port the VPN server is NATing, then any packets we want to spoof to the client can be sent directly to the server. The server will NAT them, encrypt them, and send them to the client through the VPN tunnel. From the client’s perspective the spoofed packets will be identical to legitimate packets for the tunneled connection. For spoofing replies to a DNS query from the client, we assume that we start when the DNS query is sent by the client (we can fingerprint it by size as it traverses the VPN tunnel from client to VPN server), and we also assume that no legitimate reply is coming from the DNS server because we have triggered per-IP rate limiting to the VPN server’s IP, by basically DoSing the DNS server with spoofed packets from that IP. So if we can get the right source port in, e.g., 2 seconds, and the DNS resolver on the client waits for 5 or 10 seconds (0 or 1 DNS retransmissions with 5 second lookup timeout), then we have 3-8 seconds to spoof replies with guessed TXIDs. We spoof them to the VPN server from the IP of the DNS server, and the VPN server NATs them. In practice we can get the TXID right and inject a malicious DNS reply ~10% of the time, but we’re working on improving that. For TCP, we’ve confirmed that we can get the client to respond to in-window spoofed packets, but haven’t built that out into a full TCP hijacking/spoofing attack yet. In some ways, this can be seen as a vulnerability in conntrack and the related NAT functionality in netfilter, but really it’s only performing NAT the way it’s supposed to and the attack would probably work with the VPN server running, e.g., a BSD with something other than netfilter. Conntrack could start reasoning about things in TCP headers or DNS records—like sequence numbers or TXIDs—to make NAT decisions instead of only for its own state machine, but that could lead to other side channels. (Note: we haven’t completely studied the netfilter source code, but we assume it just NATs packets when the 5-tuple is correct because that’s what we’ve observed.) So, your statement about where we inject packets and your model are correct, but our attack separates guessing the source port from guessing the TXID (or sequence number or whatever), so that we’re searching two 2^16 spaces instead of a 2^32 space (in the case of DNS and TXIDs, as an example). One way that VPNs are involved is that we use the existence of a tunneled packet to infer something about the conntrack state on the VPN server. That’s how we’re able to separate the source port from other fields we have to figure out for correct spoofing. Another way that VPNs are involved is that the most common mitigation for CVE-2019-14899 was to add a firewall rule or something similar to replace what reverse path filtering would have done, which is to stop spoofed packets from coming from non-VPN interfaces. But in this case both the VPN server and VPN client see the packets in the same format and on the same interface as legitimate packets. Funding Information This project received funding from the following sources: This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. 1518878, 1518523, and 1801613. Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation.Vintage Protocol Nonsense:Annoying the TCP Stack to Uncover Tunneled VPN Connections2020-05-25T00:00:00-06:002020-05-25T00:00:00-06:00https://www.breakpointingbad.com/2020/05/25/Vintage-Protocol-Nonsense<div style="width:100%; margin: auto;" align="justify">
<p>Virtual Private Networks (VPNs) are often advertised as a means to provide enhanced privacy for online browsing. VPN protocols, however, were not designed for this purpose—they have been retrofitted to do so. Our research reveals this retrofitting creates critical vulnerabilities which can easily be exploited by third-party attackers. Given this, we recommend users avoid using VPNs if they are doing so in an effort to increase their online browsing security. Other tools, such as <a href="https://www.torproject.org/download/">Tor Browser</a>, should be used instead. If users insist on using a VPN, we believe <a href="https://www.wireguard.com/">WireGuard</a> is the best option.</p>
<p>This post provides an in-depth explanation of our research and assessment of the VPN vulnerabilities disclosed in CVE-2019-9461 and CVE-2019-14899. A more succinct version (with less technical detail) is available <a href="https://www.opentech.fund/news/tunneling-while-exposed/">here</a>. <strong>Note:</strong> users only checking to see if their VPN vendor/operating system has addressed the issue can skip to the <a href="#user-mitigation">User Mitigation and Vendor Responses</a> section, below.</p>
<p>In a future post, we’ll discuss our assessment of the current mitigations and whether they fix the underlying vulnerability. We also plan to address the misinformation surrounding our disclosure more directly in a separate post.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Introduction</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>A modern VPN creates an encrypted tunnel from the user to the VPN server. The server then acts as a middleman, in effect, retrieving the content requested by the user without revealing the user's identity. In a standard configuration, when a user connects to a VPN the VPN software on the user's machine creates a tunneling interface (tun/tap) to direct all of the user's internet traffic. The traffic sent through this TUN interface is encrypted using security protocols, typically based on SSL/TLS, before being sent on to the VPN server. The server retrieves the requested traffic, and then reverses the process—sending the traffic back to the user in encrypted form.</p>
<p>Notably, this configuration was not designed to provide the type of security VPNs are advertised to offer today. To the contrary, VPNs (which may refer to a multitude of different services and protocols) were only designed to allow a remote user point-to-point access to a private network in order to access on-site resources. This legacy is most apparent in how TUN interfaces are addressed (in which the client and server TUN interfaces communicate like a local, private network by using one of the three blocks of IPs reserved for private networks).</p>
<p>As noted at the outset, our research is motivated by the fact that VPNs have become a fundamental part of the conversation surrounding online user privacy and anonymity, but the risks involved in using them is not fully understood. To better understand these risks, we develop attacks that exploit aspects of a typical VPN configuration. Our focus is not on otherwise vulnerable VPNs, but on properly configured commercial VPNs that are used for privacy and anonymity purposes by privacy enthusiasts, activists, and dissidents. Special attention has been paid to vulnerable communities that use VPNs to avoid persecution and communicate with otherwise inaccessible communities.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Threat Model</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>Many people in the global West associate VPNs with either avoiding cease-and-desist letters from their ISP for pirating movies, music, games, and software, or spoofing their location to access websites or features in apps that aren't available in their physical location. In many parts of the world, however, VPNs provide access to far more than region-restricted TV shows. At times, they provide the only real access users have to the international community and restricted information. In areas where the act of visiting banned websites can be severely punished, VPNs (along with <a href="https://www.torproject.org/">Tor</a> and other privacy-enhancing tools) create gateways to the rest of the world because they promise user identities and browsing habits will be hidden from unwanted third-party observation.</p>
<p>Yet for vulnerable populations concerned about the repercussions for accessing banned information or communicating with dissident groups, using a VPN at home still evokes a strong sense of paranoia. Many of these individuals therefore choose to use public networks at coffee shops or restaurants to add an additional layer of security. And, even in western nations, people who are concerned about cyber criminals and people snooping on their browsing habits are encouraged to use a VPN while on a public network. Indeed, virtually all VPN providers listed an untrusted, public network as one of the primary use cases for VPNs. The message is : If you are on an untrusted network, you should use a VPN. Our research was based on challenging this assumption.</p>
<p><a href="/assets/threatmodel.jpg"><img src="/assets/threatmodel.jpg" alt="Threat Model" /></a></p>
<p>In the threat model we employed, the attacker must be able to actively view ("sniff") all of the encrypted traffic being sent between the VPN client and the VPN server. The attacker also needs to be on the same local network as the victim, either as the access point or another local user connected to the same access point as the victim (attackers are the ninjas in the figure above). If the attacker is not the access point, then they need to have a wireless card capable of sniffing other client traffic on the same network, or they must perform a cache-poisoning attack to become the first hop for the victim's traffic.</p>
<p>In a real world scenario, this threat model would typically involve the attacker acting as a third-party public access point such as a cafe, airport, or mobile network controlled by a telecommunications company (with potentially some nation-state involvement). Even with a properly configured VPN on the victim's device, the attack we describe in the following section allows an attacker to infer existing connections and potentially reset or inject malicious payloads into those connections.</p>
<p>It is important to note that the injection of payloads is possible on all unsecured TCP connections, but does not work on protocols that implement application-level security like HTTPS (which has fortunately become significantly more common across the web in the past decade). Nonetheless, in the underdeveloped parts of the Internet that at-risk users typically operate in, there is a significant amount of traffic that is not protected by HTTPS. We used Selenium to scrape all the websites from <a href="https://github.com/citizenlab/test-lists">Citizen Lab’s list of potential blocked websites in China and Brazil</a>, and found that these two countries have 26.03% and 50.92%, respectively, as the ratio of websites that included some unencrypted element. Unencrypted ads are commonplace in China. Consider <a href="https://citizenlab.ca/2015/04/chinas-great-cannon/">China’s attack on GreatFire’s GitHub account</a>, which was a DDoS carried out by recruiting millions of users around the world by injecting JavaScript into their browser. The attack specifically targeted connections to one of Baidu’s ad servers that served plaintext JavaScript for advertisements. Furthermore, even if the victim is careful to only navigate to secure HTTPS sites, their communication with HTTPS addresses can still be inferred and reset by the attacker in this threat model.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Vulnerability Overview</a></p>
<p>The attack we developed to expose this new class of VPN vulnerability involves three phases. Although the explanation of how and why it works is nuanced, the attack is actually fairly simple in implementation and can be completed in under 30 seconds.</p>
<div style="width:100%; margin: auto; padding-left:25px;" align="justify">
<p><a href="javascript:;" class="button button--outline-success button--pill">PHASE ONE:</a></p>
<p>The attacker, after seeing that the IP address you are communicating with is associated with a commercial VPN provider, determines the IP address assigned to your VPN’s TUN interface. This takes roughly 3 seconds.</p>
<p>The VPN tunnel interface is assigned an address on the VPN server's subnet, much in the same way each device in your home is assigned an address on your local network. For instance, your modem may have an address of 192.168.1.1, and your phone may have the address 192.168.1.12. VPNs work in much the same way, where your device is connected to the VPN's local network. By default, OpenVPN (the most popular VPN client) uses 10.8.0.1 for the server, and the VPN interface on your device may be assigned 10.8.0.12 (for example).</p>
<p>Even though this address is private, the default behavior on many operating systems allows attackers to infer it with ease by spoofing traffic to a victim's device. In the default OpenVPN configuration, and all the configurations we tested in the wild, this can be determined in under 3 seconds.</p>
<p><a href="javascript:;" class="button button--outline-success button--pill">PHASE TWO:</a></p>
<p>The attacker determines if you are visiting a website of their choosing (such as 64.106.46.56) by spoofing internet traffic coming from the address sent to your VPN interface address. This takes roughly 5 seconds.</p>
<p>This phase of the attack requires a little more information than the prior one because TCP/IP connections have both an IP and a PORT. Each active connection you make to a website or service uses a PORT, typically assigned at random, and this "four-tuple" (your IP and PORT, as well as the server's IP and PORT) is used by your operating system to determine where to send your internet traffic. Most operating systems have around 30,000 ports available for TCP/IP connections, while most web servers use 80 (http) and 443 (https). The attacker will need to scan all of them on the client's machine, but this phase can still be completed in around 5 seconds.</p>
<p>For each website an attacker wants to check (for example, if they are going through a blocklist), they can determine if you are visiting it in under 10 seconds. Notably, an attacker will not need to repeat phase one for each subsequent site they check. This attack works regardless of whether TLS/SSL is being used. For many users, this alone is a significant enough threat. If a victim is visiting a website that doesn't use SSL, however, an attacker can also hijack the connection.</p>
<p><a href="javascript:;" class="button button--outline-success button--pill">PHASE THREE:</a></p>
<p>The attacker exploits the behavior of the SEQUENCE and ACKNOWLEDGEMENT numbers in the TCP protocol to obtain a number in the correct range, allowing them to spoof packets with a malicious payload and inject data into the connection. This takes roughly 20 seconds.</p>
<p>Sequence and acknowledgement numbers are how a user's device and web server keep track of the information that has been exchanged. The sequence number indicates the current segment in the connection. The acknowledgement number indicates the next expected segment in the connection.</p>
<p>As with any traffic, delays and detours will occur. Sometimes traffic is delivered out of order or much later than is expected. When this happens, the client or server will send a different response. If the numbers are not close to the current sequence and acknowledgement numbers at all, the device receiving them will not respond. If the numbers are incorrect—but still within an acceptable range—the device will respond with a CHALLENGE ACK.</p>
</div>
<p>Again, in practice these three phases are easily accomplished by an informed third-party in less than 30 seconds. Readers interested in obtaining a more technical explanation of the attack should review the following section.</p>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Vulnerability In-Depth</a></p>
<p>This section provides an in-depth explanation of the three-phase attack used to exploit the VPN vulnerability. Each phase is addressed below.</p>
<div style="width:100%; margin: auto; padding-left:25px;" align="justify">
<p><a href="javascript:;" class="button button--outline-success button--pill">1. Find the victim's internal VPN client IP:</a></p>
<p>When a VPN is configured properly on the client device, each connection that is established will set the source as the private IP address assigned by the VPN server (instead of the address it is usually assigned by the local access point). In order for the attacker to infer existing connections on the victim device, they must first find the private client address in use by the target device.</p>
<p>An attacker can use the source address of the local network gateway to spoof SYN-ACK packets to each private address the client could be using. The destination and source port of the probing packets do not matter in this phase. The probes also need to include the external MAC address of the victim or else the probes will not be route-able by the attacker. This requirement limits the attack to the local network. The response is easy to identify for the attacker in this phase since it will not be tunneled and will include the source IP address used by the tunnel interface (instead of the normal public-facing address it should be using). On a typical Linux device, a routing table entry ensures that any packet destined for the IP of the local gateway will use the default external interface instead of first reaching the tun/tap virtual device as intended for a VPN client.</p>
<p>For this initial phase, the attacker only needs to send one packet to each private address to see if the victim is using it. Accordingly, 254 packets need to be sent for each /24 subnet the attacker is probing. Using our test scripts, we were able to consistently scan a /16 subnet in under 8 seconds. This threat model also allows the attacker to observe the single IP address the victim is always talking to (the VPN server). The attacker could therefore easily connect to the same VPN server to determine the range of private addresses being served to the clients.</p>
</div>
<p><em>Example nping command that would trigger the appropriate challenge ACK response if the victim VPN client was indeed using the internal IP of 10.8.2.16:</em></p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nping <span class="nt">--tcp</span> <span class="nt">--flags</span> SA <span class="nt">--dest-ip</span> 10.8.2.16 <span class="nt">-e</span> ap0 <span class="nt">--dest-mac</span> Ma:Ca:Dd:rE:Ss:Xx
</code></pre></div></div>
<p> </p>
<div style="width:100%; margin: auto; padding-left:25px;" align="justify">
<p><a href="javascript:;" class="button button--outline-success button--pill">2. Determine if a connection exists:</a></p>
<p>Once the attacker knows the internal IP address, they can probe the victim for existing TCP connections. The goal of phase 2 is to determine if the victim is communicating with a given website (64.106.46.56, in our example) and if so, determine the exact port they are using for this connection. As stated in our threat model, we are considering a targeted attack, where a government in control of the access point has a specific list of polarizing websites they want to check.</p>
<p>For phase 2, the attacker repeatedly spoofs SYN-ACKs to the victim where the destination is the internal VPN client IP found in phase 1. The source address is the IP of whatever site the attacker wants to see if the victim is connected to, while the source port is usually held constant as either port 80 or 443 depending on HTTP vs HTTPS. The attacker cycles through the entire ephemeral port range of the client, using each as the destination port of the SYN probes.</p>
</div>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nping <span class="nt">--tcp</span> <span class="nt">--flags</span> SA <span class="nt">--source-ip</span> 64.106.46.56 <span class="nt">-g</span> 80 <span class="nt">--dest-ip</span> 10.8.2.16 <span class="nt">-p</span> <span class="o">[</span>32768-60999] <span class="nt">-e</span> ap0 <span class="nt">--dest-mac</span> Ma:Ca:Dd:rE:Ss:Xx
</code></pre></div></div>
<div style="width:100%; margin: auto; padding-left:25px;" align="justify">
<p>As the attacker scans the victim's port range for a matching four-tuple connection, the victim will respond in one of two ways.</p>
<ul>
<li>If the four tuple in the SYN-ACK does not exist, the victim will send a RST out the tun interface.</li>
<li>If there is an existing connection for the four tuple, the victim will send a challenge ACK out the tun interface.</li>
</ul>
<p>TCP RST packets do not contain the time stamp field, making the length of the packet 12 bytes fewer than a challenge ACK. The attacker can use this simple size difference to reliably determine the tunnel response that includes the challenge ACK.</p>
<p>For the initial port scan, the attacker only needs to send a single ACK to each possible port the victim could use. On Linux, the normal ephemeral port range is <em>32768</em> to <em>60999</em>. There will be a delay between the time the victim receives the spoofed packet and the time the attacker sniffs the appropriate response. In our script, the attacker can reliably find a port within ~300 of the one in the first round. The script then probes a second round at a slower rate within that range to find an estimate of the exact port in use. Finally, the attacker can verify if they found the exact port as many times as needed by spoofing the same packet and expecting to sniff the same amount of challenge-ACK responses. The script we used for testing took no more than 6 seconds to reliably determine if a victim was connected to a given website.</p>
</div>
<p><em>Example nping command that triggers an encrypted challenge-ACK from the victim, meaning the connection on port 40404 does exist:</em></p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nping <span class="nt">--tcp</span> <span class="nt">--flags</span> SA <span class="nt">--source-ip</span> 64.106.46.56 <span class="nt">-g</span> 80 <span class="nt">--dest-ip</span> 10.8.2.16 <span class="nt">-p</span> 40404 <span class="nt">-e</span> ap0 <span class="nt">--dest-mac</span> Ma:Ca:Dd:rE:Ss:Xx
</code></pre></div></div>
<p><em>Example nping command that triggers an encrypted RST from the victim, meaning the connection on a given port (40403, in this example) does NOT exist:</em></p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nping <span class="nt">--tcp</span> <span class="nt">--flags</span> SA <span class="nt">--source-ip</span> 64.106.46.56 <span class="nt">-g</span> 80 <span class="nt">--dest-ip</span> 10.8.2.16 <span class="nt">-p</span> 40403 <span class="nt">-e</span> ap0 <span class="nt">--dest-mac</span> Ma:Ca:Dd:rE:Ss:Xx
</code></pre></div></div>
<p> </p>
<div style="width:100%; margin: auto; padding-left:25px;" align="justify">
<p><a href="javascript:;" class="button button--outline-success button--pill">3. Injecting into the connection:</a></p>
<p>The third phase involves sending (and sniffing) the most packets. It is the most complicated of the three phases and ends up taking about 80% of the total attack time. At this stage the attacker knows about a specific existing TCP connection (source IP, source port, destination IP, destination port) and attempts to infer the exact sequence number and in-window ACK needed for the client to accept the malicious payload. All of these inference methods have been used before in similar off-path exploits (i.e. <a href="https://www.usenix.org/system/files/conference/usenixsecurity16/sec16_paper_cao.pdf">Cao</a>). Some additions made through IETF and RFCs to address these attacks, such as significantly narrowing the acceptable ACK window and rate-limiting challenge-ACKs, have made it more difficult but still possible for the attacker to infer.</p>
<p>This phase involves three steps:</p>
<div style="width:100%; margin: auto; padding-left:50px;" align="justify">
<p><a href="javascript:;" class="button button--outline-success button--pill">3a. Infer an in-window sequence number:</a></p>
<p>Spoof RSTs to the existing victim connection while incrementing the sequence number in large blocks. The victim will respond with an encrypted challenge ACK (always the same size) if the spoofed sequence number was in-window.</p>
<p><a href="javascript:;" class="button button--outline-success button--pill">3b. Infer an in-window ACK for the connection:</a></p>
<p>Use the in-window sequence number found in step 1 to continually spoof empty PSH-ACKs to the victim while decrementing the ACK number in large blocks. The victim will respond with an encrypted challenge ACK once the ACK number guessed by the attacker goes just below the one in use. In practice, when testing on a typical Linux device (ubuntu 19.04, kernel 5.0) the ACK number had to be less than the one in use by at most 20k to be accepted.</p>
<p><a href="javascript:;" class="button button--outline-success button--pill">3c. Infer the exact sequence number needed to inject:</a></p>
<p>Finally, use both the in-window sequence and in-window ACK found in the previous steps to find the exact sequence number. The attacker can continually spoof empty PSH-ACKs with the previously found values while decrementing the sequence number by one every send. As soon as the sequence number goes below the one in use, the victim will start responding with challenge ACKs. Our script took extra time in step 1 to make sure the in-window sequence number found was already within about 200 of the left edge of the window.</p>
</div>
<p>In phase 3, the attacker can just continually sniff for the same constant size of the encrypted packet where a single ACK is triggered. The attacker can find this by either monitoring the victim's traffic or connecting to the VPN server themselves to compare their own traffic. There is not as much noise as phase 2 since the victim is only responding to a small number of probes (instead of every single one). The attacker can also re-check that inferred values (i.e. the in-window sequence from step 1) are indeed correct by resending the same packets that may have triggered the response. If the attacker sniffs the same amount of responses as probes that are re-sent, then they know the value is correct.</p>
<p>The amount of packets needed for this phase and the time it takes for the complete inference depends mostly on the size of the TCP window and how long it takes the attacker to sniff a response from the victim. Additionally, against a Linux victim the attacker has to wait a half-second after each triggered challenge ACK due to the rate limit added to mitigate Cao's off-path attack. Our script went through three rounds during the initial in-window sequence scan to get a closer estimate of the exact one in use. Each round decreased the size of the block we skipped after each send and increased the amount of time we waited between each scan.</p>
</div>
<p> </p>
<div style="width:100%; margin: auto;" align="justify">
<p><a href="javascript:;" class="button button--success button--xl">Real World Example</a></p>
<p>To show what a real world attack using the strategies outlined above would look like on the Internet, we used our test script to create a video demonstration based on a specific scenario in which the attacker is in control of the access point that the victim is using to connect to the website of the <em>Democracy Party of China</em> (a party which has been banned in China since 1998). In this scenario, the victim is using a standard Linux machine running Ubuntu 18.04 with a properly configured Nord VPN connection. The attacker in the demo is attempting to see if the victim is connected to the <em>DPC</em> website and, if so, inject a malicious HTTP payload when the victim interacts with the page.</p>
<div>
<video width="100%" controls="controls" poster="image" preload="true">
<source src="/assets/video.ogg" type="video/ogg" />
Sorry, your browser doesn't support embedded videos.
</video>
</div>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Differences in OSes</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>There are a few small variances in how different client operating systems implement the TCP protocol and respond to the attacker's probes. On BSD based systems, unlike on current Linux systems, there is no challenge ACK rate limiting per-connection. The amount of time it takes to perform phase 2 and 3 is therefore significantly reduced because the attacker does not need to wait a half second after each triggered response. On FreeBSD, the client machine does not care what the ACK number is for the injected payload. This means the attacker can completely skip the second step of phase 3 and infer the exact sequence in use by the victim. On Android systems, the first phase is a bit more simple since it does not matter if the source address is the local gateway.</p>
<p>On the Apple devices we tested, including MacOS Mojave and iOS 13.1.2, it was more difficult to perform the very first phase of the attack. Unlike BSD, Linux, and Android, these systems do not respond in plain-text with the source of the internal VPN address. Instead, the attacker has to use phase 2 to infer the tun IP address. We found that almost all Apple devices establish a TCP connection in the background to the Apple Notification Service on port 5223, which only seems to use ~10 different IPs to serve clients. We also found that because the connection is re-established as soon as the client connects to the VPN server, most Mojave devices will use a source port very close to the one chosen for the original TCP connection being used to talk to the VPN server. As the attacker can see the port being used to communicate with the VPN server, they have a much better idea of the port the victim would be using for the notification service.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Security Disclosures</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>On December 4, 2019, we reported this vulnerability to the public oss-security mailing list after first reporting it to the Linux Security, Android, Apple, and the private oss-security distros mailing list and allowing for the maximum embargo period to expire for each. We made no other efforts to publicly disclose the vulnerability ourselves. After the vulnerability was made public, however, it was published in several places. In turn, misinformation about what the attack is and how it works was also spread. It is our hope that this blog post clears up any confusion about the attack, while also offering practical advice for affected individuals .</p>
</div>
<p> </p>
<div style="width:100%; margin: auto;" align="justify" id="user-mitigation">
<p><a href="javascript:;" class="button button--success button--xl">User Mitigation and Vendor Response</a></p>
<p><strong>User Mitigation:</strong></p>
<p>Users on Linux clients can fix the vulnerability themselves by turning reverse path filtering on with a sysctl command or an iptables command. To check and see if reverse path filtering is enabled on your machine, you can use the following command:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sysctl net.ipv4.conf.all.rp_filter
</code></pre></div> </div>
<p>If this command returns 0 or 2, your device is susceptible to the attack, but it can be easily mitigated with the following command:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sysctl net.ipv4.conf.all.rp_filter<span class="o">=</span>1
</code></pre></div> </div>
<p>To turn reverse path filtering on permanently, you should add the following lines to <em>/usr/lib/sysctl.d/50-default.conf</em>:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>net.ipv4.conf.default.rp_filter<span class="o">=</span>1
net.ipv4.conf.all.rp_filter<span class="o">=</span>1
</code></pre></div> </div>
<p>Or if you prefer, it can be mitigated by an iptables rule (or an equivalent nftables rule) such as the following:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>iptables <span class="nt">-t</span> raw <span class="se">\!</span> <span class="nt">-i</span> tun0 <span class="nt">-d</span> 10.0.0.0/8 <span class="nt">-j</span> DROP
</code></pre></div> </div>
</div>
<p> </p>
<p><strong>Vendor Response:</strong></p>
<div style="width:100%; margin: auto;" align="justify">
<p><em>OpenVPN</em> published <a href="https://openvpn.net/security-advisory/no-flaws-found-in-openvpn-software/">security advisory</a> in response to our report where they stated that the vulnerability was in the way Unix-based systems are configured and not a flaw in OpenVPN's software. Although this is true, and it would be difficult for OpenVPN to anticipate the way in which each of their users will configure their software, the vulnerability nevertheless affects anyone using their software without modification. Considering that OpenVPN is currently the most popular VPN software used for commercial VPNs, this is concerning prospect for at-risk individuals. Their decision not to address the issue with mitigations in the client side software leaves anyone using their vanilla client vulnerable to this attack.</p>
<p><em>WireGuard</em> actively participated in the development of a solution from the first day the vulnerability was privately disclosed and issued a fix on the day the vulnerability was publicly disclosed. Like OpenVPN, they also acknowledged that this is not a vulnerability in their software, but unlike OpenVPN they "are in the business of properly configuring people's networking stacks." You can follow the conversation in their mailing list archive <a href="https://lore.kernel.org/wireguard/20191205191318.GA44156@zx2c4.com/">here</a>.</p>
<p><em>Private Internet Access</em> says they have addressed the issue in a <a href="https://www.privateinternetaccess.com/blog/private-internet-access-updates-linux-desktop-client-to-prevent-against-cve-2019-14899/">post</a> on their blog shortly after the disclosure. They have not yet, however, published the details of the work they did to accomplish this.</p>
<p><em>Mullvad</em> released a <a href="https://mullvad.net/en/blog/2019/12/6/closer-look-vpn-vulnerability-cve-2019-14899/">statement</a> saying that only the first phase of the attack worked against their app and patched this the day after the public disclosure. Mullvad is one of the VPN services that we use personally. We have been more than satisfied with the quality and transparency through adoption of the open-source model. Despite this, we do think that their post might be unintentionally misleading since it talks about the app specifically, and not the service in general. Their app may be patched to mitigate this vulnerability, but if a user is using OpenVPN with Mullvad's servers, the user will need to incorporate another mitigation.</p>
<p><em>ProtonVPN</em> released a <a href="https://protonvpn.com/blog/statement-on-cve-2019-14899/">statement</a> notifying their users that they patched the vulnerability using the iptables rule above, but also acknowledged that Android would require a phone that has been rooted, and that iOS and macOS are unlikely to change their policy of multihoming to address the issue.</p>
<p><em>Linux</em> developers are considering a mitigation by binding interfaces. We will update this blog if there are any developments.</p>
<p><em>OpenBSD</em> was patched shortly after the disclosure. You can follow the conversation <a href="https://marc.info/?l=openbsd-tech&m=157580561114203&w=2">here</a>.</p>
<p>As far as we can tell, neither Google nor Apple have addressed the issue, but we will update this post when and if they do (this post was last updated on 05-25-2020).</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Media Coverage and Clarification</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>The following websites covered the basics of the attack and our disclosure.</p>
<p><a href="https://www.zdnet.com/article/new-vulnerability-lets-attackers-sniff-or-hijack-vpn-connections/">https://www.zdnet.com/article/new-vulnerability-lets-attackers-sniff-or-hijack-vpn-connections/</a></p>
<p><a href="http://www.circleid.com/posts/20200225_five_security_blind_spots_from_prolonged_implementation_of_bcp/">http://www.circleid.com/posts/20200225_five_security_blind_spots_from_prolonged_implementation_of_bcp/</a></p>
<p><a href="https://threatpost.com/linux-bug-vpns-hijacking/150891/">https://threatpost.com/linux-bug-vpns-hijacking/150891/</a></p>
<p>There are some interesting discussions on Hacker News and Slashdot, but also a few misunderstanding we hope that this post has cleared up:</p>
<p><a href="https://news.ycombinator.com/item?id=21712280">Hacker News</a></p>
<p><a href="https://linux.slashdot.org/story/19/12/05/2022205/new-linux-vulnerability-lets-attackers-hijack-vpn-connections">Slashdot</a></p>
<p>A few videos were created to explain the attack in varying levels of detail. One of the more notable examples is Tom Lawrence's video, which is really impressive considering he made it the day after the public disclosure was made. The AT&T Tech Channel also produced a quality video explaining the important aspects of the attack in a shorter video.</p>
<p><a href="https://www.youtube.com/watch?v=iPInglX55Y4">Lawrence Systems - Vulnerability Lets Attackers Sniff or Hijack VPN on *nix based Systems</a></p>
<p><a href="https://www.youtube.com/watch?v=qUEGHZL9FWY">AT&T Tech Channel - VPN Hijack Vulnerability</a></p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">TL;DR</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>We have discovered a vulnerability that allows a malicious actor connected to the same network as a victim to determine if the victim is visiting a specific website and either reset or hijack that connection. By virtue of sharing the same network, attackers can exploit the vulnerability to quickly scan a list of banned or targeted websites and determine if someone on the network is accessing them via a VPN. Even if a victim is connected using SSL/TLS, attackers can then exploit the vulnerability to deny service. And if a victim is only connected with HTTP, an attacker can go so far as to completely hijack the connection. This attack shows a fundamental flaw in the security claims made by VPN proponents who claim that using a VPN on a public network prevents malicious actors from knowing which websites you are visiting, blocking your access to specific sites, or spoofing these websites to steal your information or spy on you.</p>
<p>Even though several VPN vendors and operating systems have implemented a fix for this particular vulnerability, our research has unveiled a fundamental problem with the internet protocols used by VPNs. If the threats you are concerned about include accessing banned websites or restricted communications, you may be better served by a technology which isn't vulnerable to this class of attack such as Tor.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">About the Authors</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>This work was completed by William Tolley, as part of his OTF information control fellowship at the University of California, Berkeley, and Beau Kujath, as part of an NSF-funded project at the University of New Mexico. William and Beau are both PhD students at the University of New Mexico, where they are advised by Jedidiah Crandall. This project was overseen by Jedidiah Crandall, and Narseo Vallina-Rodriguez, who served as William's OTF advisor. Mohammad Taha Khan contributed to the development of the attack code and helped test various providers, operating systems, and VPN platforms.</p>
<p>The authors would like to thank Adam Lynn for his insight and direction and John Stith for his copyediting that helped <em>punctuate</em> the end of this project.</p>
</div>
<p> </p>
<p><a href="javascript:;" class="button button--success button--xl">Funding Information</a></p>
<div style="width:100%; margin: auto;" align="justify">
<p>This project received funding from the following sources:</p>
<p><img class="image image--xl" src="/assets/darkotf.png" /></p>
<p>OTF's <em>Information Controls Fellowship Program</em> (ICFP) supports examination into how governments in countries, regions, or areas of OTF's core focus are restricting the free flow of information, impeding access to the open internet, and implementing censorship mechanisms, thereby threatening the ability of global citizens to exercise basic human rights and democracy. The program supports fellows to work within host organizations that are established centers of expertise by offering competitively paid fellowships for three, six, nine, or twelve months in duration.</p>
<p><img class="image image--xl" src="/assets/darknsf.png" /></p>
<p>This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. 1518878, 1518523, and 1801613. Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation.</p>
</div>Jedidiah R. Crandall, Beau Kujath, and William J. Tolleyvpn-research@breakpointingbad.comVirtual Private Networks (VPNs) are often advertised as a means to provide enhanced privacy for online browsing. VPN protocols, however, were not designed for this purpose—they have been retrofitted to do so. Our research reveals this retrofitting creates critical vulnerabilities which can easily be exploited by third-party attackers. Given this, we recommend users avoid using VPNs if they are doing so in an effort to increase their online browsing security. Other tools, such as Tor Browser, should be used instead. If users insist on using a VPN, we believe WireGuard is the best option. This post provides an in-depth explanation of our research and assessment of the VPN vulnerabilities disclosed in CVE-2019-9461 and CVE-2019-14899. A more succinct version (with less technical detail) is available here. Note: users only checking to see if their VPN vendor/operating system has addressed the issue can skip to the User Mitigation and Vendor Responses section, below. In a future post, we’ll discuss our assessment of the current mitigations and whether they fix the underlying vulnerability. We also plan to address the misinformation surrounding our disclosure more directly in a separate post. Introduction A modern VPN creates an encrypted tunnel from the user to the VPN server. The server then acts as a middleman, in effect, retrieving the content requested by the user without revealing the user's identity. In a standard configuration, when a user connects to a VPN the VPN software on the user's machine creates a tunneling interface (tun/tap) to direct all of the user's internet traffic. The traffic sent through this TUN interface is encrypted using security protocols, typically based on SSL/TLS, before being sent on to the VPN server. The server retrieves the requested traffic, and then reverses the process—sending the traffic back to the user in encrypted form. Notably, this configuration was not designed to provide the type of security VPNs are advertised to offer today. To the contrary, VPNs (which may refer to a multitude of different services and protocols) were only designed to allow a remote user point-to-point access to a private network in order to access on-site resources. This legacy is most apparent in how TUN interfaces are addressed (in which the client and server TUN interfaces communicate like a local, private network by using one of the three blocks of IPs reserved for private networks). As noted at the outset, our research is motivated by the fact that VPNs have become a fundamental part of the conversation surrounding online user privacy and anonymity, but the risks involved in using them is not fully understood. To better understand these risks, we develop attacks that exploit aspects of a typical VPN configuration. Our focus is not on otherwise vulnerable VPNs, but on properly configured commercial VPNs that are used for privacy and anonymity purposes by privacy enthusiasts, activists, and dissidents. Special attention has been paid to vulnerable communities that use VPNs to avoid persecution and communicate with otherwise inaccessible communities. Threat Model Many people in the global West associate VPNs with either avoiding cease-and-desist letters from their ISP for pirating movies, music, games, and software, or spoofing their location to access websites or features in apps that aren't available in their physical location. In many parts of the world, however, VPNs provide access to far more than region-restricted TV shows. At times, they provide the only real access users have to the international community and restricted information. In areas where the act of visiting banned websites can be severely punished, VPNs (along with Tor and other privacy-enhancing tools) create gateways to the rest of the world because they promise user identities and browsing habits will be hidden from unwanted third-party observation. Yet for vulnerable populations concerned about the repercussions for accessing banned information or communicating with dissident groups, using a VPN at home still evokes a strong sense of paranoia. Many of these individuals therefore choose to use public networks at coffee shops or restaurants to add an additional layer of security. And, even in western nations, people who are concerned about cyber criminals and people snooping on their browsing habits are encouraged to use a VPN while on a public network. Indeed, virtually all VPN providers listed an untrusted, public network as one of the primary use cases for VPNs. The message is : If you are on an untrusted network, you should use a VPN. Our research was based on challenging this assumption. In the threat model we employed, the attacker must be able to actively view ("sniff") all of the encrypted traffic being sent between the VPN client and the VPN server. The attacker also needs to be on the same local network as the victim, either as the access point or another local user connected to the same access point as the victim (attackers are the ninjas in the figure above). If the attacker is not the access point, then they need to have a wireless card capable of sniffing other client traffic on the same network, or they must perform a cache-poisoning attack to become the first hop for the victim's traffic. In a real world scenario, this threat model would typically involve the attacker acting as a third-party public access point such as a cafe, airport, or mobile network controlled by a telecommunications company (with potentially some nation-state involvement). Even with a properly configured VPN on the victim's device, the attack we describe in the following section allows an attacker to infer existing connections and potentially reset or inject malicious payloads into those connections. It is important to note that the injection of payloads is possible on all unsecured TCP connections, but does not work on protocols that implement application-level security like HTTPS (which has fortunately become significantly more common across the web in the past decade). Nonetheless, in the underdeveloped parts of the Internet that at-risk users typically operate in, there is a significant amount of traffic that is not protected by HTTPS. We used Selenium to scrape all the websites from Citizen Lab’s list of potential blocked websites in China and Brazil, and found that these two countries have 26.03% and 50.92%, respectively, as the ratio of websites that included some unencrypted element. Unencrypted ads are commonplace in China. Consider China’s attack on GreatFire’s GitHub account, which was a DDoS carried out by recruiting millions of users around the world by injecting JavaScript into their browser. The attack specifically targeted connections to one of Baidu’s ad servers that served plaintext JavaScript for advertisements. Furthermore, even if the victim is careful to only navigate to secure HTTPS sites, their communication with HTTPS addresses can still be inferred and reset by the attacker in this threat model. Vulnerability Overview The attack we developed to expose this new class of VPN vulnerability involves three phases. Although the explanation of how and why it works is nuanced, the attack is actually fairly simple in implementation and can be completed in under 30 seconds. PHASE ONE: The attacker, after seeing that the IP address you are communicating with is associated with a commercial VPN provider, determines the IP address assigned to your VPN’s TUN interface. This takes roughly 3 seconds. The VPN tunnel interface is assigned an address on the VPN server's subnet, much in the same way each device in your home is assigned an address on your local network. For instance, your modem may have an address of 192.168.1.1, and your phone may have the address 192.168.1.12. VPNs work in much the same way, where your device is connected to the VPN's local network. By default, OpenVPN (the most popular VPN client) uses 10.8.0.1 for the server, and the VPN interface on your device may be assigned 10.8.0.12 (for example). Even though this address is private, the default behavior on many operating systems allows attackers to infer it with ease by spoofing traffic to a victim's device. In the default OpenVPN configuration, and all the configurations we tested in the wild, this can be determined in under 3 seconds. PHASE TWO: The attacker determines if you are visiting a website of their choosing (such as 64.106.46.56) by spoofing internet traffic coming from the address sent to your VPN interface address. This takes roughly 5 seconds. This phase of the attack requires a little more information than the prior one because TCP/IP connections have both an IP and a PORT. Each active connection you make to a website or service uses a PORT, typically assigned at random, and this "four-tuple" (your IP and PORT, as well as the server's IP and PORT) is used by your operating system to determine where to send your internet traffic. Most operating systems have around 30,000 ports available for TCP/IP connections, while most web servers use 80 (http) and 443 (https). The attacker will need to scan all of them on the client's machine, but this phase can still be completed in around 5 seconds. For each website an attacker wants to check (for example, if they are going through a blocklist), they can determine if you are visiting it in under 10 seconds. Notably, an attacker will not need to repeat phase one for each subsequent site they check. This attack works regardless of whether TLS/SSL is being used. For many users, this alone is a significant enough threat. If a victim is visiting a website that doesn't use SSL, however, an attacker can also hijack the connection. PHASE THREE: The attacker exploits the behavior of the SEQUENCE and ACKNOWLEDGEMENT numbers in the TCP protocol to obtain a number in the correct range, allowing them to spoof packets with a malicious payload and inject data into the connection. This takes roughly 20 seconds. Sequence and acknowledgement numbers are how a user's device and web server keep track of the information that has been exchanged. The sequence number indicates the current segment in the connection. The acknowledgement number indicates the next expected segment in the connection. As with any traffic, delays and detours will occur. Sometimes traffic is delivered out of order or much later than is expected. When this happens, the client or server will send a different response. If the numbers are not close to the current sequence and acknowledgement numbers at all, the device receiving them will not respond. If the numbers are incorrect—but still within an acceptable range—the device will respond with a CHALLENGE ACK. Again, in practice these three phases are easily accomplished by an informed third-party in less than 30 seconds. Readers interested in obtaining a more technical explanation of the attack should review the following section. Vulnerability In-Depth This section provides an in-depth explanation of the three-phase attack used to exploit the VPN vulnerability. Each phase is addressed below. 1. Find the victim's internal VPN client IP: When a VPN is configured properly on the client device, each connection that is established will set the source as the private IP address assigned by the VPN server (instead of the address it is usually assigned by the local access point). In order for the attacker to infer existing connections on the victim device, they must first find the private client address in use by the target device. An attacker can use the source address of the local network gateway to spoof SYN-ACK packets to each private address the client could be using. The destination and source port of the probing packets do not matter in this phase. The probes also need to include the external MAC address of the victim or else the probes will not be route-able by the attacker. This requirement limits the attack to the local network. The response is easy to identify for the attacker in this phase since it will not be tunneled and will include the source IP address used by the tunnel interface (instead of the normal public-facing address it should be using). On a typical Linux device, a routing table entry ensures that any packet destined for the IP of the local gateway will use the default external interface instead of first reaching the tun/tap virtual device as intended for a VPN client. For this initial phase, the attacker only needs to send one packet to each private address to see if the victim is using it. Accordingly, 254 packets need to be sent for each /24 subnet the attacker is probing. Using our test scripts, we were able to consistently scan a /16 subnet in under 8 seconds. This threat model also allows the attacker to observe the single IP address the victim is always talking to (the VPN server). The attacker could therefore easily connect to the same VPN server to determine the range of private addresses being served to the clients. Example nping command that would trigger the appropriate challenge ACK response if the victim VPN client was indeed using the internal IP of 10.8.2.16: nping --tcp --flags SA --dest-ip 10.8.2.16 -e ap0 --dest-mac Ma:Ca:Dd:rE:Ss:Xx 2. Determine if a connection exists: Once the attacker knows the internal IP address, they can probe the victim for existing TCP connections. The goal of phase 2 is to determine if the victim is communicating with a given website (64.106.46.56, in our example) and if so, determine the exact port they are using for this connection. As stated in our threat model, we are considering a targeted attack, where a government in control of the access point has a specific list of polarizing websites they want to check. For phase 2, the attacker repeatedly spoofs SYN-ACKs to the victim where the destination is the internal VPN client IP found in phase 1. The source address is the IP of whatever site the attacker wants to see if the victim is connected to, while the source port is usually held constant as either port 80 or 443 depending on HTTP vs HTTPS. The attacker cycles through the entire ephemeral port range of the client, using each as the destination port of the SYN probes. nping --tcp --flags SA --source-ip 64.106.46.56 -g 80 --dest-ip 10.8.2.16 -p [32768-60999] -e ap0 --dest-mac Ma:Ca:Dd:rE:Ss:Xx As the attacker scans the victim's port range for a matching four-tuple connection, the victim will respond in one of two ways. If the four tuple in the SYN-ACK does not exist, the victim will send a RST out the tun interface. If there is an existing connection for the four tuple, the victim will send a challenge ACK out the tun interface. TCP RST packets do not contain the time stamp field, making the length of the packet 12 bytes fewer than a challenge ACK. The attacker can use this simple size difference to reliably determine the tunnel response that includes the challenge ACK. For the initial port scan, the attacker only needs to send a single ACK to each possible port the victim could use. On Linux, the normal ephemeral port range is 32768 to 60999. There will be a delay between the time the victim receives the spoofed packet and the time the attacker sniffs the appropriate response. In our script, the attacker can reliably find a port within ~300 of the one in the first round. The script then probes a second round at a slower rate within that range to find an estimate of the exact port in use. Finally, the attacker can verify if they found the exact port as many times as needed by spoofing the same packet and expecting to sniff the same amount of challenge-ACK responses. The script we used for testing took no more than 6 seconds to reliably determine if a victim was connected to a given website. Example nping command that triggers an encrypted challenge-ACK from the victim, meaning the connection on port 40404 does exist: nping --tcp --flags SA --source-ip 64.106.46.56 -g 80 --dest-ip 10.8.2.16 -p 40404 -e ap0 --dest-mac Ma:Ca:Dd:rE:Ss:Xx Example nping command that triggers an encrypted RST from the victim, meaning the connection on a given port (40403, in this example) does NOT exist: nping --tcp --flags SA --source-ip 64.106.46.56 -g 80 --dest-ip 10.8.2.16 -p 40403 -e ap0 --dest-mac Ma:Ca:Dd:rE:Ss:Xx 3. Injecting into the connection: The third phase involves sending (and sniffing) the most packets. It is the most complicated of the three phases and ends up taking about 80% of the total attack time. At this stage the attacker knows about a specific existing TCP connection (source IP, source port, destination IP, destination port) and attempts to infer the exact sequence number and in-window ACK needed for the client to accept the malicious payload. All of these inference methods have been used before in similar off-path exploits (i.e. Cao). Some additions made through IETF and RFCs to address these attacks, such as significantly narrowing the acceptable ACK window and rate-limiting challenge-ACKs, have made it more difficult but still possible for the attacker to infer. This phase involves three steps: 3a. Infer an in-window sequence number: Spoof RSTs to the existing victim connection while incrementing the sequence number in large blocks. The victim will respond with an encrypted challenge ACK (always the same size) if the spoofed sequence number was in-window. 3b. Infer an in-window ACK for the connection: Use the in-window sequence number found in step 1 to continually spoof empty PSH-ACKs to the victim while decrementing the ACK number in large blocks. The victim will respond with an encrypted challenge ACK once the ACK number guessed by the attacker goes just below the one in use. In practice, when testing on a typical Linux device (ubuntu 19.04, kernel 5.0) the ACK number had to be less than the one in use by at most 20k to be accepted. 3c. Infer the exact sequence number needed to inject: Finally, use both the in-window sequence and in-window ACK found in the previous steps to find the exact sequence number. The attacker can continually spoof empty PSH-ACKs with the previously found values while decrementing the sequence number by one every send. As soon as the sequence number goes below the one in use, the victim will start responding with challenge ACKs. Our script took extra time in step 1 to make sure the in-window sequence number found was already within about 200 of the left edge of the window. In phase 3, the attacker can just continually sniff for the same constant size of the encrypted packet where a single ACK is triggered. The attacker can find this by either monitoring the victim's traffic or connecting to the VPN server themselves to compare their own traffic. There is not as much noise as phase 2 since the victim is only responding to a small number of probes (instead of every single one). The attacker can also re-check that inferred values (i.e. the in-window sequence from step 1) are indeed correct by resending the same packets that may have triggered the response. If the attacker sniffs the same amount of responses as probes that are re-sent, then they know the value is correct. The amount of packets needed for this phase and the time it takes for the complete inference depends mostly on the size of the TCP window and how long it takes the attacker to sniff a response from the victim. Additionally, against a Linux victim the attacker has to wait a half-second after each triggered challenge ACK due to the rate limit added to mitigate Cao's off-path attack. Our script went through three rounds during the initial in-window sequence scan to get a closer estimate of the exact one in use. Each round decreased the size of the block we skipped after each send and increased the amount of time we waited between each scan. Real World Example To show what a real world attack using the strategies outlined above would look like on the Internet, we used our test script to create a video demonstration based on a specific scenario in which the attacker is in control of the access point that the victim is using to connect to the website of the Democracy Party of China (a party which has been banned in China since 1998). In this scenario, the victim is using a standard Linux machine running Ubuntu 18.04 with a properly configured Nord VPN connection. The attacker in the demo is attempting to see if the victim is connected to the DPC website and, if so, inject a malicious HTTP payload when the victim interacts with the page. Sorry, your browser doesn't support embedded videos. Differences in OSes There are a few small variances in how different client operating systems implement the TCP protocol and respond to the attacker's probes. On BSD based systems, unlike on current Linux systems, there is no challenge ACK rate limiting per-connection. The amount of time it takes to perform phase 2 and 3 is therefore significantly reduced because the attacker does not need to wait a half second after each triggered response. On FreeBSD, the client machine does not care what the ACK number is for the injected payload. This means the attacker can completely skip the second step of phase 3 and infer the exact sequence in use by the victim. On Android systems, the first phase is a bit more simple since it does not matter if the source address is the local gateway. On the Apple devices we tested, including MacOS Mojave and iOS 13.1.2, it was more difficult to perform the very first phase of the attack. Unlike BSD, Linux, and Android, these systems do not respond in plain-text with the source of the internal VPN address. Instead, the attacker has to use phase 2 to infer the tun IP address. We found that almost all Apple devices establish a TCP connection in the background to the Apple Notification Service on port 5223, which only seems to use ~10 different IPs to serve clients. We also found that because the connection is re-established as soon as the client connects to the VPN server, most Mojave devices will use a source port very close to the one chosen for the original TCP connection being used to talk to the VPN server. As the attacker can see the port being used to communicate with the VPN server, they have a much better idea of the port the victim would be using for the notification service. Security Disclosures On December 4, 2019, we reported this vulnerability to the public oss-security mailing list after first reporting it to the Linux Security, Android, Apple, and the private oss-security distros mailing list and allowing for the maximum embargo period to expire for each. We made no other efforts to publicly disclose the vulnerability ourselves. After the vulnerability was made public, however, it was published in several places. In turn, misinformation about what the attack is and how it works was also spread. It is our hope that this blog post clears up any confusion about the attack, while also offering practical advice for affected individuals . User Mitigation and Vendor Response User Mitigation: Users on Linux clients can fix the vulnerability themselves by turning reverse path filtering on with a sysctl command or an iptables command. To check and see if reverse path filtering is enabled on your machine, you can use the following command: sysctl net.ipv4.conf.all.rp_filter If this command returns 0 or 2, your device is susceptible to the attack, but it can be easily mitigated with the following command: sysctl net.ipv4.conf.all.rp_filter=1 To turn reverse path filtering on permanently, you should add the following lines to /usr/lib/sysctl.d/50-default.conf: net.ipv4.conf.default.rp_filter=1 net.ipv4.conf.all.rp_filter=1 Or if you prefer, it can be mitigated by an iptables rule (or an equivalent nftables rule) such as the following: iptables -t raw \! -i tun0 -d 10.0.0.0/8 -j DROP Vendor Response: OpenVPN published security advisory in response to our report where they stated that the vulnerability was in the way Unix-based systems are configured and not a flaw in OpenVPN's software. Although this is true, and it would be difficult for OpenVPN to anticipate the way in which each of their users will configure their software, the vulnerability nevertheless affects anyone using their software without modification. Considering that OpenVPN is currently the most popular VPN software used for commercial VPNs, this is concerning prospect for at-risk individuals. Their decision not to address the issue with mitigations in the client side software leaves anyone using their vanilla client vulnerable to this attack. WireGuard actively participated in the development of a solution from the first day the vulnerability was privately disclosed and issued a fix on the day the vulnerability was publicly disclosed. Like OpenVPN, they also acknowledged that this is not a vulnerability in their software, but unlike OpenVPN they "are in the business of properly configuring people's networking stacks." You can follow the conversation in their mailing list archive here. Private Internet Access says they have addressed the issue in a post on their blog shortly after the disclosure. They have not yet, however, published the details of the work they did to accomplish this. Mullvad released a statement saying that only the first phase of the attack worked against their app and patched this the day after the public disclosure. Mullvad is one of the VPN services that we use personally. We have been more than satisfied with the quality and transparency through adoption of the open-source model. Despite this, we do think that their post might be unintentionally misleading since it talks about the app specifically, and not the service in general. Their app may be patched to mitigate this vulnerability, but if a user is using OpenVPN with Mullvad's servers, the user will need to incorporate another mitigation. ProtonVPN released a statement notifying their users that they patched the vulnerability using the iptables rule above, but also acknowledged that Android would require a phone that has been rooted, and that iOS and macOS are unlikely to change their policy of multihoming to address the issue. Linux developers are considering a mitigation by binding interfaces. We will update this blog if there are any developments. OpenBSD was patched shortly after the disclosure. You can follow the conversation here. As far as we can tell, neither Google nor Apple have addressed the issue, but we will update this post when and if they do (this post was last updated on 05-25-2020). Media Coverage and Clarification The following websites covered the basics of the attack and our disclosure. https://www.zdnet.com/article/new-vulnerability-lets-attackers-sniff-or-hijack-vpn-connections/ http://www.circleid.com/posts/20200225_five_security_blind_spots_from_prolonged_implementation_of_bcp/ https://threatpost.com/linux-bug-vpns-hijacking/150891/ There are some interesting discussions on Hacker News and Slashdot, but also a few misunderstanding we hope that this post has cleared up: Hacker News Slashdot A few videos were created to explain the attack in varying levels of detail. One of the more notable examples is Tom Lawrence's video, which is really impressive considering he made it the day after the public disclosure was made. The AT&T Tech Channel also produced a quality video explaining the important aspects of the attack in a shorter video. Lawrence Systems - Vulnerability Lets Attackers Sniff or Hijack VPN on *nix based Systems AT&T Tech Channel - VPN Hijack Vulnerability TL;DR We have discovered a vulnerability that allows a malicious actor connected to the same network as a victim to determine if the victim is visiting a specific website and either reset or hijack that connection. By virtue of sharing the same network, attackers can exploit the vulnerability to quickly scan a list of banned or targeted websites and determine if someone on the network is accessing them via a VPN. Even if a victim is connected using SSL/TLS, attackers can then exploit the vulnerability to deny service. And if a victim is only connected with HTTP, an attacker can go so far as to completely hijack the connection. This attack shows a fundamental flaw in the security claims made by VPN proponents who claim that using a VPN on a public network prevents malicious actors from knowing which websites you are visiting, blocking your access to specific sites, or spoofing these websites to steal your information or spy on you. Even though several VPN vendors and operating systems have implemented a fix for this particular vulnerability, our research has unveiled a fundamental problem with the internet protocols used by VPNs. If the threats you are concerned about include accessing banned websites or restricted communications, you may be better served by a technology which isn't vulnerable to this class of attack such as Tor. About the Authors This work was completed by William Tolley, as part of his OTF information control fellowship at the University of California, Berkeley, and Beau Kujath, as part of an NSF-funded project at the University of New Mexico. William and Beau are both PhD students at the University of New Mexico, where they are advised by Jedidiah Crandall. This project was overseen by Jedidiah Crandall, and Narseo Vallina-Rodriguez, who served as William's OTF advisor. Mohammad Taha Khan contributed to the development of the attack code and helped test various providers, operating systems, and VPN platforms. The authors would like to thank Adam Lynn for his insight and direction and John Stith for his copyediting that helped punctuate the end of this project. Funding Information This project received funding from the following sources: OTF's Information Controls Fellowship Program (ICFP) supports examination into how governments in countries, regions, or areas of OTF's core focus are restricting the free flow of information, impeding access to the open internet, and implementing censorship mechanisms, thereby threatening the ability of global citizens to exercise basic human rights and democracy. The program supports fellows to work within host organizations that are established centers of expertise by offering competitively paid fellowships for three, six, nine, or twelve months in duration. This material is based upon work supported by the U.S. National Science Foundation under Grant Nos. 1518878, 1518523, and 1801613. Any opinions, findings and conclusions or recommendations expressed in this material do not necessarily reflect the views of the National Science Foundation.