eduvpn: Wireguard over TCP (proxyguard 1.x)

Wireguard is a very nice and fast VPN solution, however it comes with some drawbacks: Most of all it only supports UDP traffic. On an open network this is not a problem, however some networks may deny UDP traffic or may even allow only very specific traffic (like (TCP based) http(s)).

So to increase the likelihood of successful VPN traffic it’d be nice to (optionally) switch to TCP-based traffic, maybe even to one of the default http(s) ports that are usable almost everywhere.

That’s where proxyguard (eduvpn page, source code) joins the game.

Proxyguard adds a server and a client component to tunnel UDP-based wireguard traffic over TCP. It even supports reverse proxies to enable port sharing with other services (the most desirable use case being port 443/https, as it will be accessible almost everywhere).

While this sounds promising in theory I found the documentation quite lacking (or misleading). So I’ll report about my attempts to get it working here – consider it work in progress 😉

As a first step I added the software repositories (like documented here).

Client

According to this page there’s no need to configure something on the client side if you use the latest eduvpn app (things should already be include there).

Server (eduvpn frontend)

In order to support TCP-based VPN enableProxy needs to be enabled and the proxyUrl set on the eduvpn (frontend) server. The port used for wireguard UDP and proxyguard TCP connection use the same port number (51821 in our example):

eduvpn-node1 # cat /etc/vpn-user-portal/config.php
return [
<...>
    'WireGuard' => [
        'listenPort' => 51821,
        // s. vpn-user-portal code: src/Cfg/WireGuardConfig.php
        'enableProxy' => true,
        'proxyUrl' => 'http://eduvpn-node1.mydomain.de:51821',
        'setMtu' => 1392,
        ],

    // VPN Profile List
    'ProfileList' => [
        [
            'profileId' => 'fulltunnel',
            'displayName' => 'Full tunnel',
            'hostName' => [ 'eduvpn-node1.mydomain.de' ],
            'defaultGateway' => true,
            'dnsServerList' => [ '192.168.1.1', '192.168.1.2' ],
            'aclPermissionList' => [ 'EduVPN Users' ],
            'dnsSearchDomainList' => ['mydomain.de'],
            'nodeUrl' => [ 'https://eduvpn-node1-int.mydomain.de:41194' ],
            'onNode' => [ 1 ],
            'preferredProto' => 'wireguard',
            'wRangeFour' => '10.101.3.0/24',
            'wRangeSix' => 'fd43:3::/64',
        ],
   ],
];

Server (eduvpn node)

So I started by installing the server component of proxyguard:

eduvpn-node1 # apt install proxyguard-server

Configuration is done via /etc/default/proxyguard-server:

eduvpn-node1 # cat /etc/default/proxyguard-server 
# HTTP Listen Port
LISTEN=127.0.0.1:51821
# UDP Destination
TO=127.0.0.1:51821

This will open a TCP port (with an HTTP-like websocket upgrade protocol) on the host and forward the traffic to the wireguard interface (on localhost:51821 UDP).

eduvpn-node1 # systemctl restart proxyguard-server

And now let’s test basic proxyguard functionality by trying to do an upgrade from command line:

eduvpn-node1 # curl -i --http1.1 --no-buffer --header 'Connection: Upgrade' --header 'Upgrade: UoTLV/1' --header "Host: eduvpn-node1.mydomain.de" http://127.0.0.1:51821/proxyguard/eduvpn-node1.mydomain.de
HTTP/1.1 101 Switching Protocols
Connection: Upgrade
Upgrade: UoTLV/1
Date: Sun, 27 Oct 2024 11:38:33 GMT

Enabling SSL via haproxy

Of course the first thing we need to enable SSL is a set of certificates. For this setup let’s assume we already got them. The path’s mentioned in this example match those of letsencrypt/certbot (the use of certbot is well documented, so I’ll skip the step of obtaining tickets here).

As the protocol of proxyguard is only http-like, we’ll use a default TCP frontend in haproxy. On the backend site we’ll do a basic HTTP check to make sure the proxyguard-server backend is really available:

eduvpn # cat /etc/haproxy/conf/proxyguard.cfg
frontend ft_tcp_localhost_51821
        mode tcp
        # the certificate file needs to contain the key and was created like:
        # cat /etc/letsencrypt/live/srv3.mydomain.de/privkey.pem /etc/letsencrypt/live/srv3.mydomain.de/fullchain.pem > /etc/haproxy/conf/srv3.pem
        # cat /etc/letsencrypt/live/eduvpn-node1.mydomain.de/privkey.pem /etc/letsencrypt/live/eduvpn-node1.mydomain.de/fullchain.pem > /etc/haproxy/conf/eduvpn-node1.pem
        bind 98.76.54.32:51821 ssl crt /etc/haproxy/certs/eduvpn-node1.mydomain.de.pem
        option tcp-check
        default_backend bk_proxyguard

backend bk_proxyguard
        option httpchk
        #http-check send meth GET uri / ver HTTP/1.1 hdr Connection Upgrade hdr Upgrade "UoTLV/1" hdr Host eduvpn-node1.mydomain.de
        #http-check expect status 101
        http-check send meth GET uri /proxyguard/eduvpn-node1.mydomain.de ver HTTP/1.1 hdr Host eduvpn-node1.mydomain.de
        http-check expect status 426

        server eduvpn_node1_ws 127.0.0.1:51821 check

Just a side note: in the above config there was a first version of the http-check including the “Upgrade UoTLV/1” test. This version however caused massive probems: Every check established an UDP connection of proxyguard-server that wasn’t closed after the check. This caused massive problems (like DNS lookup errors) when the number of UDP connections exceeded a certain number so I reverted to the basic HTTP GET test. Update 14.11.2024: This problem was resolved by merging pull request #48.

Now let’s reload haproxy:

eduvpn-node1 # systemctl reload haproxy

Make sure SSL wrapping works as expected by issuing the test command from above, however this time with the SSL URL:

eduvpn-node1 # curl -i --http1.1 --no-buffer --header 'Connection: Upgrade' --header 'Upgrade: UoTLV/1' --header "Host: eduvpn-node1.mydomain.de" https://eduvpn-node1.mydomain.de:51821/proxyguard/eduvpn-node1.mydomain.de
HTTP/1.1 101 Switching Protocols
connection: Upgrade
upgrade: UoTLV/1
date: Sun, 27 Oct 2024 11:39:43 GMT

Testing

…is still going on. While connections works fine in the beginning on my Android phone, it stops working after some idle time (something like 3 minutes). On my Ubuntu 24.04 Ubuntu machine connection breaks as soon as I try to switch to TCP mode.

I’m still investigating that … but no real idea where to look …