Hiding VPN behind Caddy web server on port 443

networks
Published

January 22, 2025

VLESS VPN protocol implemented in Xray can use HTTP for transport, allowing web servers to handle traffic proxying, encryption, and custom routing.

Caddy is a cool & lightweight web server with automatic HTTPS encryption and HTTP3. It provides reverse proxy capabilities, WebSocket1 and gRPC2 support.

1 used in real-time communication scenarios, e.g. chat applications; it provides a persistent bi-directional connection between client and server and works on top of HTTP

2 used in server-to-server communication (usually), but it can also be used for client-server communication; it works on top of HTTP too

This is not a detailed guide on setting up Caddy or Xray. This is advanced configuration and assumes you have knowledge of both. If not, do not add extra complexity to your setup. Find step-by-step guides, learn the basics, and then return here.

I assume Caddy is installed on a host machine as the main web server and Docker is used to run services like 3X-UI; also you have a domain with manageable DNS records, i.e. you can create A/AAAA and CNAME records for subdomains.

1 General idea

Caddy can route traffic to different handlers based on URL path, e.g. service.example.com/path1 and service.example.com/path2 can be handled by different services.

This allows serving your main website while directing specific paths to the Xray server. For example, you can run a website at service.example.com while the 3X-UI interface is accessible at service.example.com/secret/3x/. You can host multiple services on the same domain and port, differentiating them by path.

When you communicate with any service via HTTPS encrypted connection, any intermediaries (like your ISP) can see only the domain name (service.example.com) and the port (443), but all paths (e.g. service.example.com/secret/3x) are encrypted.

\[ \underbrace{\text{https}}_\text{protocol} \text{://} \underbrace{\text{service.example.com}}_\text{visible} \underbrace{\text{/secret/3x/panel}}_\text{hidden} \]

However, paths are encrypted, VPN servers use lots of traffic, so it’s not hard to suspect that traffic for a specific domain service.example.com is VPN traffic. To remove suspicion, you should host a real website on the same domain. This website should not be a hello world page, but a real web service with potentially high traffic, e.g. a file sharing or video streaming service which you may actually want to use.

Here /secret/* paths are used to hide services behind a real website. It’s better to choose a less obvious path in a real setup; any random path would work better.

The following diagram shows3 key components of this setup.

3 arrows indicate flow from users to services, but all connections are bidirectional

flowchart TD

    subgraph Users
        user1([User 1])
        user2([User 2])
        user3([User 3])
    end

    subgraph Intermediates
        ISP([ISP])
        CDN(["CDN (optional)"])
        Hoster([hoster])
    end

    subgraph VPS
        
        caddy443([Caddy <br> host's 0.0.0.0:443])

        subgraph Web service
            dockerweb([Docker <br> localhost:8081])
            service([web service <br> container's 0.0.0.0:80])
        end

        subgraph 3X-UI service
          dockervpn([Docker <br> localhost:8443 ])
          xxxvpn([Xray VPN <br> container's 0.0.0.0:443])
          
          dockerui([Docker <br> localhost:8080])
          xxxui([3X-UI web console <br> container's 0.0.0.0:80])
          
          dockersub([Docker <br> localhost:8180])
          xxxsub([3X-UI subscriptions <br> container's 0.0.0.0:81])
        end

    end

    user1 -->|HTTP1 :443| ISP
    user2 -->|HTTP2 :443| ISP
    user3 -->|HTTP3 :443| ISP
    ISP --> CDN --> Hoster

    Hoster -->|HTTP1 :443| caddy443
    Hoster -->|HTTP2 :443| caddy443
    Hoster -->|HTTP3 :443| caddy443

    caddy443 -->|/secret/websocket| dockervpn -->|NAT| xxxvpn
    caddy443 -->|/secret/3x*| dockerui -->|NAT| xxxui
    caddy443 -->|"/secret/sub* (optional)"| dockersub -->|NAT| xxxsub 
    
    caddy443 -->|other| dockerweb -->|NAT| service

2 Prerequisites

Choose a service you want to use. This service:

  • must have a web UI that will be accessible via the domain service.example.com when anyone visits it.

  • should have a descriptive subdomain, e.g. file-sharing.example.com to make it look like a real service. It’s no use to hide VPN traffic behind a subdomain like vpn.example.com, kill-dictate.example.com, or xxx.example.com. Here service.example.com is a placeholder, it’s better to create a better name.

  • should be recognizable and could reasonably have high traffic usage, such as video streaming (e.g. Jellyfin) or file sharing (e.g. Pingvin Share).

Maybe you already have a service like this.

3 Set up real web service

I recommend using Docker compose to run services. On your VPS, create a directory for your service, e.g. ~/example/ and a compose.yaml file in it.

~/example/compose.yaml
services:
  example:
    image: ghcr.io/stonith404/pingvin-share
    restart: unless-stopped
    ports:
        - 127.0.0.1:8081:80
1
This is the service name, yours may be filesharring, mediaserver, or just the real name of the service if it’s recognizable (e.g. plex for Plex Media Server).
2
Replace with the image name of your file sharing service.
3
Assign appropriate ports, here we bind the service to the VPS’s 8081 port.

When you bind a Docker container to 0.0.0.0 it’s accessible from outside the host machine (VPS). Instead, bind the service to 127.0.0.1 so it won’t be accessible externally - this way you don’t need firewall rules or encryption. We’ll use Caddy’s reverse proxy capabilities to handle this automatically.

4 Make website accessible via Caddy

Assuming that you’ve already set up subdomain service.example.com and configured DNS records. To make the service accessible, you need to configure Caddy to route all requests to service.example.com:

/etc/caddy/Caddyfile
https://service.example.com {
  reverse_proxy http://localhost:8081
}
1
Look carefully at the port, it should match the one you used in the compose.yaml file for the web service.

To apply new config Caddy, run sudo systemctl reload caddy, to check logs use sudo systemctl status caddy. Caddy will automatically obtain and renew SSL certificates for your domain. Sometimes it may take a few minutes. Reload Caddy and check the service is accessible via https://service.example.com.

5 Configure 3X-UI

Configure ordinary 3X-UI without reverse proxy. Assuming that similar to the previous example, you have a directory for 3X-UI, e.g. ~/3x-ui/ and a compose.yaml file in it:

~/3x-ui/compose.yaml
services:
  3x-ui:
    image: ghcr.io/mhsanaei/3x-ui
    hostname: service.example.com
    volumes:
      - ./db/:/etc/x-ui/
      - ./cert/:/root/cert/
    restart: unless-stopped
    ports:
        - 127.0.0.1:8443:443 # VPN
        - 127.0.0.1:8080:80 # web UI
        - 127.0.0.1:8181:81 # subscription service
1
Same domain as for the web service, we use it to hide VPN traffic.
2
You must configure the VPN port in the 3X-UI, here we bind it to the VPS’s 8443 port.
3
Again, configure the 3X-UI web interface port, here we bind it to the VPS’s 8080 port.
4
Subscription service is optional, ignore it if you don’t use it.

Note that Xray is bound to localhost (127.0.0.1), meaning it won’t be directly accessible from outside the host machine. Again, we will use Caddy to forward requests to Xray based on the URL path.

Temporary use network_mode: host in the compose.yaml to test Xray without Caddy, it should already work with specified ports. See more in the PS section above.

I briefly describe how to configure Xray, but you should understand what you are doing and how to apply it to your specific case:

  • In 3X-UI set listen IPs to 0.0.0.0 (for both web 80 and VPN 443 ports), binding to 0.0.0.0 inside the container makes it accessible from outside, where outside means the host machine (VPS) with Caddy.
  • Select and configure transport WebSocket protocol4.
  • Configure inbound paths (e.g. /secret/3x for UI and /websocket for transport5) in 3X-UI settings. Paths are arbitrary but they must match in both 3X-UI and Caddy configs.
  • Configure listen IPs, ports and paths for subscription service if you use it; it can be proxied through Caddy too.

4 You can use gRPC too, but it does not support path-based routing, so it will always point to service.example.com/, it can still be managed by Caddy (see example below), but it looses the point of hiding VPN traffic behind a real service

5 If the real web service uses /websocket path, choose another path for Xray, e.g. /secret/websocket

All this steps are iterative, you will need to reconfigure Xray when Caddy is set up and vice versa. Do not take this too literally.

When testing, I had better results with WebSocket transport than gRPC. However, your experience may differ. Also, Xray developers added a new VLESS transport called XHTTP in December 2024. While it works with Caddy, my tests showed issues, maybe due to limited client support. You can experiment with it too.

6 Configure Caddy to proxy specific paths to Xray

After all basic components are here, you can add Xray-specific routing based on your chosen transport method.

Transport option 1: WebSocket

To set up WebSocket transport:

  1. configure Xray’s inbound to use WebSocket transport,
  2. add WebSocket routing to your Caddy configuration.

3X-UI inbound configuration for WebSocket transport

3X-UI inbound configuration for WebSocket transport
/etc/caddy/Caddyfile
https://service.example.com {
  reverse_proxy /secret/3x* http://localhost:8080
  reverse_proxy /secret/websocket http://localhost:8443
  reverse_proxy http://localhost:8081
}
1
This line handles all requests to 3X-UI web panel.
2
This line handles all requests to VPN traffic at service.example.com/websocket.
3
All other requests are handled by the real web service.

Do not forget to reload Caddy. 3X-UI will be available at https://service.example.com/secret/3x, while Xray will operate at service.example.com/secret/websocket (you must configure this paths in 3X-UI or choose your own!). To generate appropriate user links, add external proxy to the inbound settings in 3X-UI.

As was mentioned before, the real web service may rely on /secret* paths, choose another path for Xray if needed.

Transport option 2: gRPC (no path-based routing)

As was mentioned before, gRPC does not support path-based routing, however, it can be managed by Caddy. If your web service operates on non-root path by default (e.g. service.example.com/panel), you can still set up Caddy to resolve both services, but this will make your setup messier. Alternatively, you can hide your gRPC-based VPN behind a domain with dev-like path, e.g. api.example.com, this may look more natural in explaining high traffic usage. Simple no-path routing case is covered below:

  1. Configure Xray’s inbound to use gRPC transport,
  2. add gRPC-specific routing to your Caddy configuration.
/etc/caddy/Caddyfile
https://service.example.com {
  reverse_proxy /secret/3x* http://localhost:8080
  reverse_proxy h2c://localhost:8443
}
1
This line handles all requests to VPN traffic at service.example.com, h2c is HTTP2 without encryption.

3X-UI will be available at service.example.com/secret/3x, while Xray will operate at https://service.example.com/. The web service is omitted in the config.

Both WebSocket and gRPC protocols operate over HTTP, so they can be routed through a CDN (e.g. Cloudflare). When routing Xray traffic through CDN, your server’s IP remains hidden behind their network. Keep in mind the extra hop may increase latency (ping), though it may also improve overall speeds by routing traffic through the optimized network.

7 Subscription service (optional)

Xray can advertise client configurations via a subscription service. It can be proxied through Caddy too. 3X-UI generate two types of subscription links: ‘ordinary’ and JSON.

/etc/caddy/Caddyfile

https://service.example.com {    
  reverse_proxy /secret/3x* http://localhost:8080
  reverse_proxy /secret/websocket http://localhost:8443
  
  reverse_proxy /secret/sub* http://localhost:8181
  reverse_proxy /secret/json* http://localhost:8181
}
1
Handles all requests to the ‘ordinary’ subscription service.
2
Handles all requests to the JSON subscription service.

You need to enable subscription service in 3X-UI panel settings, configure its listen IP (0.0.0.0 in this set up), port (8181 on host and 81 inside the container6), and two paths7 in order to generate correct subscription links automatically.

6 see compose.yaml for 3X-UI below

7 This is tricky as actually you need to provide full URIs: https://service.example.com/secret/sub/ and https://service.example.com/secret/json/

Subscription service returns client configurations when a secret after /sub/ (or /json/) is provided, e.g. service.example.com/secret/sub/ow32h8fq66dhxwt4. Caddy encrypts all connections and paths, however, if the secret is leaked, the client configuration can be seen. More complicated paths for subscription service can be used to mitigate this, e.g. service.example.com/secret/sub-random-string/. I.e. threat paths as another secret.

This completes the setup.

8 Post scriptum

  • Caddy distinguishes /secret/3x/, /secret/3x, /secret/3x*, and /secret/3x/* paths. If 3X-UI is not accessible via https://service.example.com/secret/3x, try adding a trailing slash and recheck paths in both Caddy and 3X-UI configurations (and in the browser!).

  • Caddy tries to use HTTP3, but it may be blocked, e.g. in Russia HTTP3 is allowed only for specific domains, international HTTP3 traffic may not work. You can disable HTTP3 in Caddy by adding the following to the Caddyfile:

    /etc/caddy/Caddyfile
    {
  servers {
    protocols h1 h2
  }
}

    This disables HTTP3 globally for all Caddy domains and services. If you want to disable HTTP3 only for a specific service, explicitly advertise HTTP v1 & v2 protocols during connection:

    /etc/caddy/Caddyfile
    https://service.example.com {
  tls {
    alpn h2 http/1.1
  }
  # ...
}

  • If you have problems setting up Xray, you can bind it to localhost explicitly by adding network_mode: host in the compose.yaml to 3X-UI service. This disables container’s network isolation and binds the container to the host’s ports. After that you can inspect ports on the host machine and check if Xray is working. Note that if you configured listen IPs to 0.0.0.0 in 3X-UI, your container will be accessible from outside the VPS without encryption, so use this only for testing8.

  • I had an issue where Xray ignored listen IPs configured via web console, so I patched its database directly. The fix was simple: I connected to my VPS via VS Code SSH extension with installed SQLite3 Editor, opened ~/3x-ui/db/x-ui.db, edited webListen and subListen fields in the settings table, then recreated the container with docker-compose up -d --force-recreate

  • 3X-UI is in active development. Sometimes updating fixes problems - run docker-compose pull and then docker-compose up -d. However, don’t update if everything works fine since new versions may introduce bugs. Also, the new XHTTP transport protocol may be updated in the near future, so you may want to revisit this setup later.

  • Caddy can modify paths. You may want to set up simple paths in 3X-UI and then manage all paths in Caddy. E.g. Xray can operate at /* paths inside the container, Caddy can handle connections to service.example.com/secret/* on the host and strip prefix /secret/secret before forwarding to Xray. See handle_path directive in Caddy documentation.

  • When Caddy routes e.g. /secret/sub* paths to Xray and an error occurs on Xray side, Caddy returns the respond from Xray and it will differ from the standard response of the real web service. This potentially exposes that you have something interesting behind the /secret/sub* paths. To mitigate this, you can set up Caddy to manage errors too. See error handling in Caddy documentation.

  • Network performance can be improved with BBR9 congestion control. BBR helps achieve better throughput on connections with packet loss. Configure in /etc/sysctl.conf:

    /etc/sysctl.conf
    net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr

    Then apply changes with:

    sudo sysctl -p

8 When network mode is set to host, Docker uses a specific network driver, which may work faster than the default (bridge) driver which performs network translation (NAT), but now your container may conflict with other ports on the host machine, e.g. if your container expects port 443 but it’s already used by Caddy. Moreover, the performance increase is negligible - expect about 1-2 ms in ping time and a few percent in throughput. You won’t notice it, but your setup will be less secure

9 Bottleneck Bandwidth and Round-trip propagation time