Hello!
At my work it seems like our corporate firewall is trying to block anything VPN and proxy related. All such websites are blocked and the VPN client I normally use refuses to connect. I also have an OpenVPN server running from home which I normally can’t connect to from work.
Eventually though I figured out that if I change settings in my VPN client to connect using OpenVPN protocol SSH > TCP (on port 22) it doesn’t get blocked. After this connection is setup I can then connect to my OpenVPN server at home as well on port 1194 with udp. After this I can close the first VPN connection and I will remain connected to my home network. How and why does this work? I don’t really get it… Is there anyone here that understands it?
I assume my second OpenVPN connection gets tunneled through the first one but why doesn’t that connection break down when the first tunnel is brought down? Or is it just the initial connection and handshake that gets tunneled and then the tunnel itself opens in parallel instead of inside the first tunnel?
Run netstat -an before and after you close the first VPN and see what changes.
I’ll try that on monday next week beause the weekend has already started here. Thanks!
I have generated lots “netstat -an” reports now but they are huge and there are a lot of things changing in each. I did:
- Before connacting any VPNs
- After connecting my standard VPN service
- After connecting to my VPN server at home
- After disconnecting my standard VPN service, with home VPN still active
Anything specific I should look for? I’m not really sure how to read these reports or what I can learn from them, and I’m not sure I should be sharing all these IP-addresses publically on the web.
Initially what I can say is that there’s a lot of connections appearing to “10.64.23.19” after starting the first VPN which I think is the private tunnel network used. After I have opened the connection to my home VPN a few entries for “10.0.8.2” appear, which is the private tunnel network I have configured for it. After disconnecting the standard VPN all the entries for “10.64.23.19” disappear but entries for “10.0.8.2” remain, although many of them with a new source port.
I don’t see any entries at all in any of the netstat reports for the public IP or port for the VPNs I’m connecting to.
Is your client running on Windows or Linux? If Linux, I believe you may have to run netstat -antup to see everything.
I’m running on Windows, the top of the output looks like this:
netstat -an
Active Connections
Proto Local Address Foreign Address State
TCP 0.0.0.0:135 0.0.0.0:0 LISTENING
TCP 0.0.0.0:445 0.0.0.0:0 LISTENING
TCP 0.0.0.0:623 0.0.0.0:0 LISTENING
TCP 0.0.0.0:902 0.0.0.0:0 LISTENING
TCP 0.0.0.0:912 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1433 0.0.0.0:0 LISTENING
TCP 0.0.0.0:2179 0.0.0.0:0 LISTENING
TCP 0.0.0.0:3000 0.0.0.0:0 LISTENING
TCP 0.0.0.0:3389 0.0.0.0:0 LISTENING
TCP 0.0.0.0:5040 0.0.0.0:0 LISTENING
TCP 0.0.0.0:5357 0.0.0.0:0 LISTENING
TCP 0.0.0.0:5985 0.0.0.0:0 LISTENING
TCP 0.0.0.0:7680 0.0.0.0:0 LISTENING
TCP 0.0.0.0:9000 0.0.0.0:0 LISTENING
Both tunnels I’m using are OpenVPN tunnels, although the first one is set up by using Eddie-UI which is a wrapper for both OpenVPN and WireGuard. This tunnel changes all my routes to go through the tunnel while the conection to my home is setup through “OpenVPN GUI” and doesn’t re-route any traffic other than packets sent to my home private network.
It’s a bit hard for me to troubleshoot with limited information. I’m not sure why you are not seeing connections to your public IP address.
Both VPN connections go to your public IP? I wonder if your company’s firewall is getting the two connections confused and thinking they are related to each other? That shouldn’t be the case since they are using different ports but who knows?
Maybe you can talk to your IT department. Of course, they may realize that you found a way to bypass their restrictions and may adjust their firewall to block future access.
No the first VPN can pretty much go to any of AirVPNs public servers as long as I do it through some non standard port/connection type. I havent extensively tested if there are more options than SSH → TCP on port 22 that work, but I tested quite a few options that didn’t work. Once that tunnel is up I can then connect to my home network VPN that is using the default OpenVPN port (1194).
I read that port 443 is the one that is least commonly blocked but I already use 80 and 443 for hosting web servers so I can’t use those two. SSH tunneling seems to require some extra steps which wouldn’t really be less work than just using Eddie-UI to connect to a random public VPN server first. Currently my home network only has port 80, 443 & 1194 open and 80 & 443 are dropping traffic from all countries but my own.
It would be nice if I could reconfigure my own VPN server so I could connect to it directly but mostly I’m just curious to understand why and how this two-step process works at all.
Not so sure I want to talk to IT because I dont really have a work-related reason to need VPNs.
I took a look at my OpenVPN server logs and there I can clearly see that the connection today was opened from the IP address of the AirVPN server I used.
Then exactly every hour after I see exactly the same log messages but instead coming from an IP my company owns. I guess this is because of my custom server setting “reneg-sec 36000” which I don’t remember why I have set. Although that should be 10 hours but maybe there’s also a client setting that does it every hour.
I also noticed I have a server setting checked that says “Allow connected clients to retain their connections if their IP address changes.”
So I guess that something special happens initially when the connection opens which the corporate firewall blocks but then when the connection is already open and changes route it fails to block it…
There’s really nothing different in the server logs I found compared to the initial connection and the key renegotiations though, other than the source ip. There’s also no log message indicating exactly when my client from work changed IP.
“Allow connected clients to retain their connections if their IP address changes.”
Yes, this setting allows mobile devices to maintain a VPN connection as they roam. It’s could be benefiting you in this situation as the VPN connection appears to be changing from the AirVPN server IP to your company’s IP.
Port strange scenario that you’ve encountered. We may never have an explanation other than something quirky is going on with your company’s firewall.
You may not want to talk to your IT department but you should read your company’s policy, provided they have one, on permitted use of VPN software in their network. Your company may take a dim view of it and could consider it against their policy. The concern would be that your VPN could create a back door into their network. This is theoretically possible if you don’t configure the routing and firewall policies correctly on your end.
I have worked for this company for 14 years now and I started in the IT department. Now I have moved on to embedded development in the R&D department and my group is one of the few in the company that are allowed to be local administrators on our machines.
Our IT department is quite small and not all that knowledgeable when it comes to IT security and networks and has over years lost some competency there and are now outsourcing a lot of the responsibilities and support.
All of the few IT guys still working in the company are very good friends of mine and have a very high trust in me, but management of the firewall is now outsourced as far as I know. I’m pretty sure the firewall is just a standard expensive corporate firewall with all the “Best practice” settings enabled.
I did find an IT policy and IT handbook document on our “intranet” (a sharepoint site) and it doesn’t say anything about VPNs. If somebody found out I was using VPNs and really wanted me to stop I’m certain it wouldn’t be worse than a light warning or “slap on the wrist”.
My home network should be quite secure since it’s protected by a tightly secured pfSense firewall with GeoIP based blocks, large pfBlockerNG blocklists and running suricata IPS, but you still made me a bit worried that IF my home network somehow gets infected with malware my tunnel to home could possibly allow devices on my home network to access the work network so I double-checked and tested the configuration now. The server is running in “Remote access mode” and not “Peer to peer” so it’s not a site-to-site VPN. I also tried pinging devices on the work network from home and that doesn’t work. So I *think* an attacker on my home network would somehow have to be able to hack into my firewall and change the OpenVPN server settings and/or modify routes to be able to get reverse access to my work network if I open a tunnel.
Furthermore I’m currently waiting for new 2.5gbe managed switches to arrive which I will use to further segregate my home network into different VLANs to properly separate trusted/untrusted LAN devices, server LAN, Wifi guest network and Wifi IoT network at home.
I mostly use the tunnel from time to time to check the status of my servers and occasionally install some updates in the background while I’m working.
Anyway thanks for answers and ideas! I think I’ll drop any further investigation of *why* my workaround works and just be happy that it does. My best theory is that the company firewall somehow recognizes when a VPN connection opens and drops those packets but fails to do the same when the data channel has already been opened and just changes route. From what I can see the company firewall is not decrypting TLS traffic (MITM) and doing deep packet inspection on it but I know there are ways to fingerprint initial TLS handshakes and do some blocking based on that (JA3), which is something my suricata setup does at home to block known malware signatures, perhaps something similar is being done by our company firewall…
Edit: Also tested visiting a lot of test sites with expired/self-signed/otherwise bad signatures here https://badssl.com/ but none were blocked by the company firewall, only chrome was giving warnings that I could bypass. So it doesn’t seem like just a self-signed certificate is being blocked which I’m using on the VPN server.
Thanks for the background of your company.
It sounds like you’ve taken reasonable steps to secure your network, but consider these two points.
- Someone else may find out that they can do this, and they may not be so diligent in securing their network. They could inadvertently create a back door to your company’s network. They could even exfiltrate company assets, though I suspect anyone can already do that with a USB stick.
- Your company’s network could become compromised and present a threat to your network.
I’m just a random Internet stranger but if I were in the IT department, I’d want to know about this and then decide whether this is a vulnerability that needs to be closed. You could even negotiate for them to make an exception for you by telling them what you told me about securing your network. You probably don’t want them to discover what you are doing by auditing their firewall logs.
Seeing as you’ve been at your company for 14 years, I’m guessing that you care about it. You could help them improve their security by telling them what you discovered. But I will acknowledge that there’s a risk that they may not view this kindly either. I’ve read too many stories of good Samaritans that have punished to know that it’s possible.
If your company has a guest network that is less restricted, perhaps you can use it instead of the internal network. And maybe do it with a personal device instead of a company computer. If a guest network doesn’t exist, you can petition for one to be created.