Tl;dr how do streaming media services (and their partner services) detect VPN?
I ask this question out of the interest of understanding how some VPN providers evade their detection while others get blocked. I realize streaming services use third parties (Google, Fastly, Akamai, etc) to perform this function, so this question is more accurately about how these providers detect VPN users.
I have researched the reputation of many IP addresses used by various VPN providers, those that work and those that are blocked, and all show up clearly as used for VPN. This leads me to the theory that it is not (at least entirely) based on IP address reputation. I did not see anything (obvious) in HTTP header or IP header (beyond the source IP) that would be unique to connections from a VPN. DNS leak tests also look clean (though I am still researching thing one). Is there something else I am missing?
I am admittedly reaching the point of assuming someone at the service providers is getting paid off to whitelist the IP addresses of the vpn providers that work but have to assume this is not the case.
Any education on how else vpn users are detected would be greatly appropriated.
One other common method is just simply logging the IP addresses that access the services. If there’s an unusually high amount of requests coming from a small IP block good chance that’s some VPN provider’s.
It is often a game of cat and mouse, and often streaming services don’t need to be 100% accurate, just enough to stop the average user from circumventing geoblock is good enough.
Most likely based on whether the ASN for the IP address is a commercial or residential ISP. Companies like MaxMind provide the databases.
Increasingly the VPN providers are relying on residential proxies hosted on commercial ISPs. It’s a somewhat murky ecosystem but I’m guessing the residential proxy companies pay users on high-speed residential ISPs to host a small box on their network. Netflix isn’t going to ban all of Comcast just because a few Comcast customers run one.
where I worked (a streaming service) mostly looked at number of logins per IP address, or also ISP name. It was a bit of a manual process but most VPNs aren’t on “Biggest residential ISP in the country” but instead on “XYZ VPS Host” and that blocked a lot of VPNs (verified by searching on social media for complaints). We struggled where a company may provide dedicated servers, and residential broadband, although we would also look at things like their IRR routes/IP blocks to see if that described what the specific block was for as well as things like PTR records.
We had developed tools that made it very easy to search for patterns and then hit the ban hammer. I quite enjoyed the cat and mouse game.
We did explore some form of active probing to further test things like latency (if you’re 0.5ms from the CDN edge, chances are you’re not residential), open ports, etc but as that was questionable for multiple reasons we never put that live
One thing I haven’t seen mentioned is device provided locational data. One service I had issues when I moved because the ISP block I was assigned had been reallocated so there was a mismatch in the IP geolocation info vs actual from my device.
This wasn’t the actual issue but the support folks could see that and thought it was which means services can take action on it. Of course not all devices have gps/locational data but services can fingerprint the device and force locational data when available which would defeat many trying to use VPN to access a services/specific content.
You could take a look at https://ipdetective.io it tracks IP addresses that originate from datacenters, vpns, proxies, tor node and bot nets. It offers a free API as well so you can test it out. It aggregates this information from 60+ sources ranging from public lists to offical cloud provider lists
I don’t know wether it’s due to this or if it’s something else, but i’ve heard that some minecraft servers for example, are able to identify proxy/VPN connections by doing a reverse lookup of the ip address you are using to connect to the server and if there are any keywords such as Cloud/hosting etc… they will block the connection. A friend of mine tried to connect to a server with a vpn i have on an Oracle VPS and it was blocked (Reverse lookup was showing Oracle Cloud). Using my private VPN hosted on my home router, connection worked fine.
I used to do tech support for Hulu, so I can tell you for sure that what they do (or at least did several years ago) is ban IPS based on a master list of “known” IPs. When a customer would call in claiming they were wrongfully VPN blocked, we’d have them read the IP from a device displaying the error and then input it on a free public site that listed allegedly confirmed VPN IPs. If it didn’t show up there, we’d just manually whitelist it, tons of people definitely got away with using VPNs that just didn’t show up on this site. No clue where they got the initial list or whether the whitelisted IPs were added to some kind of shared universal whitelist used by multiple companies, but they had no detection method other than their infrequently updated IP list.
Traffic that goes through a VPN tipically have known MTUs.
MTUs of most kinds of circuits are known. Subtract that value with the overhead of the VPN header (different VPN kinds have different header size) and you have those MTUs.
A google search on that should bring you the exact details.
I am admittedly reaching the point of assuming someone at the service providers is getting paid off to whitelist the IP addresses of the vpn providers
Not even close. What happens here is that most VPN providers basically buys hosting services everywhere, mostly bare-metal servers plus IP transit, and they will just use their private network to connect the POP’s and tie their network together. Quite often you’ll see them using their own IP’s where they have dedicated infrastructure like a few typical global city metros (eg. LHR, AMS, FRA, etc) but to all others they will just rely on OVH, AWS, GCP, CDN77, Maxihost, etc. - hence why it’s so difficult to actually map out where exactly they’re connecting from.