You can't just blanket block all VPN access, that's not how the internet works... they could pick some common/well-known providers of VPN services and block their IPs/ASN/etc., but you can't just flip a switch and make all forms of VPN/proxy stop working, as there's no way to tell with certainty that someone is using one.
back
9 comments
There are plenty of VPN and proxy detection services, either as a service (API) or downloadable database, which are surprisingly comprehensive. Disclaimer: I’ve run one since 2017. Years on, our primary data source is literally holding dozens of subscriptions to every commercial provider we can find, and enumerating the exit node IP addresses they use.
There are also other methods, like using zmap/zgrab to probe for servers that respond to VPN software handshakes, which can in theory be run against the entire IP space. (this also highlights non-commercial VPNs which are not generally the target of our detection, so we use this sparingly)
It will never cover every VPN or proxy in existence, but it gets pretty close.
> Years on, our primary data source is literally holding dozens of subscriptions to every commercial provider we can find, and enumerating the exit node IP addresses they use.
Assuming your VPN identification service operates commercially, I trust that you are in full compliance with all contractual agreements and Terms of Service for the services you utilize. Many of these agreements specifically prohibit commercial use, which could encompass the harvesting of exit node IP addresses and the subsequent sale of such information.
TOS are pretty meaningless in cases like this. It amounts to getting rejected as a customer and your account canceled.
Maybe the tables could be turned and we can build a service with dozens of subscriptions to every VPN detection service and report them for ToS violations ;)
> I trust that you are in full compliance with all contractual agreements and Terms of Service
Why? It's not like there's any real moral (or, likely, legal) reason to care beyond avoiding the service's ban hammer.
There's a little secret that most of the business world knows but individuals do not know: You don't have to follow Terms of Service. In most cases, the maximum penalty the company can impose for a ToS violation is a termination of your account. And it's not illegal to make a new account. They can legally ban you from making a new account, and you can legally evade the ban.
Unless you're the one-in-a-million unlucky user who gets prosecuted under the CFAA's very generic "unauthorized access to a protected computer" clause, like Aaron Swartz. It seems the general consensus is this doesn't apply to breaking a website ToS, and Aaron was only in so much trouble because he broke into a network closet, as well as for copyright violation. But consult a lawyer if unsure. (That's another difference: A business will ask a lawyer if it wants to do something shady, while an individual will simply avoid doing it)
Tangent: if you hold access to all VPN providers, have you thought about also releasing benchmarks for them? I would be interested in knowing which ones offer the best bandwidth / peering (ping).
> which are surprisingly comprehensive
How does the buyer even know what the precision and recall rates might be?
Probably contrary to the stealth aspect.
This will also cause problems with anyone that happens to (even accidentally/unknowingly) use apps that integrate services from companies such as BrightData/Luminati/HolaVPN/etc. where they sell idle time on your device/connection to their VPN/proxy customers.
The legitimate end-user will then no longer be able to use e.g. SoundCloud.
I fail to see the problem if people that allow their internet connection used by scammers/AI crawlers are banned from every service
Interesting. I assumed all VPNs switched to IPv6 by now, making detection much harder.
IPv6 isn't magically unrouteable, it just routes much larger blocks of "end IP addresses."
You just track and block /24 or /16 as necessary.
Much of the internet still does not support IPv6, so most providers will give you an IPv4 address. In fact only a few providers even support IPv6 at all.
Even with IPv6 it's not a huge problem. With a few samples we can know that a provider is operating in a given /64 or /48 or even /32 space, and can assign a confidence level that the range is used for VPNs.
Many websites including Soundcloud are still only accessible through IPv4, so this is moot, even if VPNs support IPv6 it's enough to block their V4 exit nodes for Soundcloud.
just out of curiosity: if i'm located in spain and i setup an ec2 or digital ocean instance in germany and use it as a socks proxy over ssh, do you will detect me?
It is even easier to block hosting providers. They typically publish official lists. Here's the full list for both of those providers:
https://ip-ranges.amazonaws.com/ip-ranges.json
https://digitalocean.com/geo/google.csv
(And even if they don't publish them, you can just look up the ranges owned by any autonomous network with the appropriate registry.)
It won’t end up in our proxy detection database, but we track hosting provider ranges separately: https://www.iplocate.io/data/hosting-providers/
That's a hosting service IP block. Some sites block them already. Netflix for instance.
who's buying your service ?
Sounds like snitching as a service
GEOIP providers often sell a database of known VPN/Proxy endpoints. They take the approach of shoot first, ask questions later. Using one of these databases bans a lot of legitimate ip addresses that have seen been the source of known VPN or proxy traffic.
Its not perfect ofc, but its not meant to be. Its usually just used as a safety blanket for geoblocked intellectual property, like netflix.
I connect to my residential ISP in the USA via VPN all the time and have never had issues with being blocked for VPN use.
Maybe they mean commercial VPN providers that run on the cloud?
[flagged]
https://ipinfo.io/what-is-my-ip
Here’s one database to check.
>I connect to my residential ISP in the USA via VPN all the time and have never had issues with being blocked for VPN use.
Bit of a non sequitur, you would have to outline your entire usage pattern to even submit that as N=1.
GEOIP providers dont sit on your home network. They do accept data from third parties, and are themselves (likely) subscribed to other IP addressing lists. Mostly they are a data aggregator, and its garbage in > garbage out.
If someone, say netflix, but other services participate, flag you as having an inconsistent location, they may forward those details on and you can get added to one of these lists. You might see ip bans at various content providers.
But the implementation is so slapshod that you can just as likely, poison a single ip in a CGNAT pool, and have it take over a month for anyone to act on it, where some other users on your same ISP might experience the issue.
These things can also be weighted by usage, larger amounts of traffic are more interesting because it can represent a pool of more users, or more IP infringement per user.
You can also get hit from poor IP reputation, hosting a webserver with a proxy or php reverse shell, or a hundred other things.
(Also, larger ISPs might deal with a GEOIP provider selling lists of VPN users that include their IP address space, legally, rather than just going through the process of getting the list updated normally. This means the GEOIP providers can get skittish around some ISPs and might just not include them in lists)
>Maybe they mean commercial VPN providers that run on the cloud?
I just tried it with a well known commercial VPN and I had no problems accessing the site and its music content.
For low-volume stuff you can always get a non-expiring 4G/5G bundle eSIM and tunnel through that. Because 4G/5G roaming always tunnels traffic through the home country, and then emerges from CGNAT so it can't be identified as foreign traffic.
But those data packages are expensive and not available with each wanted origin country. Also you need hardware on your side. But it is an option, just saying.
Yes, and email is decentralized in theory...
If using a VPN for access is forbidden by the ToS, you only need to detect a VPN connection once to prove violation.
The IPv4 address space to consider is limited and it is technically absolutely feasible to exhaustively scrape and block the majority of VPN endpoints. Realistically any VPN provider will have some rather small IPv4 subnets make do, shit's expensive. More so, for the trivial case, VPN anonymization works best, when many people share one IP endpoint, naturally the spread is limited. There are VPN providers, some may even be trustworthy, which have the mission of "flying under the radar" with residential IPs and all, but they are way, waaaay more expensive. For most people that's no option.
IPv6 is a different matter, but with the very increase in tracking and access control discussed here, that may be even more of a reason, IPv6 is not going to be a thing any time soon....
Thinking about it, maybe this AI monetization FOMO and monopoly protectionism, will incidentally lead to a technological split of the web. IPv4 will become the "corpo net" and IPv6 will be the "alt net". I think there may be a chance to make IPv6 the cool internet of the people, right now!
> you only need to detect a VPN connection once to prove violation
But an IP address is not a person (legally in the US at least), and many IPv4 addresses get re-used fairly often. My home 5G internet changes IP every single day, and it's a constant struggle because other users often get my IP blocked for things I didn't do. I cannot even visit etsy.com for example. Just for fun I even checked 4chan and the IP was banned for CP, months before I ever had this particular IP (because I'm paranoid and track all that stuff).
> But an IP address is not a person (legally in the US at least)
That's a completely different matter (and still probably reasonable suspicion for a search, anyway). If an account/service ID evidently uses a service through a VPN there is no uncertainty of ToS violation. Of course someone could have hacked your account and used a VPN, it doesn't ultimately prove you did it, but nevertheless the account can be flagged/blocked correctly for VPN usage.
> many IPv4 addresses get re-used fairly often
The VPN's servers won't be using changing, "random" IPs. That's something ISPs do when assigning residential IPs. VPNs with residential IPs are not common. (I am not sure those VPNs are even really legal offerings.)
If your ISP uses NAT for its subnet space, you could argue it's technically similar to a VPN. However, same as with VPN exit scraping/discovery, those IP spaces can be determined and processed accordingly. I am also sure those ISP subnets for residential IPs are actually publicly defined and known. Eg. the Vodafon IP may get temporarily flagged for acute suspicious behavior, but won't get your account flagged for VPN violation, or even blocked permanently, since it's known to be the subnet of a mobile ISP, which uses NAT.
Additionally, I presume e.g. SoundCloud prohibits anonymizing VPNs, not everything that's technically a VPN or similar.
As long there isn't a critical risk, these kind of business decisions won't aim for certainity.
They probably assume some amount of collateral damage, a small number of VPN users still flying under the radar, the bulk of VPN users being properly targeted, and the vast majority of users not noticing anything.
It is easier to block all non-residential addresses, than block VPNs. As an added "bonus" it also kills personal VPNs running on VPS. VPNs in residential space exist but are sold as "premium" product.
yes and those users that happen to have their bw sold as residential VPN will be caught in the crossfire... many times they are not even aware of it because it's something buried in a ToS they didn't read for some random app.
Maybe its a trick and they are logging all the people on VPN's trying to see if they are blocked over the next 24 hr. Then they can take the data and start blocking it lol. Maybe not lol?
MTU detection is the easiest one. Sucks for people with ISPs that don't do 1500 bytes but those are rare.
> but those are rare.
yeah sure, if you ignore the existence of literally every mobile isp.
Hard disagree... there are still a vast many providers around the world doing < 1500, such as PPPoE DSL.
Isn‘t sub-1500 bytes the norm for residential internet access? (DOCSIS and DSL with PPPoE are the most common access protocols here in Germany)
looks at Japan, UK (OpenReach), and a lot of other places still using PPPoE (on fiber!) for complicated reasons
Some of those (including many providers on Openreach) will support mini-jumbo frames that allow an MTU of 1500 inside pppoe.
Big part of the Internet blanket ban countries, why do you think VPNs are any different?
Countries can be isolated at the physical junctions (in the case of a country as restrictive as NK).
Banning by a hosted IP amongst billions of other IPs is different.
Hell, I remember malware (Trojans / RATs) from the 2000s that allowed you to use your victims IP as your personal proxy.
Nowadays it's called "residential IP proxy".
A lot of shady shit under that term. Used by all the harmful services - scammers, AI crawlers... :)
Now that you mention it, I never used those, but I always did wonder how they do those.