Hi! Our project addresses the limitations of existing paywall bypass tools by implementing a dynamic, community-driven approach. Key features include:
Real-time Adaptive Blacklist:
Constantly updated database of paywalled sites and effective bypass methods
User-driven reporting system for quick adaptation to paywall changes
Significantly faster response to new paywalls compared to static solutions
Multi-Method Bypass Arsenal:
Unlike single-method solutions (e.g., 12ft.io's cache access), we employ various techniques
Methods include: User-Agent spoofing, Referer header manipulation, JS disabling post-load, and web archive fallbacks (Wayback Machine, archive.is, Google cache)
Our blacklist determines the most effective method per site, improving success rates
Site-Specific Solutions:
Tracking individual websites allows for custom bypass methods when general approaches fail
Parsed and validated HTML output ensures content integrity
We believe this approach offers a more robust and adaptable solution to paywall bypassing. We're eager to hear the community's thoughts and potential improvements.
I'm really doubtful on this because most of these open source apps get banned from Github. Probably will make a dummy account to open issues and always have a copy local and in Gitlab. Should be able to clean the code a bit and share it soon.
The legal landscape surrounding this issue remains ambiguous. I've documented my analysis in the legal section of my website. Typically, the consequence is domain takedowns, which is why I proactively purchased 10 domains as a precautionary measure.
I’d like something like this, but instead of bypassing the paywalls and stealing the articles it’d pay a few cents for each article and charge me at the end of the month.
I really, really want to pay but I really, really don’t want to set up a subscription to read a one-off article.
You are not stealing anything, unless you find a way to delete the source article after you downloaded it. It might break copyright rules. I wouldn't recommend doing either though.
Hmm, I think the victims of organized corporate espionage might see this completely different. Its totally enough if "agents" of a certain country known for not really caring about copyright copy the plans for your new product. They dont have to delete yoour original plans to do damage.
The damage is different. In actual historical stealing use of the item is a zero sum game soof you take it you deprive use of the item to the person deemed the rightful owner (no comment or judgemebt on how that deeming happened for the purpose of this comment). This is breech of contract. It is not a zero sum game. There is no deprivation of use for the person deemed the owner. Instead there is an ambiguous effect on revenue because you followed the natural market outcome for information and distributed it freely. Maybe you decreased revenue potential because the person would have otherwise paid, maybe the person would have otherwise gone without the information so no revenue effect but deadweight loss to society of that person acting in a less informed way, maybe you increased revenue because the one freebie caused a purchase that otherwise wouldn't have happened. It is situation dependent but in any situation it is not correct to call this stealing despite the marketing efforts of copyright owners to get it mislabelled as such.
I'm sure there was a startup a few years ago trying to achieve this. I can't recall the name sadly, it was 10+ years ago. Agar or something perhaps?
I believe you topped up a balance and if the site supported it could access per-article resouces for a one off fee. For this to really work it needs to be site independant however.
Nobody's every been prepared to try microtransactions that are actually micro.
I'd like a browser addon that warns of paywalled links before I click them, perhaps adds a padlock icon or dollar sign (maybe with required subscription/cost on hover), to avoid nonsense like 'the first paragraph or two is free, then you hit the paywall'.
Even better if it can do the same for region-locked sites and adblocker-blocking sites, or if on mobile, sites that nag/force you to use apps instead of the web...
I tried to subscribe to an online newspaper in my country.
Subscription was very easy. When I wanted to end it, they made it purposefully hard for me to do. I think it's unacceptable, and should be outlawed: the process to unsubscribe should be as hard (or easy) as the one to subscribe.
Since newspapers (at least that specific one) have no problem doing this as long as it's legal, why should the public be more observant of ethics?
If Revolut is available in your country, create a new virtual card for each subscription and when you can't/don't want to pay them anymore just cancel the card in Revolut.
Its the old leftist anarchism attitude, stealing from someone bigger then you is no crime. Why? Because the system is evil anyways, so no need to play nice. Its starts with people grabbing a newspaper from the stand, and not throwing in the coins. And from there, its uphill... Some people think its totally fine to install a specific app to bypass youtube ads, instead of buying premium. 30 years ago, these people were cracking pay tv with their computers. And it was cool!
P.S.: I am only reporting, this is not my attitude.
> When Homer Simpson reads magazines at the convenience store without paying
A more specific example would be ripping off the plastic packaging to read a magazine. The ones without extra packaging are designed and for a long time expected to be browsed.
(This quip means that if a demagogue aggrandizes themself as a hero or savior, you should be suspicious of whether they really are as saintly as they claim to be.)
Edit:
I'm not saying this quip about paywallskip, which I think is a kindhearted service because it helps reduce the information gap, or digital divide, between the middle class and those stuck in poverty.
Rather, the joke/warning is about demagogues in politics or in wealthy churches, stuff like that.
Not about this post.
Paywallskip never called themselves a hero - an appreciative user did, which is authentic and sweet.
I think no one should ever call themselves a hero.
Just help people, like Paywallskip is doing.
I guess that what real heroes always say, both in movies and in real life, is "I was just doing my job". See e.g. this school bus driver who saved a child's life:
It fallbacks to archives, yes. Basically I use different User-Agent headers, different Referer headers, it tries disabling javascript once the page has loaded and the fallback is to fetch from web archives (Wayback Machine, archive.is, Google cache).
I'm also frustrated by how paywalls work, having to endlessly subscribe to yet another newspaper or newsletter, just because you want to read one article.
But this approach is... stealing. When it comes to big media like the NYT, it might not feel like it is, but at the end of the day, it's people doing the background research, writing the content, editing, etc.
This is _not_ the solution.
Our alternative in the making offers a legal, fair alternative: buy and sell content by the piece.
If you have a blog or newsletter, or if you're a journalist, I'm happy to help in setting up your publication and start earning in a transparent manner.
Disputing the space and then resolving the mess is a tried and true tactic in this industry.
Just because this site isn’t directly paying the content providers now doesn’t mean they won’t in the future. Providing it free, even with ambiguous legality, can allow them to build enough of a following to give them leverage in future negotiations.
> I don't see how this is different from disputing the banking sector by conducting a heist.
Let's leave aside this logical fallacy; we're all adults here.
Buying music on iTunes became popular because it was easier than pirating the music. You could buy individual songs for less than 99 cents (you still can do that [1])
News outlets have the option of selling content by the piece (as you suggest) instead of forcing you to go into a monthly or annual subscription you don't need because you just want to read 1 or 2 articles per month from a particular newspaper.
However, they don't want to do so. And because of that, pirating the content becomes again more convenient; like in the pre-iTunes years.
Your idea of using an intermediary service to get that content isn't the solution. I'm not interested in a third party profiling me based on the content I read online.
There's no profiling going on, we take a cut of the sale price and that is it. But I understand your concern and will think hard about how we could make this more obvious. I'm not sure if it's a general concern or it applies more to people like us.
Also, I assume you read stuff on Substack, Medium, Reddit, etc. Definitely so on a third party commonly known as HN.
Re iTunes: it's not about the will of the news outlets. It is impractical for them to offer a model which sells by the piece. And the most prominent cause is banal: payment fees are too high a percentage on what would be a typical price for a single piece of content.
Keeping a credit balance solves that, but keeping a credit balance at every outlet is just as bad as monthly subs.
If you look at the middle part of the domain between news. and .com, it might remind you that this is a community of people building stuff, learning to sell it, improving upon the feedback.
I don't really see how pitching a legal solution in a thread below the pitch of an obviously illegal, or at least unethical one is problematic.
Why? This is HN no? The entire point of this community is to provide technology solutions (mostly through companies). If any place should welcome this, it should be HN.
My comment or the site? If the former, I acknowledge that, I got a bit triggered by YACTS (yet another content theft service). If the latter, I'm genuinely curious and happy to make changes to our copy.
Do you get your money back if you go to the store and buy some new food/fruit/snack you don't like the taste of?
No, you throw it away, and probably won't buy it again. If you don't like NYT, don't buy from them.
If NYT is like an avocado for you, sometimes ripe and delicious, sometimes unripe, sometimes rotten, you get to decide how often you're gonna buy avocados, or if you'll develop your own methods of avocado testing before buying to increase your odds. In no case do you get to take the avocado skin back to the store asking for a refund.
Perhaps a simpler analogy, you see a new bag/flavour of chips in the store, "super crunchy" "delicious", you buy it, go home, tastes horrible, barely crunchy, do you get to take it back and get your money back?
Real-time Adaptive Blacklist:
Constantly updated database of paywalled sites and effective bypass methods User-driven reporting system for quick adaptation to paywall changes Significantly faster response to new paywalls compared to static solutions
Multi-Method Bypass Arsenal:
Unlike single-method solutions (e.g., 12ft.io's cache access), we employ various techniques Methods include: User-Agent spoofing, Referer header manipulation, JS disabling post-load, and web archive fallbacks (Wayback Machine, archive.is, Google cache) Our blacklist determines the most effective method per site, improving success rates
Site-Specific Solutions:
Tracking individual websites allows for custom bypass methods when general approaches fail Parsed and validated HTML output ensures content integrity
We believe this approach offers a more robust and adaptable solution to paywall bypassing. We're eager to hear the community's thoughts and potential improvements.