> Why would you want that? The magnet links alone are rather useless. The whole site can be crawled within 2-3 days easily.
But the entire site archive is already available as a torrent indexed on TPB - why bother crawling? That's what I'm referring to - search "pirate bay archive" on TPB and you'll find it.
The magnet links are not useless, because they are able to query the rest of the metadata from a tracker and/or DHT.
I'm thinking about this from the perspective of 'minimal amount of information needed in order to reproduce TPB', not 'how much information does TPB index (which is obviously much larger).
(Remember that TPB no longer even operates its own tracker, so if TPB itself were taken offline, the magnet links would still work fine).
But the entire site archive is already available as a torrent indexed on TPB - why bother crawling? That's what I'm referring to - search "pirate bay archive" on TPB and you'll find it.
The magnet links are not useless, because they are able to query the rest of the metadata from a tracker and/or DHT.
I'm thinking about this from the perspective of 'minimal amount of information needed in order to reproduce TPB', not 'how much information does TPB index (which is obviously much larger).
(Remember that TPB no longer even operates its own tracker, so if TPB itself were taken offline, the magnet links would still work fine).