Anonymous at Tue, 14 Jan 2025 13:25:29 UTC No. 1005399

>>1005395
>bots and scrapers
Might as well post nowhere

Anonymous at Tue, 14 Jan 2025 13:28:04 UTC No. 1005400

>>1005399
I don't mind that

Anonymous at Tue, 14 Jan 2025 15:47:23 UTC No. 1005407

>>1005395
Every place it's overrun by bots and scrapers
The internet is dead, normies killed it years ago

Anonymous at Tue, 14 Jan 2025 15:55:20 UTC No. 1005409

>>1005407
if thats the case then why cant you scrape and index discord? I get info off of there that never ever shows up in search

Anonymous at Wed, 15 Jan 2025 01:43:37 UTC No. 1005438

>>1005409
it depends on if the domain (e.g discord.com, artstation.com) permits google to scrape or not via the settings they place in their robots.txt file. now, of course, someone who is not google could write a program that joins various discord servers and downloads all of the files posted there, but as of their API update quite some time ago, discord bots can only see files that users explicitly send to them via commands.

TLDR, discord has a restrictive API that is decent at blocking a good amount of scraping, other sites are either not smart enough to block scraping, or they believe it is in their best business interest to allow scraping