Trending

The AI-Scraping Free-for-All Is Coming to an End

How to Stop Google from AI-Summarising Your Website

Is Rotten Tomatoes Still Reliable? A Statistical Analysis

Launch HN: Channel3 (YC S25) – A database of every product on the internet

Perplexity is using stealth, undeclared crawlers to evade no-crawl directives

Browser extension and local backend that automatically archives YouTube videos

I was wrong about robots.txt

Browser extensions turn nearly 1M browsers into website-scraping bots

What’s the most reliable way you’ve found to scrape sites that don’t have clean APIs?

Is Anyone Using AI to Automate Directory Submissions?

Built a browser extension that automatically checks 50,000+ stores for better prices

Creating an AI data scraper

The internet is about to get a little worse as Reddit moves to block the Internet Archive so AI companies can't scrape its content

Most people building AI data scrapers are making the same expensive mistake

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives (Cloudflare Blog)

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives

Crawling a billion web pages in just over 24 hours, in 2025

yt-dlp

Save item

yt-dlp

A feature-rich command-line audio/video downloader

Y2Z

Save item

monolith

⬛️ CLI tool and library for saving complete web pages as a single HTML file

dgtlmoon

Save item

changedetection.io

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring—all for free or enjoy our SaaS plan!

p1ngul1n0

Save item

blackbird

An OSINT tool to search for accounts by username and email in social networks.