Trending

Show HN: Stop AI scrapers from hammering your self-hosted blog

Sacrificing accessibility for not getting web scraped

FBI RFP for tool to scrape Gab, Parler, 8Kun, and Telegram (5k licenses) [pdf]

Messing with Scraper Bots

AI scrapers request commented scripts

Launch HN: Webhound (YC S23) – Research agent that builds datasets from the web

Feedmaker: URL + CSS selectors = RSS feed

The AI-Scraping Free-for-All Is Coming to an End

How to Stop Google from AI-Summarising Your Website

I had to scrape 36,000 pages and it turned into a complete mess before I figured it out

How I closed $5.1K in deals last week using this AI Agent that scrapes Google Maps

What’s the most reliable way you’ve found to scrape sites that don’t have clean APIs?

Is Anyone Using AI to Automate Directory Submissions?

Built a browser extension that automatically checks 50,000+ stores for better prices

Creating an AI data scraper

The internet is about to get a little worse as Reddit moves to block the Internet Archive so AI companies can't scrape its content

Most people building AI data scrapers are making the same expensive mistake

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives (Cloudflare Blog)

yt-dlp

Save item

yt-dlp

A feature-rich command-line audio/video downloader

dgtlmoon

Save item

changedetection.io

Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock alerts, and website defacement monitoring—all for free or enjoy our SaaS plan!

p1ngul1n0

Save item

blackbird

An OSINT tool to search for accounts by username and email in social networks.

Y2Z

Save item

monolith

⬛️ CLI tool and library for saving complete web pages as a single HTML file