ai-robots-txt/ai.robots.txtA list of AI agents and robots to block#robots.txt#web crawlers#web robots#htaccess#web scraping·github.com·Apr 13, 2025ai-robots-txt/ai.robots.txt
