p5.32-www-robotrules

v 6.20.0 Updated: 11 months ago

Parse /robots.txt file


https://metacpan.org/pod/WWW::RobotRules

To install p5.32-www-robotrules, paste this in macOS terminal after installing MacPorts

sudo port install p5.32-www-robotrules

Add to my watchlist

Installations 37
Requested Installations 0