Uses of Interface
org.apache.nutch.protocol.RobotRules

Packages that use RobotRules

org.apache.nutch.protocol
org.apache.nutch.protocol.file	Protocol plugin which supports retrieving local file resources.
org.apache.nutch.protocol.ftp	Protocol plugin which supports retrieving documents via the ftp protocol.
org.apache.nutch.protocol.http.api	Common API used by HTTP plugins (http, httpclient)

Uses of RobotRules in org.apache.nutch.protocol

Classes in org.apache.nutch.protocol that implement RobotRules

class

EmptyRobotRules

Fields in org.apache.nutch.protocol declared as RobotRules

static RobotRules

EmptyRobotRules.RULES

Methods in org.apache.nutch.protocol that return RobotRules

RobotRules

Protocol.getRobotRules(Text url, CrawlDatum datum) Retrieve robot rules applicable for this url.

Uses of RobotRules in org.apache.nutch.protocol.file

Methods in org.apache.nutch.protocol.file that return RobotRules

RobotRules

File.getRobotRules(Text url, CrawlDatum datum)

Uses of RobotRules in org.apache.nutch.protocol.ftp

Methods in org.apache.nutch.protocol.ftp that return RobotRules

RobotRules

Ftp.getRobotRules(Text url, CrawlDatum datum)

Uses of RobotRules in org.apache.nutch.protocol.http.api

Classes in org.apache.nutch.protocol.http.api that implement RobotRules

static class

RobotRulesParser.RobotRuleSet This class holds the rules which were parsed from a robots.txt file, and can test paths against those rules.

Methods in org.apache.nutch.protocol.http.api that return RobotRules

RobotRules

HttpBase.getRobotRules(Text url, CrawlDatum datum)