A B C D E F G H I L M N O P Q R S T U V W

A

AbstractDroid<T extends Task> - Class in org.apache.droids

Manage common tasks in standard Droids

AbstractDroid(TaskQueue<T>, TaskMaster<T>) - Constructor for class org.apache.droids.AbstractDroid

accept(String) - Method in class org.apache.droids.helper.factories.URLFiltersFactory

Run all defined filters.

accept(String, String) - Method in class org.apache.droids.helper.factories.URLFiltersFactory

Run a specific filter class.

accept() - Method in class org.apache.droids.net.RegexRule

Return if this rule is used for filtering-in or out.

AGENT - Variable in class org.apache.http.PostFile

The AGENT name

awaitTermination(long, TimeUnit) - Method in interface org.apache.droids.api.TaskMaster

awaitTermination(long, TimeUnit) - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

B

bufferSize - Variable in class org.apache.droids.handle.Save

C

clear() - Method in class org.apache.droids.impl.SimpleTaskQueue

clear() - Method in class org.apache.droids.impl.SimpleTaskQueueWithHistory

contentTypes - Static variable in class org.apache.droids.protocol.MediaType

Deprecated. Officially known contentTypes

CrawlingDroid - Class in org.apache.droids.robot.crawler

CrawlingDroid(TaskQueue<Link>, TaskMaster<Link>) - Constructor for class org.apache.droids.robot.crawler.CrawlingDroid

CrawlingWorker - Class in org.apache.droids.robot.crawler

CrawlingWorker(CrawlingDroid) - Constructor for class org.apache.droids.robot.crawler.CrawlingWorker

D

delaySpread - Variable in class org.apache.droids.delay.RandomDelayTimer

DelayTimer - Interface in org.apache.droids.api

Define the timer delay interface.

Droid<T extends Task> - Interface in org.apache.droids.api

Interface for a droid.

DroidFactory<T extends Task> - Class in org.apache.droids.helper.factories

Factory that will lookup a droid by its name and returns it.

DroidFactory() - Constructor for class org.apache.droids.helper.factories.DroidFactory

DroidsException - Exception in org.apache.droids.exception

Wrapper object to limit the number of different Exception we can throw.

DroidsException(String) - Constructor for exception org.apache.droids.exception.DroidsException

Constructs a new exception with the specified detail message.

DroidsException(Throwable) - Constructor for exception org.apache.droids.exception.DroidsException

For more information Exception

E

execute(T) - Method in interface org.apache.droids.api.Worker

execute(Link) - Method in class org.apache.droids.robot.crawler.CrawlingWorker

execute(FileTask) - Method in class org.apache.droids.robot.walker.FileWorker

existUrl(URL) - Static method in class org.apache.droids.net.UrlHelper

Does the url exist?

F

FileProtocol - Class in org.apache.droids.protocol.file

FileProtocol() - Constructor for class org.apache.droids.protocol.file.FileProtocol

FileTask - Class in org.apache.droids.robot.walker

FileTask(File, int) - Constructor for class org.apache.droids.robot.walker.FileTask

FileWorker - Class in org.apache.droids.robot.walker

FileWorker(TaskQueue<FileTask>) - Constructor for class org.apache.droids.robot.walker.FileWorker

filter(String) - Method in interface org.apache.droids.api.URLFilter

Transforms the URL: can pass the original URL through or "delete" the URL by returning null

filter(String) - Method in class org.apache.droids.net.RegexURLFilter

findRobotsUrl(URL) - Static method in class org.apache.droids.net.UrlHelper

Search the url of the robots.txt that is responsible for the given base url.

finished() - Method in interface org.apache.droids.api.Droid

Invoke when the droid has completed

finished() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

finished() - Method in class org.apache.droids.robot.walker.SimpleWalkingDroid

from - Variable in class org.apache.droids.protocol.HttpBase

G

GaussianRandomDelayTimer - Class in org.apache.droids.delay

GaussianRandomDelayTimer() - Constructor for class org.apache.droids.delay.GaussianRandomDelayTimer

GaussianRandomDelayTimer(int, int) - Constructor for class org.apache.droids.delay.GaussianRandomDelayTimer

GenericFactory<T> - Class in org.apache.droids.helper.factories

Basically all factories till now extend this generic factory.

GenericFactory() - Constructor for class org.apache.droids.helper.factories.GenericFactory

getAnchorText() - Method in interface org.apache.droids.api.Link

getAnchorText() - Method in class org.apache.droids.LinkTask

getCompletedTasks() - Method in interface org.apache.droids.api.TaskMaster

getCompletedTasks() - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

getContentType(String) - Method in interface org.apache.droids.api.Protocol

Returns the content type of the url

getContentType() - Method in exception org.apache.droids.exception.ParserNotFoundException

If not constructed via message only it will return the content typee which has caused the problem

getContentType(String) - Method in class org.apache.droids.protocol.file.FileProtocol

getContentType(String) - Method in class org.apache.droids.protocol.HttpBase

Will analyze and return the content type of the given url.

getData() - Method in interface org.apache.droids.api.Parse

Other data extracted from the page.

getData() - Method in class org.apache.droids.parse.ParseImpl

getDelayMillis() - Method in interface org.apache.droids.api.DelayTimer

Returns the value of the delay between request.

getDelayMillis() - Method in class org.apache.droids.delay.GaussianRandomDelayTimer

getDelayMillis() - Method in class org.apache.droids.delay.RandomDelayTimer

getDelayMillis() - Method in class org.apache.droids.delay.SimpleDelayTimer

Gets the delay time.

getDelaySpread() - Method in class org.apache.droids.delay.RandomDelayTimer

getDelayTimer() - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

getDepth() - Method in interface org.apache.droids.api.Task

getDepth() - Method in class org.apache.droids.LinkTask

getDepth() - Method in class org.apache.droids.robot.walker.FileTask

getDroid(String) - Method in class org.apache.droids.helper.factories.DroidFactory

Lookup a droid by its name and return it.

getElements() - Method in class org.apache.droids.parse.html.HtmlParser

getEventFactory() - Method in class org.apache.droids.helper.StAX

Get the ready to used EventFactory

getEventParser(InputStream) - Method in class org.apache.droids.helper.StAX

Get an event Parser based on the incoming stream

getFile() - Method in class org.apache.droids.robot.walker.FileTask

getFilteredOutlinks(Parse) - Method in class org.apache.droids.robot.crawler.CrawlingWorker

getFiltersFactory() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

getFinishedWorking() - Method in interface org.apache.droids.api.TaskMaster

getFinishedWorking() - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

getFrom() - Method in interface org.apache.droids.api.Link

From where the link was created

getFrom() - Method in class org.apache.droids.LinkTask

getFrom() - Method in class org.apache.droids.protocol.HttpBase

Returns the eMail address of the bot.

getHandlerFactory() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

getId() - Method in interface org.apache.droids.api.Task

The id of the task.

getId() - Method in class org.apache.droids.LinkTask

getId() - Method in class org.apache.droids.robot.walker.FileTask

getLastCompletedTask() - Method in interface org.apache.droids.api.TaskMaster

getLastCompletedTask() - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

getLastModifedDate() - Method in class org.apache.droids.LinkTask

getLastModifiedDate() - Method in interface org.apache.droids.api.Link

last modified date

getLastModifiedDate() - Method in class org.apache.droids.LinkTask

getMap() - Method in class org.apache.droids.helper.factories.GenericFactory

Get the register which contains all components.

getMaxDepth() - Method in class org.apache.droids.impl.SimpleTaskValidator

getMaxSize() - Method in class org.apache.droids.impl.SimpleTaskQueue

getMaxThreads() - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

Get number of maximum allowed threads

getMinimumDelay() - Method in class org.apache.droids.delay.RandomDelayTimer

getNewWorker() - Method in interface org.apache.droids.api.Droid

Ask the droid for a new worker

getNewWorker() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

getNewWorker() - Method in class org.apache.droids.robot.walker.SimpleWalkingDroid

getNewWorker() - Method in interface org.apache.droids.robot.walker.WalkingDroid

getOutlinks() - Method in class org.apache.droids.ParseData

Get the outlinks of the page.

getOutputDir() - Method in class org.apache.droids.handle.Save

Get the directory where we want to save the stream.

getParse(InputStream, Link) - Method in interface org.apache.droids.api.Parser

Creates the parse for some content.

getParse(InputStream, Link) - Method in class org.apache.droids.parse.html.HtmlParser

getParser(String) - Method in class org.apache.droids.helper.factories.ParserFactory

Lookup a parser by its identifier (content type) and return it.

getParser(InputStream) - Method in class org.apache.droids.helper.StAX

Get a stream Parser based on the incoming stream

getParserFactory() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

getProtocol(String) - Method in class org.apache.droids.helper.factories.ProtocolFactory

Will lookup a protocol based on the underlying uri

getProtocolFactory() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

getQueue() - Method in class org.apache.droids.AbstractDroid

getRefer() - Method in class org.apache.droids.protocol.HttpBase

Return the refer URI where the bot is send from.

getResponseBodyAsStream() - Method in class org.apache.http.PostFile

getSize() - Method in interface org.apache.droids.api.TaskQueue

How many task do we have left in the queue.

getSize() - Method in class org.apache.droids.impl.SimpleTaskQueue

getsolrBase() - Method in class org.apache.http.PostFile

getSrc() - Method in class org.apache.http.PostFile

getStartTime() - Method in interface org.apache.droids.api.TaskMaster

getStartTime() - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

getStreamWriter(OutputStream) - Method in class org.apache.droids.helper.StAX

Get a stream writer based on the incoming stream

getTask(String) - Method in interface org.apache.droids.api.TaskQueue

Return the task that is identified with the given id

getTask(String) - Method in class org.apache.droids.impl.SimpleTaskQueue

getTaskDate() - Method in interface org.apache.droids.api.Task

When was the task created

getTaskDate() - Method in class org.apache.droids.LinkTask

getTaskDate() - Method in class org.apache.droids.robot.walker.FileTask

getTaskMaster() - Method in class org.apache.droids.AbstractDroid

getTaskMaster() - Method in interface org.apache.droids.api.Droid

Get the task master

getTaskValidator() - Method in class org.apache.droids.impl.SimpleTaskQueue

getText() - Method in interface org.apache.droids.api.Parse

The textual content of the page.

getText() - Method in class org.apache.droids.parse.ParseImpl

getTimeout() - Method in class org.apache.droids.protocol.HttpBase

Get the timeout we want for the connection.

getTo() - Method in interface org.apache.droids.api.Link

To where the link is pointing to

getTo() - Method in class org.apache.droids.LinkTask

getURI() - Method in interface org.apache.droids.api.Link

getURI() - Method in class org.apache.droids.LinkTask

getUrl() - Method in exception org.apache.droids.exception.ParserNotFoundException

If not constructed via message only it will return the url which has caused the problem

getUrl() - Method in exception org.apache.droids.exception.ProtocolNotFoundException

Will return the url which has caused the problem

getUrlPrefix(URL) - Static method in class org.apache.droids.net.UrlHelper

Creating a valid protocol prefix.

getUserAgent() - Method in class org.apache.droids.protocol.HttpBase

Get the name of our UserAgent

getWriter(OutputStream) - Method in class org.apache.droids.helper.StAX

Get an event writer based on the incoming stream

H

handle(InputStream, URL, Parse) - Method in interface org.apache.droids.api.Handler

handle(InputStream, URL, Parse) - Method in class org.apache.droids.handle.Save

handle(InputStream, URL, Parse) - Method in class org.apache.droids.handle.Sysout

handle(InputStream, URL, Parse) - Method in class org.apache.droids.helper.factories.HandlerFactory

Will traverse all registered handler and execute them.

handle(Parse, InputStream, Link) - Method in class org.apache.droids.robot.crawler.CrawlingWorker

Handler - Interface in org.apache.droids.api

A handler is a component that uses the stream, the parse and url to invoke arbitrary business logic on the objects.

HandlerFactory - Class in org.apache.droids.helper.factories

Factory that will traverse all registered handler and execute them.

HandlerFactory() - Constructor for class org.apache.droids.helper.factories.HandlerFactory

hasNext() - Method in interface org.apache.droids.api.TaskQueue

Do we have more task waiting for service

hasNext() - Method in class org.apache.droids.impl.SimpleTaskQueue

HtmlParser - Class in org.apache.droids.parse.html

HtmlParser() - Constructor for class org.apache.droids.parse.html.HtmlParser

Http - Class in org.apache.droids.protocol.http

Simple implementation for http protocol.

Http() - Constructor for class org.apache.droids.protocol.http.Http

HttpBase - Class in org.apache.droids.protocol

Helper class that provides basic methods like returning the agent string and content type.

HttpBase() - Constructor for class org.apache.droids.protocol.HttpBase

I

init() - Method in interface org.apache.droids.api.Droid

Initialize the queue.

init() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

init() - Method in class org.apache.droids.robot.walker.SimpleWalkingDroid

InvalidTaskException - Exception in org.apache.droids.exception

InvalidTaskException(String) - Constructor for exception org.apache.droids.exception.InvalidTaskException

isAllowed(String) - Method in interface org.apache.droids.api.Protocol

Some protocols (like http) offer a mechanism to evaluate whether the client can request a given url (in http this is the robots.txt configuration)

isAllowed(String) - Method in interface org.apache.droids.norobots.Rule

Boolean.TRUE means it is allowed.

isAllowed(String) - Method in class org.apache.droids.protocol.file.FileProtocol

isAllowed(String) - Method in class org.apache.droids.protocol.http.Http

isContentType(String) - Static method in class org.apache.droids.protocol.MediaType

Deprecated. Test whether a given type is in our array of known media types.

isForceAllow() - Method in class org.apache.droids.protocol.http.Http

You can force that a site is allowed (ignoring the robots.txt).

isIncludeHost() - Method in class org.apache.droids.handle.Save

Do we want to prefix the export dir with the host name.

isUrlAllowed(URL) - Method in class org.apache.droids.norobots.NoRobotClient

Decide if the parsed website will allow this URL to be be seen.

L

Link - Interface in org.apache.droids.api

Simple extension of a Task.

LinkTask - Class in org.apache.droids

LinkTask(Link, URI, int) - Constructor for class org.apache.droids.LinkTask

log - Variable in class org.apache.droids.handle.WriterHandler

log - Variable in class org.apache.droids.helper.factories.GenericFactory

log - Variable in class org.apache.droids.helper.Loggable

Loggable - Class in org.apache.droids.helper

Simple wrapper class to easier debug/log.

Loggable() - Constructor for class org.apache.droids.helper.Loggable

M

main(String[]) - Static method in class org.apache.droids.robot.crawler.CrawlingDroid

main(String[]) - Static method in class org.apache.droids.robot.walker.SimpleWalkingDroid

match(String) - Method in class org.apache.droids.net.RegexRule

Checks if a url matches this rule.

MediaType - Class in org.apache.droids.protocol

Deprecated. Soon to be replaced via the tika mediaType support

merge(T) - Method in interface org.apache.droids.api.TaskQueue

Add a Task to the queue...

merge(Collection<? extends T>) - Method in interface org.apache.droids.api.TaskQueue

Merge a given list of tasks with the current queue.

merge(T) - Method in class org.apache.droids.impl.SimpleTaskQueue

merge(Collection<? extends T>) - Method in class org.apache.droids.impl.SimpleTaskQueue

Tasks with duplicate keys will be overwritten

merge(T) - Method in class org.apache.droids.impl.SimpleTaskQueueWithHistory

minimumDelay - Variable in class org.apache.droids.delay.RandomDelayTimer

MultiThreadedTaskMaster<T extends Task> - Class in org.apache.droids.impl

MultiThreadedTaskMaster() - Constructor for class org.apache.droids.impl.MultiThreadedTaskMaster

N

next() - Method in interface org.apache.droids.api.TaskQueue

Return the next task that is waiting for service

next() - Method in class org.apache.droids.impl.SimpleTaskQueue

NoRobotClient - Class in org.apache.droids.norobots

A Client which may be used to decide which urls on a website may be looked at, according to the norobots specification located at: http://www.robotstxt.org/wc/norobots-rfc.html

NoRobotClient(String) - Constructor for class org.apache.droids.norobots.NoRobotClient

Create a Client for a particular user-agent name.

NoRobotException - Exception in org.apache.droids.norobots

Application exception for anything that might go wrong in the checking of a robots.txt file.

NoRobotException(String) - Constructor for exception org.apache.droids.norobots.NoRobotException

NoRobotException(String, Throwable) - Constructor for exception org.apache.droids.norobots.NoRobotException

O

openStream(String) - Method in interface org.apache.droids.api.Protocol

Return the stream represent of the url

openStream(String) - Method in class org.apache.droids.protocol.file.FileProtocol

openStream(String) - Method in class org.apache.droids.protocol.http.Http

org.apache.droids - package org.apache.droids

This package is the principal package for Apache Droids.

org.apache.droids.api - package org.apache.droids.api

This package defines all interfaces that we are using for droids.

org.apache.droids.delay - package org.apache.droids.delay

This package is the principal package for Apache Droids Delay Timers.

org.apache.droids.exception - package org.apache.droids.exception

This package defines some custom exceptions that we are using in droids.

org.apache.droids.handle - package org.apache.droids.handle

This package contains some basic implementations of various handlers.

org.apache.droids.helper - package org.apache.droids.helper

This package contains various helper.

org.apache.droids.helper.factories - package org.apache.droids.helper.factories

This package contains all core factories that we use in Droids.

org.apache.droids.impl - package org.apache.droids.impl

org.apache.droids.net - package org.apache.droids.net

This package contains various helper for the work with protocols and network communication.

org.apache.droids.norobots - package org.apache.droids.norobots

Using norobots-rfc

org.apache.droids.parse - package org.apache.droids.parse

This package contains various helper and implementations around parsing.

org.apache.droids.parse.html - package org.apache.droids.parse.html

This package contains various parser.

org.apache.droids.protocol - package org.apache.droids.protocol

This package contains various class around the support of protocol specific classes.

org.apache.droids.protocol.file - package org.apache.droids.protocol.file

This package contains various file protocol implementations.

org.apache.droids.protocol.http - package org.apache.droids.protocol.http

This package contains various http protocol implementations.

org.apache.droids.robot.crawler - package org.apache.droids.robot.crawler

org.apache.droids.robot.walker - package org.apache.droids.robot.walker

org.apache.http - package org.apache.http

P

Parse - Interface in org.apache.droids.api

Wrapper object that encapsulate the result of the parsing of the underlying document.

parse(URL) - Method in class org.apache.droids.norobots.NoRobotClient

Head to a website and suck in their robots.txt file.

ParseData - Class in org.apache.droids

The result object that are filled by a parser

ParseData(Collection<Link>) - Constructor for class org.apache.droids.ParseData

Create a new instance of Parse data for the given outlinks

ParseImpl - Class in org.apache.droids.parse

Default implementation of Parse

ParseImpl(String, ParseData) - Constructor for class org.apache.droids.parse.ParseImpl

Create a new instance of a Parse for the given text and ParseData

Parser - Interface in org.apache.droids.api

Simple parser that is only forcing to return a parse object.

ParserFactory - Class in org.apache.droids.helper.factories

Factory that will lookup a parser by its identifier and return it.

ParserFactory() - Constructor for class org.apache.droids.helper.factories.ParserFactory

ParserNotFoundException - Exception in org.apache.droids.exception

ParserNotFoundException gives a detailed exception for problems that can occur while parsing a task.

ParserNotFoundException(String, String) - Constructor for exception org.apache.droids.exception.ParserNotFoundException

Create an exception for the given url and content type

ParserNotFoundException(String, String, String) - Constructor for exception org.apache.droids.exception.ParserNotFoundException

Create an exception for the given url and content type

ParserNotFoundException(String) - Constructor for exception org.apache.droids.exception.ParserNotFoundException

Constructs a new exception with the specified detail message.

parseText(String) - Method in class org.apache.droids.norobots.NoRobotClient

pipe(Reader, Writer) - Static method in class org.apache.droids.handle.WriterHandler

Pipes everything from the reader to the writer via a buffer

post(String, String) - Method in class org.apache.http.PostFile

PostFile - Class in org.apache.http

PostFile(String, InputStream) - Constructor for class org.apache.http.PostFile

prepareConnection(URL) - Method in class org.apache.droids.protocol.HttpBase

Will prepare a HttpURLConnection with the userAgent, from, the refer and the timeout

processAllTasks(TaskQueue<T>, Droid<T>) - Method in interface org.apache.droids.api.TaskMaster

processAllTasks(TaskQueue<T>, Droid<T>) - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

The queue has been initialized

Protocol - Interface in org.apache.droids.api

The protocol interface is a wrapper to hide the underlying implementation of the communication at protocol level.

ProtocolFactory - Class in org.apache.droids.helper.factories

Factory that will lookup a protocol plugin and return it.

ProtocolFactory() - Constructor for class org.apache.droids.helper.factories.ProtocolFactory

ProtocolNotFoundException - Exception in org.apache.droids.exception

If we do not have any instance of a protocol registered for the iven url.

ProtocolNotFoundException(String) - Constructor for exception org.apache.droids.exception.ProtocolNotFoundException

Create an exception for the given url

ProtocolNotFoundException(String, String) - Constructor for exception org.apache.droids.exception.ProtocolNotFoundException

Create an exception for the given url and detailed message

Q

queue - Variable in class org.apache.droids.AbstractDroid

R

random - Variable in class org.apache.droids.delay.RandomDelayTimer

RandomDelayTimer - Class in org.apache.droids.delay

RandomDelayTimer() - Constructor for class org.apache.droids.delay.RandomDelayTimer

RandomDelayTimer(int, int) - Constructor for class org.apache.droids.delay.RandomDelayTimer

refer - Variable in class org.apache.droids.protocol.HttpBase

RegexRule - Class in org.apache.droids.net

A generic regular expression rule.

RegexRule(boolean) - Constructor for class org.apache.droids.net.RegexRule

Constructs a new regular expression rule.

RegexURLFilter - Class in org.apache.droids.net

Regular expression implementation of an UrlFilter.

RegexURLFilter() - Constructor for class org.apache.droids.net.RegexURLFilter

resolve(String) - Method in class org.apache.droids.helper.factories.GenericFactory

Will lookup which component is linked to the name and will return it.

Rule - Interface in org.apache.droids.norobots

A robots.txt rule.

S

Save - Class in org.apache.droids.handle

Handler which is writing the stream to the file system.

Save() - Constructor for class org.apache.droids.handle.Save

setAnchorText(String) - Method in class org.apache.droids.LinkTask

setDelayMillis(long) - Method in class org.apache.droids.delay.SimpleDelayTimer

Sets the delay time.

setDelaySpread(int) - Method in class org.apache.droids.delay.RandomDelayTimer

setDelayTimer(DelayTimer) - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

setElements(Map<String, String>) - Method in class org.apache.droids.parse.html.HtmlParser

setFile(String) - Method in class org.apache.droids.net.RegexURLFilter

setFiltersFactory(URLFiltersFactory) - Method in class org.apache.droids.robot.crawler.CrawlingDroid

setForceAllow(boolean) - Method in class org.apache.droids.protocol.http.Http

You can force that a site is allowed (ignoring the robot.txt).

setFrom(String) - Method in class org.apache.droids.protocol.HttpBase

Set the eMail address of the bot.

setHandlerFactory(HandlerFactory) - Method in class org.apache.droids.robot.crawler.CrawlingDroid

setIncludeHost(boolean) - Method in class org.apache.droids.handle.Save

Do we want to prefix the export dir with the host name.

setInitialFiles(Collection<File>) - Method in class org.apache.droids.robot.walker.SimpleWalkingDroid

setInitialFiles(Collection<File>) - Method in interface org.apache.droids.robot.walker.WalkingDroid

setInitialLocations(Collection<String>) - Method in class org.apache.droids.robot.crawler.CrawlingDroid

setLastModifedDate(Date) - Method in class org.apache.droids.LinkTask

setLinksTo(Collection<URI>) - Method in class org.apache.droids.LinkTask

setMap(Map) - Method in class org.apache.droids.helper.factories.GenericFactory

Set the register which contains all components.

setMaxDepth(int) - Method in class org.apache.droids.impl.SimpleTaskValidator

setMaxSize(int) - Method in class org.apache.droids.impl.SimpleTaskQueue

setMaxThreads(int) - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

Adjust number of allowed threads

setMinimumDelay(int) - Method in class org.apache.droids.delay.RandomDelayTimer

setOutputDir(String) - Method in class org.apache.droids.handle.Save

Set the directory where we want to save the stream.

setParserFactory(ParserFactory) - Method in class org.apache.droids.robot.crawler.CrawlingDroid

setProtocolFactory(ProtocolFactory) - Method in class org.apache.droids.robot.crawler.CrawlingDroid

setRefer(String) - Method in class org.apache.droids.protocol.HttpBase

Set the refer URI where the bot is send from.

setTaskValidator(TaskValidator<T>) - Method in class org.apache.droids.impl.SimpleTaskQueue

setTimeout(int) - Method in class org.apache.droids.protocol.HttpBase

Set the timeout we want for the connection.

setUserAgent(String) - Method in class org.apache.droids.protocol.HttpBase

Set the name of our UserAgent

shutdownAndAwaitTermination() - Method in class org.apache.droids.impl.MultiThreadedTaskMaster

Shutdown all threads, close the pools and leave.

SimpleDelayTimer - Class in org.apache.droids.delay

SimpleDelayTimer() - Constructor for class org.apache.droids.delay.SimpleDelayTimer

SimpleDelayTimer(long) - Constructor for class org.apache.droids.delay.SimpleDelayTimer

SimpleTaskQueue<T extends Task> - Class in org.apache.droids.impl

A simple

SimpleTaskQueue() - Constructor for class org.apache.droids.impl.SimpleTaskQueue

Simple queue constructor.

SimpleTaskQueueWithHistory<T extends Task> - Class in org.apache.droids.impl

Extend the task queue to ignore any tasks we have already seen

SimpleTaskQueueWithHistory() - Constructor for class org.apache.droids.impl.SimpleTaskQueueWithHistory

Simple queue constructor.

SimpleTaskValidator<T extends Task> - Class in org.apache.droids.impl

A simple

SimpleTaskValidator() - Constructor for class org.apache.droids.impl.SimpleTaskValidator

SimpleTaskValidator(int) - Constructor for class org.apache.droids.impl.SimpleTaskValidator

SimpleWalkingDroid - Class in org.apache.droids.robot.walker

SimpleWalkingDroid(TaskQueue<FileTask>, TaskMaster<FileTask>) - Constructor for class org.apache.droids.robot.walker.SimpleWalkingDroid

start() - Method in class org.apache.droids.AbstractDroid

start() - Method in interface org.apache.droids.api.Droid

Invoke an instance of the worker used in the droid

start() - Method in class org.apache.droids.robot.crawler.CrawlingDroid

statusCode() - Method in class org.apache.http.PostFile

StAX - Class in org.apache.droids.helper

Helper class that eases the usage of StAX in your plugins.

StAX() - Constructor for class org.apache.droids.helper.StAX

Easy helper to get StAX based parser and writer.

Sysout - Class in org.apache.droids.handle

Handler that write the stream to the sysout.

Sysout() - Constructor for class org.apache.droids.handle.Sysout

T

Task - Interface in org.apache.droids.api

A task is a working instruction for a droid.

taskMaster - Variable in class org.apache.droids.AbstractDroid

TaskMaster<T extends Task> - Interface in org.apache.droids.api

Responsible for running all the tasks

TaskQueue<T extends Task> - Interface in org.apache.droids.api

A queue is the data structure where the different tasks are waiting for service.

TaskValidator<T extends Task> - Interface in org.apache.droids.api

timeout - Variable in class org.apache.droids.protocol.HttpBase

U

URLFilter - Interface in org.apache.droids.api

Filter to limit the urls that we want to allow in our queue.

URLFiltersFactory - Class in org.apache.droids.helper.factories

Factory that will traverse all registered filter and execute them.

URLFiltersFactory() - Constructor for class org.apache.droids.helper.factories.URLFiltersFactory

UrlHelper - Class in org.apache.droids.net

Helper class that offers a couple of method to work with urls

userAgent - Variable in class org.apache.droids.protocol.HttpBase

V

validateTask(T) - Method in interface org.apache.droids.api.TaskValidator

This will take a task and make sure it is valid.

validateTask(T) - Method in class org.apache.droids.impl.SimpleTaskQueue

validateTask(T) - Method in class org.apache.droids.impl.SimpleTaskValidator

W

WalkingDroid - Interface in org.apache.droids.robot.walker

Worker<T extends Task> - Interface in org.apache.droids.api

A worker is the unit that is doing the actual work.

WriterHandler - Class in org.apache.droids.handle

Wrapper that allows you to pipe a stream from a reader to a writer via a buffer

WriterHandler() - Constructor for class org.apache.droids.handle.WriterHandler

A B C D E F G H I L M N O P Q R S T U V W