Modifier and Type | Field and Description |
---|---|
static org.slf4j.Logger |
LOG |
accept, acceptLanguage, BUFFER_SIZE, maxContent, maxCrawlDelay, proxyHost, proxyPort, RESPONSE_TIME, responseTime, timeout, useHttp11, useProxy, userAgent
CHECK_BLOCKING, CHECK_ROBOTS, X_POINT_ID
Constructor and Description |
---|
Http()
Public default constructor.
|
Modifier and Type | Method and Description |
---|---|
protected Response |
getResponse(URL url,
CrawlDatum datum,
boolean redirect) |
static void |
main(String[] args) |
void |
setConf(org.apache.hadoop.conf.Configuration conf)
Set the
Configuration
object. |
getAccept, getAcceptLanguage, getConf, getMaxContent, getProtocolOutput, getProxyHost, getProxyPort, getRobotRules, getTimeout, getUseHttp11, getUserAgent, logConf, main, processDeflateEncoded, processGzipEncoded, useProxy
public void setConf(org.apache.hadoop.conf.Configuration conf)
Configuration
object.protected Response getResponse(URL url, CrawlDatum datum, boolean redirect) throws ProtocolException, IOException
getResponse
in class HttpBase
ProtocolException
IOException
Copyright © 2014 The Apache Software Foundation