org.apache.nutch.tools

Class CommonCrawlFormatSimple

java.lang.Object

org.apache.nutch.tools.AbstractCommonCrawlFormat

org.apache.nutch.tools.CommonCrawlFormatSimple

All Implemented Interfaces:

CommonCrawlFormat

public class CommonCrawlFormatSimple
extends AbstractCommonCrawlFormat

This class provides methods to map crawled data on JSON using a StringBuilder object.

Field Summary

Fields inherited from class org.apache.nutch.tools.AbstractCommonCrawlFormat

conf, content, jsonArray, keyPrefix, metadata, reverseKey, reverseKeyValue, simpleDateFormat, url

Constructor Summary

Constructors

Constructor and Description
CommonCrawlFormatSimple(String url, byte[] content, Metadata metadata, Configuration nutchConf, CommonCrawlConfig config)

Method Summary

Methods

Modifier and Type	Method and Description
protected void	closeArray(String key, boolean nested, boolean newline)
protected void	closeObject(String key)
protected String	generateJson()
protected void	startArray(String key, boolean nested, boolean newline)
protected void	startObject(String key)
protected void	writeArrayValue(String value)
protected void	writeKeyNull(String key)
protected void	writeKeyValue(String key, String value)

Methods inherited from class org.apache.nutch.tools.AbstractCommonCrawlFormat

getImported, getJsonData, getKey, getMethod, getRequestAccept, getRequestAcceptEncoding, getRequestAcceptLanguage, getRequestContactEmail, getRequestContactName, getRequestHostAddress, getRequestHostName, getRequestRobots, getRequestSoftware, getRequestUserAgent, getResponseAddress, getResponseContent, getResponseContentEncoding, getResponseContentType, getResponseDate, getResponseHostName, getResponseServer, getResponseStatus, getTimestamp, getUrl

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CommonCrawlFormatSimple

public CommonCrawlFormatSimple(String url,
                       byte[] content,
                       Metadata metadata,
                       Configuration nutchConf,
                       CommonCrawlConfig config)
                        throws IOException

Throws:

IOException

Method Detail

writeKeyValue

protected void writeKeyValue(String key,
                 String value)
                      throws IOException

Specified by:

writeKeyValue in class AbstractCommonCrawlFormat

Throws:

IOException

writeKeyNull

protected void writeKeyNull(String key)
                     throws IOException

Specified by:

writeKeyNull in class AbstractCommonCrawlFormat

Throws:

IOException

startArray

protected void startArray(String key,
              boolean nested,
              boolean newline)
                   throws IOException

Specified by:

startArray in class AbstractCommonCrawlFormat

Throws:

IOException

closeArray

protected void closeArray(String key,
              boolean nested,
              boolean newline)
                   throws IOException

Specified by:

closeArray in class AbstractCommonCrawlFormat

Throws:

IOException

writeArrayValue

protected void writeArrayValue(String value)

Specified by:

writeArrayValue in class AbstractCommonCrawlFormat

startObject

protected void startObject(String key)
                    throws IOException

Specified by:

startObject in class AbstractCommonCrawlFormat

Throws:

IOException

closeObject

protected void closeObject(String key)
                    throws IOException

Specified by:

closeObject in class AbstractCommonCrawlFormat

Throws:

IOException

generateJson

protected String generateJson()
                       throws IOException

Specified by:

generateJson in class AbstractCommonCrawlFormat

Throws:

IOException