org.apache.nutch.segment

Class SegmentChecker

java.lang.Object

org.apache.nutch.segment.SegmentChecker

public class SegmentChecker
extends Object

Checks whether a segment is valid, or has a certain status (generated, fetched, parsed), or can be used safely for a certain processing step (e.g., indexing).

Field Summary

Fields

Modifier and Type	Field and Description
static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
SegmentChecker()

Method Summary

Methods

Modifier and Type	Method and Description
static boolean	checkSegmentDir(Path segmentPath, FileSystem fs) Check the segment to see if it is valid based on the sub directories.
static boolean	isIndexable(Path segmentPath, FileSystem fs) Check if the segment is indexable.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

Constructor Detail

SegmentChecker

public SegmentChecker()

Method Detail

isIndexable

public static boolean isIndexable(Path segmentPath,
                  FileSystem fs)
                           throws IOException

Check if the segment is indexable. May add new check methods here.

Throws:

IOException

checkSegmentDir

public static boolean checkSegmentDir(Path segmentPath,
                      FileSystem fs)
                               throws IOException

Check the segment to see if it is valid based on the sub directories.

Throws:

IOException