|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.nutch.tools.ResolveUrls
public class ResolveUrls
A simple tool that will spin up multiple threads to resolve urls to ip addresses. This can be used to verify that pages that are failing due to UnknownHostException during fetching are actually bad and are not failing due to a dns problem in fetching.
Field Summary | |
---|---|
static org.slf4j.Logger |
LOG
|
Constructor Summary | |
---|---|
ResolveUrls(String urlsFile)
Create a new ResolveUrls with a file from the local file system. |
|
ResolveUrls(String urlsFile,
int numThreads)
Create a new ResolveUrls with a urls file and a number of threads for the Thread pool. |
Method Summary | |
---|---|
static void |
main(String[] args)
Runs the resolve urls tool. |
void |
resolveUrls()
Creates a thread pool for resolving urls. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final org.slf4j.Logger LOG
Constructor Detail |
---|
public ResolveUrls(String urlsFile)
urlsFile
- The local urls file, one url per line.public ResolveUrls(String urlsFile, int numThreads)
urlsFile
- The local urls file, one url per line.numThreads
- The number of threads used to resolve urls in parallel.Method Detail |
---|
public void resolveUrls()
public static void main(String[] args)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |