Log of /lucene/nutch/trunk/src/java/org/apache/nutch/indexer/FsDirectory.java

---

Revision 823614 - (view) (annotate) - [select for diffs]
Modified Fri Oct 9 17:02:32 2009 UTC (6 weeks, 5 days ago) by ab
File length: 7454 byte(s)
Diff to previous 722475 (colored)

NUTCH-758 Set subversion eol-style to "native".

---

Revision 722475 - (view) (annotate) - [select for diffs]
Modified Tue Dec 2 14:41:09 2008 UTC (11 months, 3 weeks ago) by kubes
File length: 7454 byte(s)
Diff to previous 678533 (colored)

NUTCH-662: Upgrade Nutch to use Lucene 2.4

---

Revision 678533 - (view) (annotate) - [select for diffs]
Modified Mon Jul 21 19:20:21 2008 UTC (16 months ago) by ab
File length: 6941 byte(s)
Diff to previous 613378 (colored)

NUTCH-634 Upgrade Nutch to Hadoop 0.17.1 .

---

Revision 613378 - (view) (annotate) - [select for diffs]
Modified Sat Jan 19 08:59:29 2008 UTC (22 months, 1 week ago) by siren
File length: 6797 byte(s)
Diff to previous 591791 (colored)

NUTCH-580 Remove deprecated hadoop api calls (FS)

---

Revision 591791 - (view) (annotate) - [select for diffs]
Modified Sun Nov 4 15:38:35 2007 UTC (2 years ago) by kubes
File length: 6642 byte(s)
Diff to previous 549638 (colored)

NUTCH-552 - Upgrade Nutch to Hadoop 0.15.x.

---

Revision 549638 - (view) (annotate) - [select for diffs]
Modified Thu Jun 21 22:52:02 2007 UTC (2 years, 5 months ago) by ab
File length: 6927 byte(s)
Diff to previous 473936 (colored)

Upgrade to Lucene 2.2.0 and Hadoop 0.12.3.

---

Revision 473936 - (view) (annotate) - [select for diffs]
Modified Sun Nov 12 11:37:02 2006 UTC (3 years ago) by siren
File length: 6910 byte(s)
Diff to previous 405204 (colored)

NUTCH-400 update headers

---

Revision 405204 - (view) (annotate) - [select for diffs]
Modified Mon May 8 22:34:29 2006 UTC (3 years, 6 months ago) by cutting
File length: 6722 byte(s)
Diff to previous 374796 (colored)

Change parameters passed to Hadoop's FileSystem from (now-deprecated) java.io.File to (new) org.apache.hadoop.fs.Path.

---

Revision 374796 - (view) (annotate) - [select for diffs]
Modified Sat Feb 4 00:38:32 2006 UTC (3 years, 9 months ago) by cutting
File length: 6720 byte(s)
Copied from: lucene/nutch/trunk/src/java/org/apache/nutch/indexer/NdfsDirectory.java revision 374762
Diff to previous 373853 (colored)

NUTCH-193: MapReduce and NDFS code moved to new project, Hadoop.  See bug report for details.

---

Revision 373853 - (view) (annotate) - [select for diffs]
Modified Tue Jan 31 16:08:58 2006 UTC (3 years, 9 months ago) by ab
Original Path: lucene/nutch/trunk/src/java/org/apache/nutch/indexer/NdfsDirectory.java
File length: 6746 byte(s)
Diff to previous 357197 (colored)

Apply patches from NUTCH-169 (remove static NutchConf).

Submitted by: Marko Bauhardt, Stefan Groschupf, Jerome Charron.

---

Revision 357197 - (view) (annotate) - [select for diffs]
Modified Fri Dec 16 17:51:05 2005 UTC (3 years, 11 months ago) by cutting
Original Path: lucene/nutch/trunk/src/java/org/apache/nutch/indexer/NdfsDirectory.java
File length: 6445 byte(s)
Copied from: lucene/nutch/branches/mapred/src/java/org/apache/nutch/indexer/NdfsDirectory.java revision 357183
Diff to previous 320835 (colored)

Merge mapred branch to trunk & remove it.

---

Revision 320835 - (view) (annotate) - [select for diffs]
Modified Thu Oct 13 17:59:30 2005 UTC (4 years, 1 month ago) by cutting
Original Path: lucene/nutch/branches/mapred/src/java/org/apache/nutch/indexer/NdfsDirectory.java
File length: 6445 byte(s)
Diff to previous 307203 (colored)

Store checksums for all files written and verify them on read.  CRCs are stored for every 512 bytes of data, so that randomly accessed data may be verified.  Errors are reported to the filesystem implementation.  Local file errors cause files to be moved to a bad file directory, so that bad disk areas are not reused.  NDFS file errors should cause blocks to be moved to a bad block directory on the datanode, forcing the use of replicas of the bad blocks with no loss of data.  This is not yet implemented for NDFS.

---

Revision 307203 - (view) (annotate) - [select for diffs]
Modified Fri Oct 7 22:16:27 2005 UTC (4 years, 1 month ago) by cutting
Original Path: lucene/nutch/branches/mapred/src/java/org/apache/nutch/indexer/NdfsDirectory.java
File length: 6437 byte(s)
Diff to previous 213607 (colored)

First working version of MapReduce-based dedup.

---

Revision 213607 - (view) (annotate) - [select for diffs]
Modified Mon Jul 11 21:30:22 2005 UTC (4 years, 4 months ago) by cutting
Original Path: lucene/nutch/branches/mapred/src/java/org/apache/nutch/indexer/NdfsDirectory.java
File length: 6242 byte(s)
Diff to previous 210199 (colored)

Get search working on NDFS-resident, MapReduce-created crawl.

---

Revision 210199 - (view) (annotate) - [select for diffs]
Added Mon Jul 11 20:04:09 2005 UTC (4 years, 4 months ago) by cutting
Original Path: lucene/nutch/branches/mapred/src/java/org/apache/nutch/indexer/NdfsDirectory.java
File length: 6241 byte(s)

New class to permit reading of Lucene indexes stored in NDFS.  Writing
is not yet supported, since Lucene (in only one place!) requires
random access when writing indexes, and NDFS does not support random
access when writing files.