NUTCH-436 resolved. Fixed behavior of urls with param (i.e. ;xxxx) information. Finally found workaround for problems that I was experiencing with EOL characters.
lucene/nutch/trunk/src/plugin/parse-html/src/java/org/apache/nutch/parse/html/DOMContentUtils.java