Changeset 11245

Timestamp:

2006-02-14T15:36:29+13:00 (18 years ago)

Author:

kjdon

Message:

by default, lucene indexer will only index the first 10,000 words of a document, to avoid out of memory errors. I have set the max doc lenght to be max integer value. hope this is ok.

Location:

trunk

Files:

• gsdl/src/java/org/nzdl/gsdl/LuceneWrap/Indexer.java (modified) (1 diff)
• indexers/lucene-gs/src/org/greenstone/LuceneWrapper/Indexer.java (modified) (1 diff)

trunk/gsdl/src/java/org/nzdl/gsdl/LuceneWrap/Indexer.java

@@ -52 +52 @@
 
         writer_ = new IndexWriter(index_dir.getPath(), new StandardAnalyzer(), create);
+        // by default, will only index 10,000 words per document
+        // Can throw out_of_memory errors
+        writer_.maxFieldLength = Integer.MAX_VALUE;
         if (create) {
         writer_.optimize();

trunk/indexers/lucene-gs/src/org/greenstone/LuceneWrapper/Indexer.java

@@ -52 +52 @@
 
         writer_ = new IndexWriter(index_dir.getPath(), new StandardAnalyzer(), create);
+        // by default, will only index 10,000 words per document
+        // Can throw out_of_memory errors
+        writer_.maxFieldLength = Integer.MAX_VALUE;
         if (create) {
         writer_.optimize();