===== Terrier: Parallel Processing =====

Intended to be run on processor intensive collections containing Images and/or Videos (support available in the 'video-and-audio' extension).

==== Installation Instructions ====

1. Link/Copy the new simple file indexer application into Terrier:

cd <greenstone_path>/ext/parallel-building/opt/Terrier/
ln -s FileIndexer.java <terrier_path>/src/core/org/terrier/applications/FileIndexer.java

2. Recompile Terrier:

cd <terrier>
ant

3. Ensure Greenstone's "source setup.bash" has been run, and that parallel_terrier_fileindex.pl and mpiterrierfileindexer are on the path.

4. You can then parallel ingest a collection using a command like this:

parallel_terrier_fileindexer.pl
  -workers <no_of_workers>
  -terrier <terrier_path>
  -collection <collection_path>
  -batchsize <size_of_batch>
  [-debug]

5. Review your collection in the web interface to ensure it built correctly.