source: gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_import.pl

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @28015   11 years jmt12 Add an extra option that allows me to pass in the directory to write …
(edit) @27913   11 years jmt12 Made the ingester to be used (version 1 without reduce phase, or …
(edit) @27732   11 years jmt12 Nice the copy itself too
(edit) @27686   11 years jmt12 A little more progress comments
(edit) @27654   11 years jmt12 Add the ability to stagger the starting of Mappers by placing a …
(edit) @27644   11 years jmt12 Extended to support HDFS-access via NFS. This applies to both the call …
(edit) @27594   11 years jmt12 Extend hadoop_import.pl to be able to start and stop the Thrift server(s)
(edit) @27584   11 years jmt12 I wasn't doing -r when attempting to clear directories left in /tmp by …
(edit) @27550   11 years jmt12 Ensure the hostname is added to the Hadoop logs so we can identify the …
(edit) @27530   11 years jmt12 Clear out old logs, and adding more comments about what the script is …
(edit) @27495   11 years jmt12 removing doubled up debug comments and putting some paths in …
(edit) @27414   11 years jmt12 Allowing more processing arguments to be configured at the call, and …
(edit) @27126   11 years jmt12 Extra clean up commands (like removing cached versions of video …
(edit) @27058   11 years jmt12 Adding data locality report generation to Hadoop greenstone imports
(edit) @27001   11 years jmt12 Passing more environment variables (HADOOPPREFIX, HDFSHOST, HDFSPORT) …
(add) @26949   11 years jmt12 Parallel import using Hadoop
Note: See TracRevisionLog for help on using the revision log.