Changeset 27550

Show
Ignore:
Timestamp:
05.06.2013 13:06:32 (6 years ago)
Author:
jmt12
Message:

Ensure the hostname is added to the Hadoop logs so we can identify the compute node (maybe)

Files:
1 modified

Legend:

Unmodified
Added
Removed
  • gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_import.pl

    r27530 r27550  
    216216print " * Running import using Hadoop..."; 
    217217my $hadoop_log = $gs_results_dir . '/hadoop.log'; 
    218 my $hadoop_command = $hadoop_exe . ' jar ' . $gsdl_hadoop_ext . '/lib/hadoop-greenstone.jar org.nzdl.gsdl.HadoopGreenstoneIngest "' . $gsdl_home . '" "' . $hdfs_fs_prefix . '" "' . $ENV{'HADOOP_PREFIX'} . '" ' . $collection . ' "' .  $hdfs_input_dir . '" "' . $hdfs_output_dir . '" > ' . $hadoop_log . ' 2>&1'; 
     218&shellCommand('echo "host:$HDFSHOST" > ' . $hadoop_log); 
     219my $hadoop_command = $hadoop_exe . ' jar ' . $gsdl_hadoop_ext . '/lib/hadoop-greenstone.jar org.nzdl.gsdl.HadoopGreenstoneIngest "' . $gsdl_home . '" "' . $hdfs_fs_prefix . '" "' . $ENV{'HADOOP_PREFIX'} . '" ' . $collection . ' "' .  $hdfs_input_dir . '" "' . $hdfs_output_dir . '" >> ' . $hadoop_log . ' 2>&1'; 
    219220&shellCommand($hadoop_command); 
    220221print "Done!\n";