# # ChangeLog for gs2-extensions/parallel-building # # Generated by Trac 1.4.2 # 2024-03-29T05:22:19+13:00 Wed, 05 Jun 2013 23:16:31 GMT jmt12 [27560] * gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_report.pl (modified) Fixing typo in regexp that meant filenames sometimes ignored Wed, 05 Jun 2013 23:15:28 GMT jmt12 [27559] * gs2-extensions/parallel-building/trunk/src/bin/script/generate_gantt.pl (modified) Changed mime-type away from binary - I hope. Meanwhile, generate ... Wed, 05 Jun 2013 23:11:10 GMT jmt12 [27558] * gs2-extensions/parallel-building/trunk/src/src/java/org/nzdl/gsdl/HadoopGreenstoneIngest.java (modified) Forgot that Hadoop Map processes no longer have the environment ... Wed, 05 Jun 2013 01:07:43 GMT jmt12 [27551] * gs2-extensions/parallel-building/trunk/src/bin/script/generate_gantt.pl (modified) Altered so that it expects to be given a CSV containing parallel ... Wed, 05 Jun 2013 01:06:32 GMT jmt12 [27550] * gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_import.pl (modified) Ensure the hostname is added to the Hadoop logs so we can identify ... Wed, 05 Jun 2013 01:04:58 GMT jmt12 [27549] * gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_report.pl (added) Extract information from the logs generated by parallel Greenstone ... Wed, 05 Jun 2013 01:04:30 GMT jmt12 [27548] * gs2-extensions/parallel-building/trunk/src/bin/script/openmpi_report.pl (added) Extract information from the logs generated by parallel Greenstone ... Wed, 05 Jun 2013 01:03:03 GMT jmt12 [27547] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) Rejigging some processing comments Wed, 05 Jun 2013 01:02:06 GMT jmt12 [27546] * gs2-extensions/parallel-building/trunk/src/src/java/org/nzdl/gsdl/HadoopGreenstoneIngest.java (modified) Adding the ability for the Hadoop Mapper to determine what CPU number ... Wed, 05 Jun 2013 01:00:33 GMT jmt12 [27545] * gs2-extensions/parallel-building/trunk/src/src/getcpu-src/.svnignore (added) Ignoring just the compiled file (for now) Wed, 05 Jun 2013 01:00:05 GMT jmt12 [27544] * gs2-extensions/parallel-building/trunk/src/src/getcpu-src (added) * gs2-extensions/parallel-building/trunk/src/src/getcpu-src/getcpu.cpp (added) A tiny C script to guesstimate the CPU the calling Process is on Tue, 04 Jun 2013 23:53:16 GMT jmt12 [27543] * gs2-extensions/parallel-building/trunk/src/bin/script/generate_gantt.pl (added) Adding generate_gantt.pl script in its original form - i.e. directly ... Mon, 03 Jun 2013 23:12:36 GMT jmt12 [27532] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) Add the ability to configure the Thrift connector using a ... Mon, 03 Jun 2013 23:11:39 GMT jmt12 [27531] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils.pm (modified) Only output the message about using copy instead of hard/soft link once Mon, 03 Jun 2013 23:08:37 GMT jmt12 [27530] * gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_import.pl (modified) Clear out old logs, and adding more comments about what the script is ... Mon, 03 Jun 2013 21:28:48 GMT jmt12 [27526] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils.pm (modified) Adding in a 'isHDFS()' function so that some plugins ... Mon, 03 Jun 2013 21:27:12 GMT jmt12 [27525] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDFSShell.pm (modified) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/LocalFS.pm (modified) Adding in a 'isHDFS()' function so that some plugins ... Thu, 30 May 2013 00:15:06 GMT jmt12 [27515] * gs2-extensions/parallel-building/trunk/src/bin/script/test_fileutils.pl (modified) Making the file used durig buffertes be configurable Wed, 29 May 2013 23:08:57 GMT jmt12 [27514] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) Altering code to allow configurable length of read/write buffer when ... Wed, 29 May 2013 22:16:22 GMT jmt12 [27512] * gs2-extensions/parallel-building/trunk/src/bin/script/test_fileutils.pl (modified) Adding in a special test for measuring the effect of altering ... Mon, 27 May 2013 23:39:12 GMT jmt12 [27496] * gs2-extensions/parallel-building/trunk/src/perllib/plugins/DirectoryPlugin.pm (modified) Replacing a smelly old util::file_exists() with a snazzy new ... Mon, 27 May 2013 23:38:08 GMT jmt12 [27495] * gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_import.pl (modified) removing doubled up debug comments and putting some paths in ... Mon, 27 May 2013 23:36:00 GMT jmt12 [27494] * gs2-extensions/parallel-building/trunk/src/src/java/org/nzdl/gsdl/HadoopGreenstoneIngest.java (modified) Fixing a truncated comment - or maybe I never wrote an end to it... Mon, 27 May 2013 23:35:07 GMT jmt12 [27493] * gs2-extensions/parallel-building/trunk/src/perllib/plugouts/GreenstoneXMLPlugout.pm (deleted) No longer required - not that sure why it was required in the first place Mon, 27 May 2013 23:34:27 GMT jmt12 [27492] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) Some versions of Hadoop add host and protocol information into paths ... Mon, 27 May 2013 23:33:26 GMT jmt12 [27491] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils.pm (modified) Repairing three bugs in makeAllDirectories - incorrect pattern meant ... Mon, 27 May 2013 22:53:27 GMT jmt12 [27490] * gs2-extensions/parallel-building/trunk/src/perllib/plugouts/BasePlugout.pm (deleted) No longer requires Mon, 27 May 2013 21:39:07 GMT jmt12 [27489] * gs2-extensions/parallel-building/trunk/src/packages/cpan/Thrift-0.9.0.tar.gz (deleted) Shouldn't have been here Mon, 27 May 2013 21:37:48 GMT jmt12 [27488] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/CPAN.sh (modified) Since I've got rid of the thousand DateTime prereq modules, I can ... Mon, 27 May 2013 21:33:33 GMT jmt12 [27487] * gs2-extensions/parallel-building/trunk/src/src/java/org/nzdl/gsdl/HadoopGreenstoneIngest.java (modified) Ensure Parallel Building path in environment (for ThriftFS) and that ... Mon, 27 May 2013 00:27:31 GMT jmt12 [27481] * gs2-extensions/parallel-building/trunk/src/bin/script/test_fileutils.pl (modified) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils.pm (modified) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/LocalFS.pm (modified) Adding makeAllDirectories() (which I'd only implemented in LocalFS) ... Mon, 27 May 2013 00:22:03 GMT jmt12 [27480] * gs2-extensions/parallel-building/trunk/src/bin/script/test_fileutils.pl (modified) Removing DateTime dependency (so HDFSShell will always fail ... Mon, 27 May 2013 00:14:51 GMT jmt12 [27479] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDFSShell.pm (modified) Remove time parsing as DateTime is a fricking nightmare to install ... Mon, 27 May 2013 00:14:01 GMT jmt12 [27478] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) Be a bit smarter about locating Perl version if not provided (rather ... Sun, 26 May 2013 22:29:09 GMT jmt12 [27477] * gs2-extensions/parallel-building/trunk/src/packages/ThriftFS-0.9.0.tar.gz (modified) Changed the order of additions to java classpath to ensure that my ... Sun, 26 May 2013 21:49:06 GMT jmt12 [27476] * gs2-extensions/parallel-building/trunk/src/packages/ThriftFS-0.9.0.tar.gz (modified) Sat, 25 May 2013 00:06:47 GMT jmt12 [27475] * gs2-extensions/parallel-building/trunk/src/packages/cpan/Thrift-0.9.0.tar.gz (added) Fri, 24 May 2013 23:59:06 GMT jmt12 [27474] * gs2-extensions/parallel-building/trunk/src/packages/cpan (modified) * gs2-extensions/parallel-building/trunk/src/packages/cpan/.svnignore (added) Ignoring extracted modules Fri, 24 May 2013 23:56:59 GMT jmt12 [27473] * gs2-extensions/parallel-building/trunk/src/packages/cpan/Class-Load-0.20.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Class-Load-XS-0.06.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Class-Singleton-1.4.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Data-OptList-0.107.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/DateTime-1.03.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/DateTime-Locale-0.45.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/DateTime-TimeZone-1.59.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/ExtUtils-CBuilder-0.280205.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/List-MoreUtils-0.33.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Build-0.4005.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Implementation-0.06.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Metadata-1.000014.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Runtime-0.013.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Params-Util-1.07.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Params-Validate-1.07.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Perl-OSType-1.003.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Sub-Install-0.926.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Test-Fatal-0.010.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Test-Requires-0.06.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Test-Simple-0.98.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Try-Tiny-0.12.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/parent-0.225.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/cpan/version-0.9902.tar.gz (deleted) I give up - I'll just do without DateTime Fri, 24 May 2013 23:13:11 GMT jmt12 [27472] * gs2-extensions/parallel-building/trunk/src/packages/cpan/ExtUtils-CBuilder-0.280205.tar.gz (added) Fri, 24 May 2013 23:11:26 GMT jmt12 [27471] * gs2-extensions/parallel-building/trunk/src/packages/cpan/Class-Load-XS-0.06.tar.gz (added) Fri, 24 May 2013 22:01:36 GMT jmt12 [27470] * gs2-extensions/parallel-building/trunk/src/packages (modified) * gs2-extensions/parallel-building/trunk/src/packages/.svnignore (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) No longer need to ignore all the files I moved to cpan Fri, 24 May 2013 21:57:11 GMT jmt12 [27469] * gs2-extensions/parallel-building/trunk/src/packages/cpan (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Bit-Vector-7.2.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Class-Load-0.20.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Class-Singleton-1.4.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Data-OptList-0.107.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/DateTime-1.03.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/DateTime-Locale-0.45.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/DateTime-TimeZone-1.59.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/IPC-Run-0.90.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/List-MoreUtils-0.33.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Build-0.4005.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Implementation-0.06.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Metadata-1.000014.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Module-Runtime-0.013.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Params-Util-1.07.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Params-Validate-1.07.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Perl-OSType-1.003.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Proc-Daemon-0.14.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Sort-Key-1.32.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Sub-Install-0.926.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Test-Fatal-0.010.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Test-Requires-0.06.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Test-Simple-0.98.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/Try-Tiny-0.12.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/parent-0.225.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/cpan/version-0.9902.tar.gz (added) A dedicated folder to hold the many, many CPAN modules I've ended up ... Fri, 24 May 2013 21:55:19 GMT jmt12 [27468] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/BITVECTOR.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/DATETIME.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULEBUILD.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULEIMPLEMENTATION.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULEMETADATA.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULERUNTIME.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/PARAMSVALIDATE.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/PERLOSTYPE.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TESTFATAL.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TESTREQUIRES.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TESTSIMPLE.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TRYTINY.sh (deleted) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/VERSION.sh (deleted) The (repeated) functionality of all these scripts moved into CPAN.sh Fri, 24 May 2013 21:53:24 GMT jmt12 [27467] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/CPAN.sh (added) An advanced cascade script that will loop through an (evergrowing) ... Fri, 24 May 2013 21:51:42 GMT jmt12 [27466] * gs2-extensions/parallel-building/trunk/src/packages/Bit-Vector-7.2.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Class-Singleton-1.4.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Data-OptList-0.107.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/DateTime-Locale-0.45.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/DateTime-TimeZone-1.59.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/IPC-Run-0.90.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/List-MoreUtils-0.33.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Module-Build-0.4005.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Module-Implementation-0.06.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Module-Metadata-1.000014.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Module-Runtime-0.013.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Params-Util-1.07.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Params-Validate-1.07.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Perl-OSType-1.003.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Proc-Daemon-0.14.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Sort-Key-1.32.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Test-Fatal-0.010.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Test-Requires-0.06.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Test-Simple-0.98.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Try-Tiny-0.12.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/parent-0.225.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/version-0.9902.tar.gz (deleted) moved to cpan folder in packages Fri, 24 May 2013 21:36:15 GMT jmt12 [27465] * gs2-extensions/parallel-building/trunk/src/packages/Class-Load-0.20.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/DateTime-1.03.tar.gz (deleted) * gs2-extensions/parallel-building/trunk/src/packages/Sub-Install-0.926.tar.gz (deleted) Moved to cpan directory so I can use smarter cascade make Fri, 24 May 2013 05:44:51 GMT jmt12 [27463] * gs2-extensions/parallel-building/trunk/src/packages/Sub-Install-0.926.tar.gz (added) Fri, 24 May 2013 05:42:29 GMT jmt12 [27462] * gs2-extensions/parallel-building/trunk/src/packages/Params-Util-1.07.tar.gz (added) Fri, 24 May 2013 05:40:07 GMT jmt12 [27461] * gs2-extensions/parallel-building/trunk/src/packages/Data-OptList-0.107.tar.gz (added) Fri, 24 May 2013 05:38:04 GMT jmt12 [27460] * gs2-extensions/parallel-building/trunk/src/packages/Class-Load-0.20.tar.gz (added) Fri, 24 May 2013 05:35:49 GMT jmt12 [27459] * gs2-extensions/parallel-building/trunk/src/packages/Class-Singleton-1.4.tar.gz (added) Fri, 24 May 2013 05:33:34 GMT jmt12 [27458] * gs2-extensions/parallel-building/trunk/src/packages/parent-0.225.tar.gz (added) Fri, 24 May 2013 05:28:11 GMT jmt12 [27457] * gs2-extensions/parallel-building/trunk/src/packages/DateTime-TimeZone-1.59.tar.gz (added) Fri, 24 May 2013 04:20:33 GMT jmt12 [27456] * gs2-extensions/parallel-building/trunk/src/packages/List-MoreUtils-0.33.tar.gz (added) Fri, 24 May 2013 04:17:15 GMT jmt12 [27455] * gs2-extensions/parallel-building/trunk/src/packages/DateTime-Locale-0.45.tar.gz (added) Fri, 24 May 2013 04:13:17 GMT jmt12 [27454] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULERUNTIME.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TESTREQUIRES.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TESTSIMPLE.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TRYTINY.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/Module-Runtime-0.013.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/Test-Requires-0.06.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/Test-Simple-0.98.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/Try-Tiny-0.12.tar.gz (added) This is like a satanic game of whack-a-mole. Here are the four ... Fri, 24 May 2013 03:57:43 GMT jmt12 [27453] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULEIMPLEMENTATION.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/Module-Implementation-0.06.tar.gz (added) Surprise - now Params::Validate has a prereq too, namely ... Fri, 24 May 2013 03:51:02 GMT jmt12 [27452] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/PARAMSVALIDATE.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/Params-Validate-1.07.tar.gz (added) Yet another prereq for DateTime - Params::Validate Fri, 24 May 2013 03:33:26 GMT jmt12 [27450] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/DATETIME.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULEBUILD.sh (modified) Moved opt_modulebuild_* functions into a library, and altered the ... Fri, 24 May 2013 03:18:47 GMT jmt12 [27448] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULEMETADATA.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/PERLOSTYPE.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/VERSION.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/Module-Metadata-1.000014.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/Perl-OSType-1.003.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/version-0.9902.tar.gz (added) Even more prerequisites for Module::Build - Perl::OSType, ... Fri, 24 May 2013 03:07:58 GMT jmt12 [27447] * gs2-extensions/parallel-building/trunk/src/packages (modified) ignore extracted Module::Build module Fri, 24 May 2013 03:07:39 GMT jmt12 [27446] * gs2-extensions/parallel-building/trunk/src/packages/.svnignore (modified) ignore extracted Module::Build module Fri, 24 May 2013 03:06:42 GMT jmt12 [27445] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/MODULEBUILD.sh (added) cascade files for Module::Build perl module Fri, 24 May 2013 03:05:57 GMT jmt12 [27444] * gs2-extensions/parallel-building/trunk/src/packages/Module-Build-0.4005.tar.gz (added) Craphat - ModuleBuild required for compiling DateTime Fri, 24 May 2013 02:47:53 GMT jmt12 [27443] * gs2-extensions/parallel-building/trunk/src/packages (modified) ignore new extracted directories (DateTime and Test::Fatal) Fri, 24 May 2013 02:47:24 GMT jmt12 [27442] * gs2-extensions/parallel-building/trunk/src/packages/.svnignore (modified) ignore new extracted directories (DateTime and Test::Fatal) Fri, 24 May 2013 02:46:02 GMT jmt12 [27441] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) Calling the cascade files for Test::Fatal and DateTime module building Fri, 24 May 2013 02:45:24 GMT jmt12 [27440] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/DATETIME.sh (added) cascade file for DateTime module. Uses customized functions as ... Fri, 24 May 2013 02:44:16 GMT jmt12 [27439] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/TESTFATAL.sh (added) cascade file for Test::Fatal module Fri, 24 May 2013 02:43:38 GMT jmt12 [27438] * gs2-extensions/parallel-building/trunk/src/packages/Test-Fatal-0.010.tar.gz (added) Module required by DateTime, which in turn is required by the ... Thu, 23 May 2013 23:15:12 GMT jmt12 [27437] * gs2-extensions/parallel-building/trunk/src/packages/DateTime-1.03.tar.gz (added) Why doesn't this exist on Medusa... I thought it was a built-in Thu, 23 May 2013 23:13:28 GMT jmt12 [27436] * gs2-extensions/parallel-building/trunk/src/bin/script/test_fileutils.pl (added) Adding the actual script - rather than a symlink to my dropbox. doh Thu, 23 May 2013 23:12:52 GMT jmt12 [27435] * gs2-extensions/parallel-building/trunk/src/bin/script/test_fileutils.pl (deleted) Gah - only a symbolic link Thu, 23 May 2013 22:58:39 GMT jmt12 [27434] * gs2-extensions/parallel-building/trunk/src/packages/ThriftFS-0.9.0.tar.gz (modified) Updated the starting script to look in the correct place (in this ... Thu, 23 May 2013 22:14:14 GMT jmt12 [27433] * gs2-extensions/parallel-building/trunk/src/src/java/org/nzdl/gsdl/HadoopGreenstoneIngest.java (modified) Adding a few more details to the Hadoop log to assist with later work Thu, 23 May 2013 21:40:07 GMT jmt12 [27424] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (modified) Adding canRead() and isAbsolute() functions, and some more debugging ... Thu, 23 May 2013 21:39:00 GMT jmt12 [27423] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDFSShell.pm (modified) Adding canRead() and isAbsolute() functions Thu, 23 May 2013 21:38:39 GMT jmt12 [27422] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/LocalFS.pm (modified) Adding canRead() and isAbsolute() functions Thu, 23 May 2013 21:38:05 GMT jmt12 [27421] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils.pm (modified) Fixing bug in filePutContents (wasn't shifting parameters off front ... Thu, 23 May 2013 21:34:37 GMT jmt12 [27420] * gs2-extensions/parallel-building/trunk/src/README.txt (modified) Updated to reflect changes in the perllib customization (most of it ... Thu, 23 May 2013 21:30:57 GMT jmt12 [27419] * gs2-extensions/parallel-building/trunk/src/setup.bash (modified) Adding more reasonable default values for HDFS host and port (so the ... Thu, 23 May 2013 21:27:45 GMT jmt12 [27418] * gs2-extensions/parallel-building/trunk/src/src/CASCADE-MAKE.sh (modified) Also recompile the HadoopGreenstoneIngest.java file during cascade make Thu, 23 May 2013 21:27:16 GMT jmt12 [27417] * gs2-extensions/parallel-building/trunk/src/src/CASCADE-MAKE/HADOOPGREENSTONEINGEST.sh (added) A 'pseudo'-CASCASE-MAKE file for building java code. It doesn't ... Thu, 23 May 2013 21:26:06 GMT jmt12 [27416] * gs2-extensions/parallel-building/trunk/src/src/java/org/nzdl/gsdl/HadoopGreenstoneIngest.java (modified) Forgot to ensure the manifests generated here are also version 2 Thu, 23 May 2013 21:25:00 GMT jmt12 [27415] * gs2-extensions/parallel-building/trunk/src/src/mpiimport-src/mpiimport.cpp (modified) Ensuring the manifests generated are marked as version 2 - i.e. ... Thu, 23 May 2013 21:24:16 GMT jmt12 [27414] * gs2-extensions/parallel-building/trunk/src/bin/script/hadoop_import.pl (modified) Allowing more processing arguments to be configured at the call, and ... Thu, 23 May 2013 21:23:29 GMT jmt12 [27413] * gs2-extensions/parallel-building/trunk/src/src/java/org/nzdl/gsdl/HadoopGreenstoneIngest.java (modified) Making the Hadoop processing aware that there are two different ... Thu, 23 May 2013 21:22:13 GMT jmt12 [27412] * gs2-extensions/parallel-building/trunk/src/bin/script/parse_task_info_from_hadoop_log.pl (modified) I obviously hadn't run this script on Karearea before - assumed all ... Thu, 23 May 2013 01:54:45 GMT jmt12 [27409] * gs2-extensions/parallel-building/trunk/src/bin/script/test_fileutils.pl (added) Unit test like testing for the FileUtils class and LocalFS, ... Thu, 23 May 2013 01:54:09 GMT jmt12 [27408] * gs2-extensions/parallel-building/trunk/src/bin/script/start_thrift_server.sh (added) A symbolic link to the actual script in the packages directory Thu, 23 May 2013 01:50:17 GMT jmt12 [27407] * gs2-extensions/parallel-building/trunk/src/packages (modified) updating ignore Thu, 23 May 2013 01:49:44 GMT jmt12 [27406] * gs2-extensions/parallel-building/trunk/src/packages/.svnignore (modified) Updating ignore Thu, 23 May 2013 01:48:10 GMT jmt12 [27405] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/IPCRUN.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/PROCDAEMON.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/SORTKEY.sh (modified) Now that I can get the Perl version, make the cascade make script ... Thu, 23 May 2013 01:45:36 GMT jmt12 [27404] * gs2-extensions/parallel-building/trunk/src/packages/Bit-Vector-7.2.tar.gz (added) * gs2-extensions/parallel-building/trunk/src/packages/ThriftFS-0.9.0.tar.gz (added) Thrift access to HDFS and required Bit-Vector Perl module Tue, 21 May 2013 03:38:04 GMT jmt12 [27386] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDFSShell.pm (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/LocalFS.pm (added) Forgot these were just symbolic links to my Dropbox folder - adding ... Tue, 21 May 2013 03:37:43 GMT jmt12 [27385] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDFSShell.pm (deleted) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (deleted) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/LocalFS.pm (deleted) removing soft links to my dropbox folder - sorry Tue, 21 May 2013 03:32:30 GMT jmt12 [27384] * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils.pm (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDFSShell.pm (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS.pm (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/HDThriftFS/ThriftFH.pm (added) * gs2-extensions/parallel-building/trunk/src/perllib/FileUtils/LocalFS.pm (added) An overriding version of FileUtils that provides several drivers ... Tue, 21 May 2013 03:30:45 GMT jmt12 [27383] * gs2-extensions/parallel-building/trunk/src/perllib/MIME (added) * gs2-extensions/parallel-building/trunk/src/perllib/MIME/Base91.pm (added) Adding a perl module for the slightly more efficient Base91 encoding ... Tue, 21 May 2013 03:17:39 GMT jmt12 [27382] * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE.sh (modified) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/BITVECTOR.sh (added) * gs2-extensions/parallel-building/trunk/src/packages/CASCADE-MAKE/THRIFT.sh (added) Adding Thrift and its prerequisite BitVector packages