source: gs2-extensions/parallel-building/trunk

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @27408   11 years jmt12 A symbolic link to the actual script in the packages directory
(edit) @27407   11 years jmt12 updating ignore
(edit) @27406   11 years jmt12 Updating ignore
(edit) @27405   11 years jmt12 Now that I can get the Perl version, make the cascade make script …
(edit) @27404   11 years jmt12 Thrift access to HDFS and required Bit-Vector Perl module
(edit) @27386   11 years jmt12 Forgot these were just symbolic links to my Dropbox folder - adding in …
(edit) @27385   11 years jmt12 removing soft links to my dropbox folder - sorry
(edit) @27384   11 years jmt12 An overriding version of FileUtils that provides several drivers …
(edit) @27383   11 years jmt12 Adding a perl module for the slightly more efficient Base91 encoding …
(edit) @27382   11 years jmt12 Adding Thrift and its prerequisite BitVector packages
(edit) @27381   11 years jmt12 An image used during FileUtils testing to check binary transfer in …
(edit) @27378   11 years jmt12 Parallel processing support now added (via buildcolutil subclass) to …
(edit) @27377   11 years jmt12 Updating calls to intermediate util functions to the new FileUtils …
(edit) @27373   11 years jmt12 No longer needs a custom util.pm - file utility functions now moved …
(edit) @27372   11 years jmt12 Fixing where I'd used uppercase letters in a package name that didn't …
(edit) @27371   11 years jmt12 Replacing with a proper class inheriting from gdbm and just overriding …
(edit) @27370   11 years jmt12 fixing minor bug in inheritence tree
(edit) @27280   11 years jmt12 A subclass of buildcolutils.pm with added functionality for parallel …
(edit) @27279   11 years jmt12 A subclass of inexport.pm with added functionality for parallel …
(edit) @27278   11 years jmt12 Renamed so it can be found in an automated fashion (prefixed by the …
(edit) @27237   11 years jmt12 Moved several generally applicable changes (manifest version, …
(edit) @27236   11 years jmt12 extended loader (to support extensions) and conditional additions to …
(edit) @27235   11 years jmt12 Moving the conditional additions to INC into the main trunk's version
(edit) @27234   11 years jmt12 renamed prepare_build_recipe() to the more meaningful …
(edit) @27233   11 years jmt12 Recipe generation code moved into subclass of mgbuilder called …
(edit) @27232   11 years jmt12 Recipe generation code moved into subclass of mgppbuilder called …
(edit) @27231   11 years jmt12 Recipe generation code moved into subclass of mgppbuilder called …
(edit) @27230   11 years jmt12 Recipe generation code moved into subclass of mgbuilder called …
(edit) @27229   11 years jmt12 A subclass of basebuilder that includes an 'abstract' function for the …
(edit) @27228   11 years jmt12 A subclass of lucenebuilder that includes a function to generate a …
(edit) @27227   11 years jmt12 The indexlevel processing code has been moved into the main trunk's …
(edit) @27226   11 years jmt12 Moved the version property into the main trunk's version
(edit) @27216   11 years jmt12 renaming (away from Java/PHP conventions) to make my life easier when …
(edit) @27215   11 years jmt12 Moved the conditional INC and PATH addition code into the main trunk
(edit) @27214   11 years jmt12 Copied the two new strings (buildcol.indexname and …
(edit) @27213   11 years jmt12 Unchanged from version in main trunk
(edit) @27212   11 years jmt12 No difference from version in main trunk (apart from some debugging …
(edit) @27211   11 years jmt12 The parallel processing changes (pretty much just the placeholder …
(edit) @27210   11 years jmt12 DavidB already ported the few minor changes (complexmeta, indexname, …
(edit) @27209   11 years jmt12 No difference from version in trunk
(edit) @27208   11 years jmt12 The only change was to ensure that Greenstone's 'perllib' and …
(edit) @27207   11 years jmt12 Classify had been edited to do a better job of searching for …
(edit) @27206   11 years jmt12 The only real change in doc.pm was the replacement of a -e with a call …
(edit) @27205   11 years jmt12 Unchanged from base perllib version
(edit) @27204   11 years jmt12 Unchanged from base perllib version
(edit) @27203   11 years jmt12 SQLite support for Write Ahead Logging (offering basic parallel …
(edit) @27202   11 years jmt12 Unchanged from base perllib version
(edit) @27201   11 years jmt12 Should never have been checked in as I made no changes here - caused …
(edit) @27200   11 years jmt12 A subclass of basebuilder that provides a placeholder function for the …
(edit) @27195   11 years davidb Not changed for this extension
(edit) @27194   11 years davidb Not changed for this extension
(edit) @27193   11 years davidb Not changed for this extension
(edit) @27184   11 years davidb Removing 'downloaders' from parallel importing extension, as there are …
(edit) @27179   11 years davidb Mods to allow code to run with Greenstone3
(edit) @27177   11 years davidb Test for and create (if needed) the 'tmp' directory
(edit) @27126   11 years jmt12 Extra clean up commands (like removing cached versions of video …
(edit) @27125   11 years jmt12 A script to try and flush all caches - I'm certain it's flushing disk …
(edit) @27124   11 years jmt12 Use the new perl version script to extract the version number - so as …
(edit) @27119   11 years jmt12 Merging version finder from Medusa with the one lurking on Karearea
(edit) @27102   11 years jmt12 Adding in a progress reporter, as the set timeout to 10 hours trick …
(edit) @27058   11 years jmt12 Adding data locality report generation to Hadoop greenstone imports
(edit) @27052   11 years jmt12 Turns out the Perl on Medusa doesn't support $V, so I've had to …
(edit) @27051   11 years jmt12 Trying to force Sort::Key to install to a sensible location on Medusa …
(edit) @27048   11 years jmt12 Ignoring untarred Hadoop and Sort::Key directories
(edit) @27047   11 years jmt12 Script to allow the Perl module Sort::Key to be compiled/installed as …
(edit) @27046   11 years jmt12 Adding in Sort::Key cascade makefile
(edit) @27045   11 years jmt12 Ignoring untarred Hadoop and Sort::Key directories
(edit) @27044   11 years jmt12 The Hadoop reporting stuff needs natural sorting
(edit) @27043   11 years jmt12 Sort appears to be platform specific, so I'll need to include it as a …
(edit) @27042   11 years jmt12 Sort appears to be platform specific, so I'll need to include it as a …
(edit) @27041   11 years jmt12 INC path now includes the installed extensions perl path (including …
(edit) @27040   11 years jmt12 A simple script that returns just the version number of Perl
(edit) @27039   11 years jmt12 Adding natural sort for Perl. Shared object files appear to be ignored …
(edit) @27038   11 years jmt12 Allow natural sorting in Perl
(edit) @27037   11 years jmt12 Allow natural sorting in Perl
(edit) @27036   11 years jmt12 A script to extract data locality and other task information from the …
(edit) @27006   11 years jmt12 A companion script to stop-hadoop-processes that just reports running …
(edit) @27005   11 years jmt12 Similar to stop-impt.pl, this script uses kill to stop runaway Hadoop …
(edit) @27004   11 years jmt12 A script to stop (using kill) a runaway import process and any related …
(edit) @27003   11 years jmt12 Adding some Hadoopy stuff to setup.bash - will need to be configured …
(edit) @27002   11 years jmt12 Fixing bug in maxdocs processing where I compared to 'file_count' …
(edit) @27001   11 years jmt12 Passing more environment variables (HADOOPPREFIX, HDFSHOST, HDFSPORT) …
(edit) @26999   11 years jmt12 Ensuring MPI binds to correct interface, and passing through …
(edit) @26998   11 years jmt12 Adding maxdocs variable, lots of debug comments, added some tests for …
(edit) @26987   11 years jmt12 Removing some debug comments I was using to track down why TDB support …
(edit) @26986   11 years jmt12 Commenting out some debug comments
(edit) @26985   11 years jmt12 Removing a couple of debuggy type flags to MPI just to make logs a …
(edit) @26984   11 years jmt12 Removing some debug comments
(edit) @26983   11 years jmt12 Changed the prefix in a log filename to a 'W' for worker rather than a …
(edit) @26982   11 years jmt12 Altered environment setup for subprocesses (due to Hadoop now moving …
(edit) @26981   11 years jmt12 ignoring compiled java classes (hgi_classes)
(edit) @26980   11 years jmt12 setting svnignore
(edit) @26979   11 years jmt12 Fixed up lingering environment variables from when I was going to make …
(edit) @26965   11 years jmt12 Makes use of util library functions (rather than perl built-in file …
(edit) @26964   11 years jmt12 Altered to use util functions rather than built-in perl file tests. …
(edit) @26963   11 years jmt12 Altered to use functions in util library when opening file handles to …
(edit) @26962   11 years jmt12 Altering to use functions in util library so as to support HDFS …
(edit) @26961   11 years jmt12 Several file handling functions extended to be aware of and support …
(edit) @26960   11 years jmt12 Allow plugouts from be loaded from within extensions (assuming no …
(edit) @26959   11 years jmt12 Adding the ability to include a version attribute in manifest files to …
Note: See TracRevisionLog for help on using the revision log.