source: gs2-extensions/parallel-building

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @27051   11 years jmt12 Trying to force Sort::Key to install to a sensible location on Medusa …
(edit) @27048   11 years jmt12 Ignoring untarred Hadoop and Sort::Key directories
(edit) @27047   11 years jmt12 Script to allow the Perl module Sort::Key to be compiled/installed as …
(edit) @27046   11 years jmt12 Adding in Sort::Key cascade makefile
(edit) @27045   11 years jmt12 Ignoring untarred Hadoop and Sort::Key directories
(edit) @27044   11 years jmt12 The Hadoop reporting stuff needs natural sorting
(edit) @27043   11 years jmt12 Sort appears to be platform specific, so I'll need to include it as a …
(edit) @27042   11 years jmt12 Sort appears to be platform specific, so I'll need to include it as a …
(edit) @27041   11 years jmt12 INC path now includes the installed extensions perl path (including …
(edit) @27040   11 years jmt12 A simple script that returns just the version number of Perl
(edit) @27039   11 years jmt12 Adding natural sort for Perl. Shared object files appear to be ignored …
(edit) @27038   11 years jmt12 Allow natural sorting in Perl
(edit) @27037   11 years jmt12 Allow natural sorting in Perl
(edit) @27036   11 years jmt12 A script to extract data locality and other task information from the …
(edit) @27006   11 years jmt12 A companion script to stop-hadoop-processes that just reports running …
(edit) @27005   11 years jmt12 Similar to stop-impt.pl, this script uses kill to stop runaway Hadoop …
(edit) @27004   11 years jmt12 A script to stop (using kill) a runaway import process and any related …
(edit) @27003   11 years jmt12 Adding some Hadoopy stuff to setup.bash - will need to be configured …
(edit) @27002   11 years jmt12 Fixing bug in maxdocs processing where I compared to 'file_count' …
(edit) @27001   11 years jmt12 Passing more environment variables (HADOOPPREFIX, HDFSHOST, HDFSPORT) …
(edit) @26999   11 years jmt12 Ensuring MPI binds to correct interface, and passing through …
(edit) @26998   11 years jmt12 Adding maxdocs variable, lots of debug comments, added some tests for …
(edit) @26987   11 years jmt12 Removing some debug comments I was using to track down why TDB support …
(edit) @26986   11 years jmt12 Commenting out some debug comments
(edit) @26985   11 years jmt12 Removing a couple of debuggy type flags to MPI just to make logs a …
(edit) @26984   11 years jmt12 Removing some debug comments
(edit) @26983   11 years jmt12 Changed the prefix in a log filename to a 'W' for worker rather than a …
(edit) @26982   11 years jmt12 Altered environment setup for subprocesses (due to Hadoop now moving …
(edit) @26981   11 years jmt12 ignoring compiled java classes (hgi_classes)
(edit) @26980   11 years jmt12 setting svnignore
(edit) @26979   11 years jmt12 Fixed up lingering environment variables from when I was going to make …
(edit) @26965   11 years jmt12 Makes use of util library functions (rather than perl built-in file …
(edit) @26964   11 years jmt12 Altered to use util functions rather than built-in perl file tests. …
(edit) @26963   11 years jmt12 Altered to use functions in util library when opening file handles to …
(edit) @26962   11 years jmt12 Altering to use functions in util library so as to support HDFS …
(edit) @26961   11 years jmt12 Several file handling functions extended to be aware of and support …
(edit) @26960   11 years jmt12 Allow plugouts from be loaded from within extensions (assuming no …
(edit) @26959   11 years jmt12 Adding the ability to include a version attribute in manifest files to …
(edit) @26958   11 years jmt12 Using the util library instead of built-in perl file tests
(edit) @26954   11 years jmt12 checking in Java code from Hadoop parallel processing projects... phew
(edit) @26953   11 years jmt12 Checking in the script rather than a symbolic link to the script :P
(edit) @26952   11 years jmt12 Accidentally checked in symbolic link rather than script
(edit) @26951   11 years jmt12 Adding in required enviroment details for Hadoop
(edit) @26950   11 years jmt12 The version of Hadoop on which Hadoop parallel processing work is …
(edit) @26949   11 years jmt12 Parallel import using Hadoop
(edit) @26947   11 years jmt12 empty directory removed
(edit) @26946   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26945   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26944   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26943   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26942   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26941   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26940   11 years jmt12 Identical to those in default perllib/plugins - should never have been …
(edit) @26939   11 years jmt12 Replacing uses of TextFile with TextPlugin... don't know how the …
(edit) @26938   11 years jmt12 Identical to those in default perllib - should never have been checked-in
(edit) @26937   11 years jmt12 Identical to files in default perllib - should never have been checked-in
(edit) @26935   11 years jmt12 Identical to those in default perllib - should never have checked in
(edit) @26934   11 years jmt12 Renamed to gdbml to avoid confusion with non-locking version of GDBM …
(edit) @26933   11 years jmt12 Renamed to gdbml to avoid confusion with non-locking version of GDBM …
(edit) @26932   11 years jmt12 Altered all calls to built-in perl file tests to instead use util …
(edit) @26930   11 years jmt12 Randomized order of files, and added the ability to specify a maximum …
(edit) @26929   11 years jmt12 A script to comprehensively clean up a collection between imports... …
(edit) @26928   11 years jmt12 Ignoring the compiled executable
(edit) @26926   11 years jmt12 Adding the executable file to the ignore set
(edit) @26923   11 years jmt12 Generates a specficied-size subset of a larger import directory
(edit) @26922   11 years jmt12 [bugfix] missing double quote in call to greenstone import.pl
(edit) @26242   12 years jmt12 Modifications to progress messages to improve extracting information …
(edit) @26240   12 years jmt12 Modifications to progress messages to improve extracting information …
(edit) @26239   12 years jmt12 Modifications to progress messages to improve extracting information …
(edit) @26237   12 years jmt12 Allows the specification of a maxfiles - limiting the number of files …
(edit) @26194   12 years jmt12 Fixed bug where I wasn't reseting the number of files handled when …
(edit) @26188   12 years jmt12 Fixing bug where I passed the wrong number of arguments to a sprintf …
(edit) @26187   12 years jmt12 Adding the rest of parallel processing support for Terrier into SVN. …
(edit) @26072   12 years jmt12 Extended to notice and load mpi.conf file on clusters and to use full …
(edit) @26071   12 years jmt12 Randomizing the order files are imported and restoring debugging flags …
(edit) @26070   12 years jmt12 ignoring files using svnignore
(edit) @26069   12 years jmt12 Initial checkin of Terrier parallel processing code
(edit) @26068   12 years jmt12 cascade make files for the new terrier parallel processing executable
(edit) @25945   12 years jmt12 more debugging and a few tweaks - and between this and changes to the …
(edit) @25944   12 years jmt12 More debug comments, fixed memory leak - still hanging unfortunately
(edit) @25943   12 years jmt12 Updated script to support cluster processing
(edit) @25841   12 years jmt12 Renaming log file to make it easier to delete programatically
(edit) @25839   12 years jmt12 Adding more timing to threads
(edit) @25814   12 years jmt12 Added extra test to prevent warning when block_hash's manifest …
(edit) @25813   12 years jmt12 Adding MPI DSpace media filter to the cascade makey stuff
(edit) @25812   12 years jmt12 Adding conditional blocks so that packages are only rebuilt if they …
(edit) @25811   12 years jmt12 Adding svnignore properties
(edit) @25810   12 years jmt12 MPI executable to provide parallel processing functionality to DSpace …
(edit) @25809   12 years jmt12 Script to parallel media filter DSpace
(edit) @25416   12 years jmt12 Actually make the client sleep the retry time it says it will
(edit) @25401   12 years jmt12 Extend the code that 'pre-started' the GDBMServer to pre-start any …
(edit) @25120   12 years jmt12 Shorten the directory names of folders in archives to avoid the …
(edit) @25118   12 years jmt12 Significant changes to support the latest round of batch tests (for …
(edit) @25117   12 years jmt12 Added some debug comments
(edit) @25116   12 years jmt12 (Temporarily) removing the global file scan and the tmp file deletion
(edit) @25115   12 years jmt12 This is sometimes called without the site argument... detect and match …
(edit) @25114   12 years jmt12 Some more tests as it appears there are ways to run import such as …
(edit) @25113   12 years jmt12 A TextPlugin extended to do a little more processing in order to …
(edit) @24848   12 years jmt12 Extended the manifestinator to correctly recurse into directories
(edit) @24847   12 years jmt12 Added the ability for the server to detect if it's calling process has …
Note: See TracRevisionLog for help on using the revision log.