source: gs2-extensions/parallel-building/trunk/src/src

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @30354   8 years jmt12 Extending manifest v2 support to allow for directories to be listed in …
(edit) @30301   9 years jmt12 Missed displaying one variable in a fprintf statement (threw a …
(edit) @30300   9 years jmt12 Bunch of changes (including whitespace safety ones) due to different …
(edit) @30299   9 years jmt12 Making the removal of the jar file conditional on it actually being …
(edit) @30298   9 years jmt12 Correcting path to manifest files for use in Greenstone3. Started to …
(edit) @30297   9 years jmt12 Altering the Makefile.in to determine whether it is in GSDL2 or GSDL3 …
(edit) @30294   9 years jmt12 Typo would have prevented the generated configure script from working …
(edit) @30293   9 years jmt12 The missing script was, ironically, missing
(edit) @30291   9 years jmt12 Minor changes in generated configs mostly to do with whitespace safety
(edit) @30290   9 years jmt12 Extended build script to make Hadoop support optional. If no …
(edit) @28312   11 years jmt12 Working on finer control over data locality - so I can configure a run …
(edit) @28192   11 years jmt12 Need to still output Greenstone messages to log otherwise I can't …
(edit) @28012   11 years jmt12 Express start time as a double as well
(edit) @28011   11 years jmt12 Turn off debugging in the copy in SVN
(edit) @28010   11 years jmt12 Correctly set up the environment for calls to txt2tdb and also replace …
(edit) @27912   11 years jmt12 Modified the compilation to include the new ingester and its co-requisites.
(edit) @27911   11 years jmt12 Modified the compilation to include the new ingester and its co-requisites
(edit) @27910   11 years jmt12 Extended the existing HadoopGreenstoneIngest with proper Reduce phase …
(edit) @27654   11 years jmt12 Add the ability to stagger the starting of Mappers by placing a …
(edit) @27641   11 years jmt12 Altered order of arguments and allow archives dir to be passed as …
(edit) @27571   11 years jmt12 increase timeout to 4 hours per map
(edit) @27566   11 years jmt12 Making the getcpu optional - as it isn't available on Medusa (but then …
(edit) @27561   11 years jmt12 Adding very basic compile file for getcpu - can't be bothered going …
(edit) @27558   11 years jmt12 Forgot that Hadoop Map processes no longer have the environment …
(edit) @27546   11 years jmt12 Adding the ability for the Hadoop Mapper to determine what CPU number …
(edit) @27545   11 years jmt12 Ignoring just the compiled file (for now)
(edit) @27544   11 years jmt12 A tiny C script to guesstimate the CPU the calling Process is on
(edit) @27494   11 years jmt12 Fixing a truncated comment - or maybe I never wrote an end to it…
(edit) @27487   11 years jmt12 Ensure Parallel Building path in environment (for ThriftFS) and that …
(edit) @27433   11 years jmt12 Adding a few more details to the Hadoop log to assist with later work
(edit) @27418   11 years jmt12 Also recompile the HadoopGreenstoneIngest.java file during cascade make
(edit) @27417   11 years jmt12 A 'pseudo'-CASCASE-MAKE file for building java code. It doesn't really …
(edit) @27416   11 years jmt12 Forgot to ensure the manifests generated here are also version 2
(edit) @27415   11 years jmt12 Ensuring the manifests generated are marked as version 2 - i.e. should …
(edit) @27413   11 years jmt12 Making the Hadoop processing aware that there are two different …
(edit) @27179   11 years davidb Mods to allow code to run with Greenstone3
(edit) @27102   11 years jmt12 Adding in a progress reporter, as the set timeout to 10 hours trick …
(edit) @26983   11 years jmt12 Changed the prefix in a log filename to a 'W' for worker rather than a …
(edit) @26982   11 years jmt12 Altered environment setup for subprocesses (due to Hadoop now moving …
(edit) @26981   11 years jmt12 ignoring compiled java classes (hgi_classes)
(edit) @26980   11 years jmt12 setting svnignore
(edit) @26979   11 years jmt12 Fixed up lingering environment variables from when I was going to make …
(edit) @26954   11 years jmt12 checking in Java code from Hadoop parallel processing projects... phew
(edit) @26928   11 years jmt12 Ignoring the compiled executable
(edit) @26926   11 years jmt12 Adding the executable file to the ignore set
(edit) @26922   11 years jmt12 [bugfix] missing double quote in call to greenstone import.pl
(edit) @26239   12 years jmt12 Modifications to progress messages to improve extracting information …
(edit) @26188   12 years jmt12 Fixing bug where I passed the wrong number of arguments to a sprintf …
(edit) @26070   12 years jmt12 ignoring files using svnignore
(edit) @26069   12 years jmt12 Initial checkin of Terrier parallel processing code
(edit) @26068   12 years jmt12 cascade make files for the new terrier parallel processing executable
(edit) @25945   12 years jmt12 more debugging and a few tweaks - and between this and changes to the …
(edit) @25944   12 years jmt12 More debug comments, fixed memory leak - still hanging unfortunately
(edit) @25841   12 years jmt12 Renaming log file to make it easier to delete programatically
(edit) @25839   12 years jmt12 Adding more timing to threads
(edit) @25813   12 years jmt12 Adding MPI DSpace media filter to the cascade makey stuff
(edit) @25811   12 years jmt12 Adding svnignore properties
(edit) @25810   12 years jmt12 MPI executable to provide parallel processing functionality to DSpace …
(edit) @24836   12 years jmt12 Altered lockfile generation so that it occurs in specific collections …
(edit) @24835   12 years jmt12 Altered lockfile generation so that it occurs in specific collections …
(edit) @24834   12 years jmt12 Typo - missed -I for this dir include path
(edit) @24833   12 years jmt12 Adding support for setting affinity of worker processes. Lots of debug …
(edit) @24702   13 years jmt12 apparently the missing script was missing... teehee
(edit) @24699   13 years jmt12 Write the manifest files to the collection's tmp directory so we can …
(edit) @24660   13 years jmt12 Cascade make script (and script adjustments) for GDBMCLI tool
(edit) @24659   13 years jmt12 Several small changes to CLI such as new commands
(edit) @24639   13 years jmt12 A command line interface for GDBM to allow a bidirectional pipe from …
(edit) @24625   13 years jmt12 Cascade building files for the locking version of GDBM tools
(edit) @24624   13 years jmt12 Rather than having the Greenstone output sent to some obscure log, use …
(edit) @24621   13 years jmt12 Adding the version of GDBM txt2db with simply file locking so as to …
(edit) @24620   13 years jmt12 Cascade building files for the locking version of GDBM tools
(edit) @24619   13 years jmt12 Ignoring configured and building files
(edit) @24618   13 years jmt12 Adding the version of GDBM txt2db with simply file locking so as to …
(edit) @24594   13 years jmt12 Ignoring certain files
(edit) @24593   13 years jmt12 Ignoring certain files
(add) @24589   13 years jmt12 Initial checkin
Note: See TracRevisionLog for help on using the revision log.