|
|
@27914
|
11 years |
jmt12 |
Trying to get around a couple of divide-by-zero issues when generating …
|
|
|
@27913
|
11 years |
jmt12 |
Made the ingester to be used (version 1 without reduce phase, or …
|
|
|
@27753
|
11 years |
jmt12 |
Adding Handbrake's percentage complete to report - although this is …
|
|
|
@27752
|
11 years |
jmt12 |
Data locality file not being found is no longer fatal (HDFS-NFS-Proxy …
|
|
|
@27732
|
11 years |
jmt12 |
Nice the copy itself too
|
|
|
@27686
|
11 years |
jmt12 |
A little more progress comments
|
|
|
@27685
|
11 years |
jmt12 |
in the case of multiple attempts you need to retain the information …
|
|
|
@27684
|
11 years |
jmt12 |
Adding natural sorting into report generation - so also needed to add …
|
|
|
@27683
|
11 years |
jmt12 |
moving a few more headings around to help with information block layout
|
|
|
@27669
|
11 years |
jmt12 |
Sort compute nodes naturally before labelling them with incremental …
|
|
|
@27654
|
11 years |
jmt12 |
Add the ability to stagger the starting of Mappers by placing a …
|
|
|
@27644
|
11 years |
jmt12 |
Extended to support HDFS-access via NFS. This applies to both the call …
|
|
|
@27643
|
11 years |
jmt12 |
Changed the script generator so it can recurse through directories and …
|
|
|
@27642
|
11 years |
jmt12 |
A script I downloaded that successfully splits video files - something …
|
|
|
@27594
|
11 years |
jmt12 |
Extend hadoop_import.pl to be able to start and stop the Thrift server(s)
|
|
|
@27590
|
11 years |
jmt12 |
Adding statistics about data locality, and highlighting tasks where …
|
|
|
@27589
|
11 years |
jmt12 |
Fixing up some minor bugs in regex's
|
|
|
@27588
|
11 years |
jmt12 |
Extend parser to support jobs that are split over several logs. Also …
|
|
|
@27587
|
11 years |
jmt12 |
Allow debug mode to be enabled from the command line
|
|
|
@27586
|
11 years |
jmt12 |
Updating script to date date of hadoop job into account when searching …
|
|
|
@27585
|
11 years |
jmt12 |
The perl on Medusa won't let you immediately treat a returned array in …
|
|
|
@27584
|
11 years |
jmt12 |
I wasn't doing -r when attempting to clear directories left in /tmp by …
|
|
|
@27583
|
11 years |
jmt12 |
Adding code to differentiate between workers in a cluster - all of …
|
|
|
@27560
|
11 years |
jmt12 |
Fixing typo in regexp that meant filenames sometimes ignored
|
|
|
@27559
|
11 years |
jmt12 |
Changed mime-type away from binary - I hope. Meanwhile, generate …
|
|
|
@27551
|
11 years |
jmt12 |
Altered so that it expects to be given a CSV containing parallel …
|
|
|
@27550
|
11 years |
jmt12 |
Ensure the hostname is added to the Hadoop logs so we can identify the …
|
|
|
@27549
|
11 years |
jmt12 |
Extract information from the logs generated by parallel Greenstone …
|
|
|
@27548
|
11 years |
jmt12 |
Extract information from the logs generated by parallel Greenstone …
|
|
|
@27543
|
11 years |
jmt12 |
Adding generate_gantt.pl script in its original form - i.e. directly …
|
|
|
@27530
|
11 years |
jmt12 |
Clear out old logs, and adding more comments about what the script is …
|
|
|
@27515
|
11 years |
jmt12 |
Making the file used durig buffertes be configurable
|
|
|
@27512
|
11 years |
jmt12 |
Adding in a special test for measuring the effect of altering ThriftFS …
|
|
|
@27495
|
11 years |
jmt12 |
removing doubled up debug comments and putting some paths in …
|
|
|
@27481
|
11 years |
jmt12 |
Adding makeAllDirectories() (which I'd only implemented in LocalFS) to …
|
|
|
@27480
|
11 years |
jmt12 |
Removing DateTime dependency (so HDFSShell will always fail …
|
|
|
@27436
|
11 years |
jmt12 |
Adding the actual script - rather than a symlink to my dropbox. doh
|
|
|
@27435
|
11 years |
jmt12 |
Gah - only a symbolic link
|
|
|
@27414
|
11 years |
jmt12 |
Allowing more processing arguments to be configured at the call, and …
|
|
|
@27412
|
11 years |
jmt12 |
I obviously hadn't run this script on Karearea before - assumed all …
|
|
|
@27409
|
11 years |
jmt12 |
Unit test like testing for the FileUtils class and LocalFS, HDFSShell, …
|
|
|
@27408
|
11 years |
jmt12 |
A symbolic link to the actual script in the packages directory
|
|
|
@27378
|
11 years |
jmt12 |
Parallel processing support now added (via buildcolutil subclass) to …
|
|
|
@27126
|
11 years |
jmt12 |
Extra clean up commands (like removing cached versions of video …
|
|
|
@27125
|
11 years |
jmt12 |
A script to try and flush all caches - I'm certain it's flushing disk …
|
|
|
@27124
|
11 years |
jmt12 |
Use the new perl version script to extract the version number - so as …
|
|
|
@27119
|
11 years |
jmt12 |
Merging version finder from Medusa with the one lurking on Karearea
|
|
|
@27058
|
11 years |
jmt12 |
Adding data locality report generation to Hadoop greenstone imports
|
|
|
@27052
|
11 years |
jmt12 |
Turns out the Perl on Medusa doesn't support $V, so I've had to …
|
|
|
@27041
|
11 years |
jmt12 |
INC path now includes the installed extensions perl path (including …
|
|
|
@27040
|
11 years |
jmt12 |
A simple script that returns just the version number of Perl
|
|
|
@27036
|
11 years |
jmt12 |
A script to extract data locality and other task information from the …
|
|
|
@27006
|
11 years |
jmt12 |
A companion script to stop-hadoop-processes that just reports running …
|
|
|
@27005
|
11 years |
jmt12 |
Similar to stop-impt.pl, this script uses kill to stop runaway Hadoop …
|
|
|
@27004
|
11 years |
jmt12 |
A script to stop (using kill) a runaway import process and any related …
|
|
|
@27001
|
11 years |
jmt12 |
Passing more environment variables (HADOOPPREFIX, HDFSHOST, HDFSPORT) …
|
|
|
@26999
|
11 years |
jmt12 |
Ensuring MPI binds to correct interface, and passing through …
|
|
|
@26998
|
11 years |
jmt12 |
Adding maxdocs variable, lots of debug comments, added some tests for …
|
|
|
@26953
|
11 years |
jmt12 |
Checking in the script rather than a symbolic link to the script :P
|
|
|
@26952
|
11 years |
jmt12 |
Accidentally checked in symbolic link rather than script
|
|
|
@26949
|
11 years |
jmt12 |
Parallel import using Hadoop
|
|
|
@26930
|
11 years |
jmt12 |
Randomized order of files, and added the ability to specify a maximum …
|
|
|
@26929
|
11 years |
jmt12 |
A script to comprehensively clean up a collection between imports... …
|
|
|
@26923
|
11 years |
jmt12 |
Generates a specficied-size subset of a larger import directory
|
|
|
@26242
|
12 years |
jmt12 |
Modifications to progress messages to improve extracting information …
|
|
|
@26187
|
12 years |
jmt12 |
Adding the rest of parallel processing support for Terrier into SVN. …
|
|
|
@25943
|
12 years |
jmt12 |
Updated script to support cluster processing
|
|
|
@25809
|
12 years |
jmt12 |
Script to parallel media filter DSpace
|
|
|
@25118
|
12 years |
jmt12 |
Significant changes to support the latest round of batch tests (for …
|
|
|
@25117
|
12 years |
jmt12 |
Added some debug comments
|
|
|
@24848
|
12 years |
jmt12 |
Extended the manifestinator to correctly recurse into directories
|
|
|
@24847
|
12 years |
jmt12 |
Added the ability for the server to detect if it's calling process has …
|
|
|
@24846
|
12 years |
jmt12 |
A small SocketSwimmingThreadPoolServer to allow testing of the client …
|
|
|
@24845
|
12 years |
jmt12 |
Similar to poll-gsdl.pl except that this uses mpstat to gather …
|
|
|
@24844
|
12 years |
jmt12 |
Script to take the raw output from poll-gsdl.pl and collate the …
|
|
|
@24843
|
12 years |
jmt12 |
A script that launches an (affinity assigned) Greenstone import while …
|
|
|
@24842
|
12 years |
jmt12 |
Script to take the raw output from poll-processor.pl and collate the …
|
|
|
@24841
|
12 years |
jmt12 |
|
|
|
@24695
|
13 years |
jmt12 |
Script to run a battery of tests and record the results in a database
|
|
|
@24694
|
13 years |
jmt12 |
Setup script to generate a number of collections or varying sizes …
|
|
|
@24693
|
13 years |
jmt12 |
A hack to allow sudoedit to modify the drop_caches value non-interactively
|
|
|
@24680
|
13 years |
jmt12 |
Removed sanity check for GDBM (where was my sanity - GDBM works, it's …
|
|
|
@24679
|
13 years |
jmt12 |
Added code to determine the open file handles and persist them through …
|
|
|
@24678
|
13 years |
jmt12 |
A tool for comparing the contents of two GDBM databases
|
|
|
@24677
|
13 years |
jmt12 |
A tool for generating manifest files from import directories
|
|
|
@24667
|
13 years |
jmt12 |
Adding another sanity test to prevent parallel building when infodb is …
|
|
|
@24666
|
13 years |
jmt12 |
A daemonizable server that modifies a GDBM database (via the GDBMCLI …
|
|
|
@24665
|
13 years |
jmt12 |
A simple client for testing GDBMServer and for properly shutting it …
|
|
|
@24622
|
13 years |
jmt12 |
Parallel enabled version of buildcol.pl and the parallel_import.pl …
|