|
|
@30953
|
7 years |
davidb |
Need to specify _output_dir as part of output JSON filename
|
|
|
@30952
|
7 years |
davidb |
Further text tidy up
|
|
|
@30951
|
7 years |
davidb |
Save a JSONObject as a file in the output directory
|
|
|
@30950
|
7 years |
davidb |
Tweak to text
|
|
|
@30949
|
7 years |
davidb |
Use better name than 'foo'. Further fix to JSON name generated
|
|
|
@30947
|
7 years |
davidb |
Correction to 'pages-' part of JSON.bz2 output filename used
|
|
|
@30946
|
7 years |
davidb |
Correction to output JSON.bz2 name generated
|
|
|
@30945
|
7 years |
davidb |
Getting closer to writing out JSON files
|
|
|
@30944
|
7 years |
davidb |
Forcer higher partition (6) than default, which seems to be 2
|
|
|
@30943
|
7 years |
davidb |
Extra debug info
|
|
|
@30942
|
7 years |
davidb |
Improved output printing for slave node
|
|
|
@30941
|
7 years |
davidb |
Moved to getFileSystemInstance() method to play nice on cluster
|
|
|
@30940
|
7 years |
davidb |
Change to using URI not fileIn directly
|
|
|
@30939
|
7 years |
davidb |
Minor tweaks
|
|
|
@30938
|
7 years |
davidb |
Experiment with using Hadoop's FileSystem class for local file:// access
|
|
|
@30937
|
7 years |
davidb |
Expanded set of ClusterFileIO methods
|
|
|
@30936
|
7 years |
davidb |
Refinement of Spark Monitor echo statements
|
|
|
@30935
|
7 years |
davidb |
Fixed variable name typo, plus added a couple of 'sleep' pauses of 1 sec
|
|
|
@30934
|
7 years |
davidb |
Providing json-filelist now a compulsory argument, rather than an option
|
|
|
@30933
|
7 years |
davidb |
More careful parsing of file prefix
|
|
|
@30932
|
7 years |
davidb |
Support both file:// and hdfs://
|
|
|
@30931
|
7 years |
davidb |
Version that runs using fil:// tested
|
|
|
@30930
|
7 years |
davidb |
Expansion of useful alias commands for Hadoop and Spark
|
|
|
@30929
|
7 years |
davidb |
Tweaks made while testing the script
|
|
|
@30928
|
7 years |
davidb |
Forgot to set json_filelist
|
|
|
@30927
|
7 years |
davidb |
Fixed silly typo in stdout redirect
|
|
|
@30926
|
7 years |
davidb |
Restructuring of RUN scripts to be more flexible
|
|
|
@30925
|
7 years |
davidb |
Improved instrutions
|
|
|
@30924
|
7 years |
davidb |
Tidy up of code. Removed commented out code
|
|
|
@30923
|
7 years |
davidb |
Rough cut version that reads in each JSON file over HDFS
|
|
|
@30922
|
7 years |
davidb |
Additional rough-cut notes
|
|
|
@30921
|
7 years |
davidb |
Code change to read in JSON file over HDFS
|
|
|
@30919
|
8 years |
davidb |
More consistent naming of folders used
|
|
|
@30918
|
8 years |
davidb |
More flexible command-line args
|
|
|
@30917
|
8 years |
davidb |
Changes resulting from a fresh run at provisioning, which yielded the …
|
|
|
@30916
|
8 years |
davidb |
Some additional details -- note form
|
|
|
@30915
|
8 years |
davidb |
Initial cut at instructions to follow to get code set up and running
|
|
|
@30914
|
8 years |
davidb |
Tidy up of setup description
|
|
|
@30913
|
8 years |
davidb |
Renaming to better represent what the cluster is designed for
|
|
|
@30912
|
8 years |
davidb |
Changed to Unix style line-endings
|
|
|
@30911
|
8 years |
davidb |
Changed name of input directory
|
|
|
@30910
|
8 years |
davidb |
Additional finesse added in as a result of further testing on Vagrant …
|
|
|
@30909
|
8 years |
davidb |
Additional finesse added in as a result of further testing on Vagrant …
|
|
|
@30908
|
8 years |
davidb |
Additional finesse added in as a result of further testing on Vagrant …
|
|
|
@30907
|
8 years |
davidb |
Name change to reflect need for 'bash' not 'sh'
|
|
|
@30906
|
8 years |
davidb |
Bash version of BAT script
|
|
|
@30905
|
8 years |
davidb |
Additional resources
|
|
|
@30904
|
8 years |
davidb |
Extra resource/links added
|
|
|
@30903
|
8 years |
davidb |
Vagrant provisioning files for a 4-node Hadoop cluster. See …
|
|
|
@30902
|
8 years |
davidb |
Details of what packages are needed
|
|
|
@30901
|
8 years |
davidb |
Template setup file
|
|
|
@30900
|
8 years |
davidb |
For support Java packages
|
|
|
@30899
|
8 years |
davidb |
Files for compilation using Eclipse
|
|
|
@30898
|
8 years |
davidb |
Scripts for downloading sample JSON data from public domain extracted …
|
|
|
@30897
|
8 years |
davidb |
Sub-project for converted HTRC Extract Feature dataset into a form …
|
|
|
@30890
|
8 years |
davidb |
folder to group together hathitrust related projects
|
|
|
@30846
|
8 years |
ak19 |
Wrong module in script name.
|
|
|
@30818
|
8 years |
ak19 |
Script needs to get rid of another intermediate file.
|
|
|
@30722
|
8 years |
ak19 |
Remove repeated empty lines, leaving just a single blank line between …
|
|
|
@30720
|
8 years |
ak19 |
Getting the nightly gti email to be sent again on the new machine …
|
|
|
@30652
|
8 years |
ak19 |
Committing outstanding files for diffcol supporting jdb for GS3 …
|
|
|
@30613
|
8 years |
ak19 |
Don't send nightly email messages about updates to the test language …
|
|
|
@30611
|
8 years |
ak19 |
Modified version of remove_extra_lines script to handle the Updated …
|
|
|
@30605
|
8 years |
ak19 |
Committing the changes necessary to get the GTI crons to work on the …
|
|
|
@30594
|
8 years |
kjdon |
modifying the input and results areas to get it looking the same in …
|
|
|
@30590
|
8 years |
kjdon |
this change was made on puka, and when I tried this on commdev, it …
|
|
|
@30581
|
8 years |
ak19 |
GTI related changes to add gs3 demo collection config files' …
|
|
|
@30425
|
8 years |
davidb |
Save metadata as JSON file. Create sub-directories to spreadout the …
|
|
|
@30424
|
8 years |
ak19 |
Dr Bainbridge improved the code so that the script always gets the …
|
|
|
@30422
|
8 years |
davidb |
Tidier treatment of 'bin' and 'audio' directories from an SVN point of view
|
|
|
@30421
|
8 years |
davidb |
Removed from SVN tree
|
|
|
@30420
|
8 years |
davidb |
svn:ignore 'audio' and 'bin'
|
|
|
@30419
|
8 years |
davidb |
Support for JSON added
|
|
|
@30418
|
8 years |
davidb |
Code updated to work through a sequence of pages for one artist
|
|
|
@30417
|
8 years |
davidb |
Switched order of stdout and stderr redirects
|
|
|
@30416
|
8 years |
davidb |
Initial cut at code for scraping music excerpts from AMC site
|
|
|
@30415
|
8 years |
davidb |
Main trunk.
|
|
|
@30414
|
8 years |
davidb |
Top-level folder for Western Sydney University led Music Affect …
|
|
|
@30410
|
8 years |
ak19 |
Finally got the upload of the binary to turn up.
|
|
|
@30408
|
8 years |
ak19 |
Fix (from Puka) for the upload destination machine to work out the …
|
|
|
@30407
|
8 years |
ak19 |
Script for clearing old binaries and logs from the machine where they …
|
|
|
@30406
|
8 years |
ak19 |
Minor change. rke log for expeditee should have the os suffix on all OS.
|
|
|
@30405
|
8 years |
ak19 |
Swap month and day in date string for generated nightly binary, to get …
|
|
|
@30404
|
8 years |
ak19 |
Cosmetic change: more helpful comment for configuring rke-setup on mac.
|
|
|
@30399
|
8 years |
ak19 |
Minor changes to mirror the way the expeditee nightly generation …
|
|
|
@30398
|
8 years |
ak19 |
Parallel changes for the linux machines to use variables in …
|
|
|
@30397
|
8 years |
ak19 |
Fixed values in environment setup file, and used as variables in …
|
|
|
@30396
|
8 years |
ak19 |
Uploading expeditee nightly on linux 32 bit works.
|
|
|
@30395
|
8 years |
ak19 |
Got uploading to work at last.
|
|
|
@30393
|
8 years |
ak19 |
Bat file also needs to delete stale expeditee snapshot before …
|
|
|
@30392
|
8 years |
ak19 |
Deleting expeditee snapshot before regenerating.
|
|
|
@30391
|
8 years |
sjm84 |
Additional setup.sh.in file for linux-lsb VM environments. To be used …
|
|
|
@30390
|
8 years |
sjm84 |
New expeditee SVN repository URL.
|
|
|
@30387
|
8 years |
ak19 |
First step in creating nightly scripts for generating expeditee …
|
|
|
@30137
|
9 years |
jmt12 |
Obsolete - merged with documentation/wiki
|
|
|
@30136
|
9 years |
jmt12 |
Merged with Greenstone Dokuwiki files in documentation/wiki - …
|
|
|
@30126
|
9 years |
davidb |
Some svn:ignore values to cover oof VS automatically generated folders
|
|
|
@30125
|
9 years |
davidb |
VS seems to want this icon in the top level
|
|
|
@30124
|
9 years |
davidb |
branded icon for project
|
|
|
@30123
|
9 years |
davidb |
Refactoring to use better playing-in-the-street (PITS) 'branding'
|
|
|