source: other-projects

Revision Log Mode:


Legend:

Added
Modified
Copied or renamed
Diff Rev Age Author Log Message
(edit) @30956   6 years davidb Initial commit of files for setting up with Vagrant a Solr cloud
(edit) @30953   6 years davidb Need to specify _output_dir as part of output JSON filename
(edit) @30952   6 years davidb Further text tidy up
(edit) @30951   6 years davidb Save a JSONObject as a file in the output directory
(edit) @30950   6 years davidb Tweak to text
(edit) @30949   6 years davidb Use better name than 'foo'. Further fix to JSON name generated
(edit) @30947   6 years davidb Correction to 'pages-' part of JSON.bz2 output filename used
(edit) @30946   6 years davidb Correction to output JSON.bz2 name generated
(edit) @30945   6 years davidb Getting closer to writing out JSON files
(edit) @30944   6 years davidb Forcer higher partition (6) than default, which seems to be 2
(edit) @30943   6 years davidb Extra debug info
(edit) @30942   6 years davidb Improved output printing for slave node
(edit) @30941   6 years davidb Moved to getFileSystemInstance() method to play nice on cluster
(edit) @30940   6 years davidb Change to using URI not fileIn directly
(edit) @30939   6 years davidb Minor tweaks
(edit) @30938   6 years davidb Experiment with using Hadoop's FileSystem class for local file:// access
(edit) @30937   6 years davidb Expanded set of ClusterFileIO methods
(edit) @30936   6 years davidb Refinement of Spark Monitor echo statements
(edit) @30935   6 years davidb Fixed variable name typo, plus added a couple of 'sleep' pauses of 1 sec
(edit) @30934   6 years davidb Providing json-filelist now a compulsory argument, rather than an option
(edit) @30933   6 years davidb More careful parsing of file prefix
(edit) @30932   6 years davidb Support both file:// and hdfs://
(edit) @30931   6 years davidb Version that runs using fil:// tested
(edit) @30930   6 years davidb Expansion of useful alias commands for Hadoop and Spark
(edit) @30929   6 years davidb Tweaks made while testing the script
(edit) @30928   6 years davidb Forgot to set json_filelist
(edit) @30927   6 years davidb Fixed silly typo in stdout redirect
(edit) @30926   6 years davidb Restructuring of RUN scripts to be more flexible
(edit) @30925   6 years davidb Improved instrutions
(edit) @30924   6 years davidb Tidy up of code. Removed commented out code
(edit) @30923   6 years davidb Rough cut version that reads in each JSON file over HDFS
(edit) @30922   6 years davidb Additional rough-cut notes
(edit) @30921   6 years davidb Code change to read in JSON file over HDFS
(edit) @30919   6 years davidb More consistent naming of folders used
(edit) @30918   6 years davidb More flexible command-line args
(edit) @30917   6 years davidb Changes resulting from a fresh run at provisioning, which yielded the …
(edit) @30916   6 years davidb Some additional details -- note form
(edit) @30915   6 years davidb Initial cut at instructions to follow to get code set up and running
(edit) @30914   6 years davidb Tidy up of setup description
(edit) @30913   6 years davidb Renaming to better represent what the cluster is designed for
(edit) @30912   6 years davidb Changed to Unix style line-endings
(edit) @30911   6 years davidb Changed name of input directory
(edit) @30910   6 years davidb Additional finesse added in as a result of further testing on Vagrant …
(edit) @30909   6 years davidb Additional finesse added in as a result of further testing on Vagrant …
(edit) @30908   6 years davidb Additional finesse added in as a result of further testing on Vagrant …
(edit) @30907   6 years davidb Name change to reflect need for 'bash' not 'sh'
(edit) @30906   6 years davidb Bash version of BAT script
(edit) @30905   6 years davidb Additional resources
(edit) @30904   6 years davidb Extra resource/links added
(edit) @30903   6 years davidb Vagrant provisioning files for a 4-node Hadoop cluster. See …
(edit) @30902   6 years davidb Details of what packages are needed
(edit) @30901   6 years davidb Template setup file
(edit) @30900   6 years davidb For support Java packages
(edit) @30899   6 years davidb Files for compilation using Eclipse
(edit) @30898   6 years davidb Scripts for downloading sample JSON data from public domain extracted …
(edit) @30897   6 years davidb Sub-project for converted HTRC Extract Feature dataset into a form …
(edit) @30890   6 years davidb folder to group together hathitrust related projects
(edit) @30846   6 years ak19 Wrong module in script name.
(edit) @30818   6 years ak19 Script needs to get rid of another intermediate file.
(edit) @30722   6 years ak19 Remove repeated empty lines, leaving just a single blank line between …
(edit) @30720   6 years ak19 Getting the nightly gti email to be sent again on the new machine …
(edit) @30652   7 years ak19 Committing outstanding files for diffcol supporting jdb for GS3 …
(edit) @30613   7 years ak19 Don't send nightly email messages about updates to the test language …
(edit) @30611   7 years ak19 Modified version of remove_extra_lines script to handle the Updated …
(edit) @30605   7 years ak19 Committing the changes necessary to get the GTI crons to work on the …
(edit) @30594   7 years kjdon modifying the input and results areas to get it looking the same in …
(edit) @30590   7 years kjdon this change was made on puka, and when I tried this on commdev, it …
(edit) @30581   7 years ak19 GTI related changes to add gs3 demo collection config files' …
(edit) @30425   7 years davidb Save metadata as JSON file. Create sub-directories to spreadout the …
(edit) @30424   7 years ak19 Dr Bainbridge improved the code so that the script always gets the …
(edit) @30422   7 years davidb Tidier treatment of 'bin' and 'audio' directories from an SVN point of view
(edit) @30421   7 years davidb Removed from SVN tree
(edit) @30420   7 years davidb svn:ignore 'audio' and 'bin'
(edit) @30419   7 years davidb Support for JSON added
(edit) @30418   7 years davidb Code updated to work through a sequence of pages for one artist
(edit) @30417   7 years davidb Switched order of stdout and stderr redirects
(edit) @30416   7 years davidb Initial cut at code for scraping music excerpts from AMC site
(edit) @30415   7 years davidb Main trunk.
(edit) @30414   7 years davidb Top-level folder for Western Sydney University led Music Affect …
(edit) @30410   7 years ak19 Finally got the upload of the binary to turn up.
(edit) @30408   7 years ak19 Fix (from Puka) for the upload destination machine to work out the …
(edit) @30407   7 years ak19 Script for clearing old binaries and logs from the machine where they …
(edit) @30406   7 years ak19 Minor change. rke log for expeditee should have the os suffix on all OS.
(edit) @30405   7 years ak19 Swap month and day in date string for generated nightly binary, to get …
(edit) @30404   7 years ak19 Cosmetic change: more helpful comment for configuring rke-setup on mac.
(edit) @30399   7 years ak19 Minor changes to mirror the way the expeditee nightly generation …
(edit) @30398   7 years ak19 Parallel changes for the linux machines to use variables in …
(edit) @30397   7 years ak19 Fixed values in environment setup file, and used as variables in …
(edit) @30396   7 years ak19 Uploading expeditee nightly on linux 32 bit works.
(edit) @30395   7 years ak19 Got uploading to work at last.
(edit) @30393   7 years ak19 Bat file also needs to delete stale expeditee snapshot before …
(edit) @30392   7 years ak19 Deleting expeditee snapshot before regenerating.
(edit) @30391   7 years sjm84 Additional setup.sh.in file for linux-lsb VM environments. To be used …
(edit) @30390   7 years sjm84 New expeditee SVN repository URL.
(edit) @30387   7 years ak19 First step in creating nightly scripts for generating expeditee …
(edit) @30137   7 years jmt12 Obsolete - merged with documentation/wiki
(edit) @30136   7 years jmt12 Merged with Greenstone Dokuwiki files in documentation/wiki - …
(edit) @30126   7 years davidb Some svn:ignore values to cover oof VS automatically generated folders
(edit) @30125   7 years davidb VS seems to want this icon in the top level
(edit) @30124   7 years davidb branded icon for project
Note: See TracRevisionLog for help on using the revision log.