and .


10:44 Changeset [33718] by davidb
Useful top-level script
10:44 Changeset [33717] by davidb
Code tidy up; better error checking on running Java cmd; public/private …
10:43 Changeset [33716] by davidb
Changed to also write out resource id as file, based on inputFileName


21:10 Changeset [33715] by ak19
Much shorter Eclipse project file, currently including just the jar files …
14:40 Changeset [33714] by ak19
Updating Eclipse .classpath file's reference to jar lib files correct for …
14:37 Changeset [33713] by kjdon
refactoring LibraryServlet?. runSecurityChecks was happening too late. it …
14:23 Changeset [33712] by kjdon
modified createBasicRequest to use userContext methods, and to include …
14:20 Changeset [33711] by kjdon
added editEnabled into UserContext?


23:23 Changeset [33710] by ak19
Working queries and map coords for geojson.tools (ironically, Lat and Lng …
18:49 Changeset [33709] by ak19
Forgot to commit the zip file before deleting it
11:30 Changeset [33708] by davidb
Changed code so api key can be in separate file, and passed in on the …
11:27 Changeset [33707] by davidb
Adds in maven to path if untarred in java/packages area
11:14 Changeset [33706] by davidb
Tempalte file


14:08 Changeset [33705] by kjdon
reindented the file, no code changes
14:04 Changeset [33704] by kjdon
added some authentication error strings. some are used by the …
14:03 Changeset [33703] by kjdon
added more breadcrumbs for ease of finding your way back to the start
14:00 Changeset [33702] by kjdon
added depositorTitleAndLink template. TODO - get depositor text from the …
13:59 Changeset [33701] by kjdon
added more breadcrumbs to the page. And now it displays an error if there …
13:56 Changeset [33700] by kjdon
starting to put some of the strings into a dictionary - using …
13:53 Changeset [33699] by kjdon
first stab at requiring a user to be logged in to use the depositor, and …


23:14 Changeset [33698] by ak19
Links to more reading
23:10 Changeset [33697] by ak19
Changes to the README to provide instructions on making the fewest changes …
22:09 Changeset [33696] by ak19
Moved the individual READMEs into the top level too along with …
20:29 Changeset [33695] by ak19
Minor corrects and file rename
20:03 Changeset [33694] by ak19
interfaces\images folder structure can be customised for a site as an …
20:01 Changeset [33693] by ak19
Forgot we needed a toplevel collect folder
19:57 Changeset [33692] by ak19
Math collection toplevel folder restructure
19:56 Changeset [33691] by ak19
Math collection renames and moving things about
19:54 Changeset [33690] by ak19
Moving the science collection to the top level
19:52 Changeset [33689] by ak19
Rename again
19:51 Changeset [33688] by ak19
Moving the science collection related README, screenshot and preparation …
19:49 Changeset [33687] by ak19
Renaming science collection to not have period mark in its name, on advice …
19:46 Changeset [33686] by ak19
18:50 Changeset [33685] by davidb
Upstream change related to Solr ext
18:49 Changeset [33684] by davidb
Changes made around the time of the launch
18:46 Changeset [33683] by davidb
Updated to process latest version of spreadsheet
18:45 Changeset [33682] by davidb
Changes made around the time of the launch
18:44 Changeset [33681] by davidb
Added in flock technique to avoid multiple people running the same script
18:43 Changeset [33680] by davidb
Greenstone3 is fixed, so don't need to print out message about runing …
18:38 Changeset [33679] by davidb
Folder for working on updates (PDFs to del, PDFs to add) from Kiri
17:57 Changeset [33678] by davidb
setup for greenstone ext
17:57 Changeset [33677] by davidb
Intro text
17:55 Changeset [33676] by davidb
Some initial work getting a plugin going that call's Alex's VirusTotal?
00:22 Changeset [33675] by ak19
Committing the newer query results (but from before today's reingestion in …
00:21 Changeset [33674] by ak19
Changes to support the top 5 predicted langcodes and their confidence …
00:17 Changeset [33673] by ak19
Waikato Education Department's Science Activities and Maths Activities …


14:14 Changeset [33672] by kjdon
modified slightly so that the error messages come from the dictionary …
14:12 Changeset [33671] by kjdon
added a static getTextString method - currently this is in Action.java and …
14:10 Changeset [33670] by kjdon
added editEnabled att string
14:10 Changeset [33669] by kjdon
removed an annoying debug message
10:03 Changeset [33668] by kjdon
a few changes to debuginfo texts
09:55 Changeset [33667] by kjdon
preProcess.xsl renamed to expand-gslib.xsl to better indicate what it does


23:08 Changeset [33666] by ak19
Having finished sending all the crawl data to mongodb 1. Recrawled the 2 …
17:18 Changeset [33665] by davidb
Fixed jar name
17:17 Changeset [33664] by davidb
Initial version code for running VirusTotal? API against files, CLI scripts
17:12 Changeset [33663] by davidb
Changes after testing the scripts
17:04 Changeset [33662] by davidb
Scripts to compile and run java code
16:54 Changeset [33661] by davidb
Compiling needs to use Maven
16:53 Changeset [33660] by davidb
For Java source code
16:40 Changeset [33659] by davidb
Top-level folder for new extension based on TotalVirus? API which scans …
16:40 Changeset [33658] by davidb
Top-level folder for new extension based on TotalVirus? API which scans …


21:33 Changeset [33657] by ak19
Some fixes after brief testing against 1/3 of the crawl. Restarted …
21:11 Changeset [33656] by ak19
Final minor changes before I start processing the crawls of node2.
20:56 Changeset [33655] by ak19
Minor change to print statement
20:54 Changeset [33654] by ak19
Removing jar file that wasn't used after all.
20:51 Changeset [33653] by ak19
1. As suggested by Dr Bainbridge, made the code changes to use Morphia as …
20:41 Changeset [33652] by ak19
Introducing morphia subpackage
18:11 Changeset [33651] by ak19
1. Bugfix: overlappingSentences works. 2. storing numSentencesInMaor
12:06 Changeset [33650] by kjdon
updated to match the new xsl file names; lots of variable renames to …
12:04 Changeset [33649] by kjdon
renamed config_format and text_fragment_format to better represent what …
12:04 Changeset [33648] by kjdon
changed the debuginfo xsl and strings to match the new o=xxx debug options
09:30 Changeset [33647] by kjdon
added/changed a few of the output values for debugging the transform


18:46 Changeset [33646] by ak19
Saving the mongodb queries and learning links that Dr Bainbridge found …
18:45 Changeset [33645] by ak19
Fix to 2 bugs when sending data to MongoDB: 1. overlappingSentences was …
11:50 Changeset [33644] by ak19
Just committing the growing mongodb.txt file with links and instructions …
11:46 Changeset [33643] by ak19
Brought the template log4j.properties.in back up to speed. I forgot it …
11:06 Changeset [33642] by ak19
Forgot to commit the java driver for mongodb when I committed the Java …
10:53 Changeset [33641] by kjdon
commented out some debug statements
10:48 Changeset [33640] by kjdon
oops, I must have 'tidied' up the file and then not compiled it to check …
10:23 Changeset [33639] by kjdon
need to select child nodes, otherwise the gsf:default node ends up in the …
10:22 Changeset [33638] by kjdon
gslib doesn't use xml-to-string.xsl. its only used by formatmanager, so …
10:21 Changeset [33637] by kjdon
we can now use gsf and gslib in layout files.
10:04 Changeset [33636] by kjdon
include means the stylesheet gets added inline, import mea s it gets …
09:38 Changeset [33635] by ak19
Maori-language-detection doesn't use Greenstone 3 at present, it's not a …


23:59 Changeset [33634] by ak19
Rewrote NutchTextDumpProcessor? as NutchTextDumpToMongoDB.java, which uses …
19:43 Changeset [33633] by ak19
1. TextLanguageDetector? now has methods for collecting all sentences and …


14:53 Changeset [33632] by kjdon
overhaul of TransformingReceptionist?. changed the order of inlining …
14:52 Changeset [33631] by kjdon
added a bit more error reporting
14:44 Changeset [33630] by kjdon
minor comment changes
14:20 Changeset [33629] by kjdon
added methods using Parameter2 - for params with text node values
13:52 Changeset [33628] by kjdon
not sure why documentNode was a gsf:template here. Can't be like that as …
09:28 Changeset [33627] by kjdon
removed unnecessary comments


21:59 Changeset [33626] by ak19
21:58 Changeset [33625] by ak19
A file listing domains with seedurls containing /mi(/) that are located …
21:48 Changeset [33624] by ak19
Some cleanup surrounding the now renamed function createSeedURLsFile, now …
21:04 Changeset [33623] by ak19
1. Incorporated Dr Nichols earlier suggestion of storing page modified …
15:42 Changeset [33622] by ak19
File rename


20:35 Changeset [33621] by ak19
Comitting jotted down mongodb related instructions from what Dr Bainbridge …
14:24 Changeset [33620] by ak19
Final crawl, done on vagrant VM node6. Crawl site IDs 01407-01462.
11:36 Changeset [33619] by kjdon
need to handle the case where a collection file (eg image) gets library in …


20:14 Changeset [33618] by ak19
Adding in the download URL
17:13 Changeset [33617] by ak19
Node5 is now full and here is the finished crawl (up to and including site …


20:05 Changeset [33616] by ak19
Beginnings of Java class that is to interact with MongoDB. I don't yet …
20:03 Changeset [33615] by ak19
1. Worked out how to configure log4j to log both to console and logfile, …
11:22 Changeset [33614] by kjdon
added a new line
11:18 Changeset [33613] by kjdon
added allowdocumentediting and allowmapgpsediting options, plus also added …
11:00 Changeset [33612] by kjdon
work to do with params. add in default values to params if they are not …
10:55 Changeset [33611] by kjdon
added global setting to params - thesea re for params that are valid …
10:54 Changeset [33610] by kjdon
USER_SESSION_CACHE_ATT moved to GSParams, as it is stored in session likea …


23:03 Changeset [33609] by ak19
The tar files containing the crawled sites data shouldn't be called tar.gz …
23:02 Changeset [33608] by ak19
1. New script to export from HBase so that we could in theory reimport …


18:33 Changeset [33607] by ak19
Updated with the remaining successfully crawled sites on node4 before …
15:18 Changeset [33606] by ak19
1. Committing crawl data from node3 (2nd VM for nutch crawling). 2. …
14:54 Changeset [33605] by ak19
Node 4 VM still works, but committing first set of crawled sites on there


23:22 Changeset [33604] by ak19
1. Better output into possible-product-sites.txt including the overseas …
22:04 Changeset [33603] by ak19
Incorporating Dr Nichols suggestion to help weed out product sites: if tld …


23:49 Changeset [33602] by ak19
1. The final csv file, mri-sentences.csv, is now written out. 2. Only …
23:22 Changeset [33601] by ak19
Creates the 2nd csv file, with info about webpages. At present stores …
23:05 Changeset [33600] by ak19
Work in progress of writing out CSV files. In future, may write the same …
Note: See TracTimeline for information about the timeline view.