Changeset 33535 for gs3-extensions

Timestamp:

2019-09-30T16:49:19+13:00 (5 years ago)

Author:

ak19

Message:

1. New setup.sh script for on a hadoop system to setup the git projects we use for processing CommonCrawl data the way we've been doing so far. This script can be split into INSTALL.sh and COMPILE.sh hereafter. 2. Modifications to the overall Readme.txt

Location:

gs3-extensions/maori-lang-detection/hdfs-cc-work

Files:

• Readme.txt (modified) (2 diffs)
• scripts/setup.sh (added)

gs3-extensions/maori-lang-detection/hdfs-cc-work/Readme.txt

@@ -337 +337 @@
 1. Grab our 1st bash script and put it into the /home/vagrant/cc-index-table/src/script:
        cd cc-index-table/src/script
-       wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/bin/script/get_maori_WET_records_for_crawl.sh
+       wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/get_maori_WET_records_for_crawl.sh
        chmod u+x get_maori_WET_records_for_crawl.sh
 
@@ -359 +359 @@
 
 
-2. Grab our 2nd bash script and put it into the top level of the vagrant VM (/home/vagrant):
-
-       cd /home/vagrant
-       wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/bin/script/get_Maori_WET_records_from_CCSep2018_on.sh
+2. Grab our 2nd bash script and put it into the top level of cc-index-table (/home/vagrant/cc-index/table):
+
+       cd cc-index-table
+       wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/get_Maori_WET_records_from_CCSep2018_on.sh
        chmod u+x get_Maori_WET_records_from_CCSep2018_on.sh
 
-RUN AS:
+RUN FROM cc-index-table DIRECTORY AS:
+    (cd cc-index-table)
     ./get_Maori_WET_records_from_CCSep2018_on.sh