Changeset 33535


Ignore:
Timestamp:
09/30/19 16:49:19 (14 months ago)
Author:
ak19
Message:
  1. New setup.sh script for on a hadoop system to setup the git projects we use for processing CommonCrawl data the way we've been doing so far. This script can be split into INSTALL.sh and COMPILE.sh hereafter. 2. Modifications to the overall Readme.txt
Location:
gs3-extensions/maori-lang-detection/hdfs-cc-work
Files:
1 added
1 edited

Legend:

Unmodified
Added
Removed
  • gs3-extensions/maori-lang-detection/hdfs-cc-work/Readme.txt

    r33530 r33535  
    3373371. Grab our 1st bash script and put it into the /home/vagrant/cc-index-table/src/script:
    338338       cd cc-index-table/src/script
    339        wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/bin/script/get_maori_WET_records_for_crawl.sh
     339       wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/get_maori_WET_records_for_crawl.sh
    340340       chmod u+x get_maori_WET_records_for_crawl.sh
    341341
     
    359359
    360360
    361 2. Grab our 2nd bash script and put it into the top level of the vagrant VM (/home/vagrant):
    362 
    363        cd /home/vagrant
    364        wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/bin/script/get_Maori_WET_records_from_CCSep2018_on.sh
     3612. Grab our 2nd bash script and put it into the top level of cc-index-table (/home/vagrant/cc-index/table):
     362
     363       cd cc-index-table
     364       wget http://svn.greenstone.org/gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/get_Maori_WET_records_from_CCSep2018_on.sh
    365365       chmod u+x get_Maori_WET_records_from_CCSep2018_on.sh
    366366
    367 RUN AS:
     367RUN FROM cc-index-table DIRECTORY AS:
     368    (cd cc-index-table)
    368369    ./get_Maori_WET_records_from_CCSep2018_on.sh
    369370
Note: See TracChangeset for help on using the changeset viewer.