Changeset 33563 for gs3-extensions

Timestamp:

2019-10-11T23:29:40+13:00 (5 years ago)

Author:

ak19

Message:

Committing inactive testing batch scripts (only creates the regex-urlfilter.txt files and puts them in the correct location before printing the crawl command. Somehow the crawl command printed for any of the 1st, 2nd and last site (with the supposedly correct regex-urlfilter.txt manually put in place) does not work yet. For a while I couldn't get the davidb homepage to work either, but then after removing the nutch bin/crawl binary script and recompiling nutch it worked again. But the similar commands generated buy this batchcrawl.sh script are not working yet.

File:

• gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/batchcrawl.sh (added)