Changeset 33570 for gs3-extensions

Timestamp:

2019-10-16T20:04:44+13:00 (5 years ago)

Author:

ak19

Message:

Need to check if UNFINISHED file actually exists before moving it across to destination.

File:

• gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/batchcrawl.sh (modified) (1 diff)

gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/batchcrawl.sh

@@ -76 +76 @@
 
 
-    # move the peripheral crawl products (the log.out and UNFINISHED files)
+    # move the peripheral crawl products (the log.out and optional UNFINISHED file)
     # from the input to the output folder. This way we can re-run the crawl and
-    # the original output will still have been preserved
+    # these files will still have been preserved as long as the output folder
+    # isn't deleted
     mv ${siteDir}log.out $outputDir/$crawlId/log.out
-    mv ${siteDir}UNFINISHED $outputDir/$crawlId/UNFINISHED
+    if [ -e "${siteDir}UNFINISHED" ]; then
+    mv ${siteDir}UNFINISHED $outputDir/$crawlId/UNFINISHED
+    fi
 }