Changeset 33570

Show
Ignore:
Timestamp:
16.10.2019 20:04:44 (4 weeks ago)
Author:
ak19
Message:

Need to check if UNFINISHED file actually exists before moving it across to destination.

Files:
1 modified

Legend:

Unmodified
Added
Removed
  • gs3-extensions/maori-lang-detection/hdfs-cc-work/scripts/batchcrawl.sh

    r33569 r33570  
    7676 
    7777 
    78     # move the peripheral crawl products (the log.out and UNFINISHED files) 
     78    # move the peripheral crawl products (the log.out and optional UNFINISHED file) 
    7979    # from the input to the output folder. This way we can re-run the crawl and 
    80     # the original output will still have been preserved 
     80    # these files will still have been preserved as long as the output folder 
     81    # isn't deleted 
    8182    mv ${siteDir}log.out $outputDir/$crawlId/log.out 
    82     mv ${siteDir}UNFINISHED $outputDir/$crawlId/UNFINISHED 
     83    if [ -e "${siteDir}UNFINISHED" ]; then 
     84    mv ${siteDir}UNFINISHED $outputDir/$crawlId/UNFINISHED 
     85    fi 
    8386} 
    8487