Timestamp:
2019-08-09T20:37:23+12:00 (5 years ago)
Author:
ak19
Message:
  1. Started a file on feasibility with the data now available and some links that have interesting or useful information. 2. Minor simplification to get_commoncrawl_nz_urls.sh script. 3. config.props file to be used by Java. Can't find wget configuration settings to limit mirroring of a site to a certain number of pages, but can limit overall download to size (--quote or -Q).
File:
1 added

Note: See TracChangeset for help on using the changeset viewer.