https://codereview.stackexchange.com/questions/198343/crawl-and-gather-all-the-urls-recursively-in-a-domain
http://lucene.472066.n3.nabble.com/Using-nutch-just-for-the-crawler-fetcher-td611918.html

https://www.quora.com/What-are-some-Web-crawler-tips-to-avoid-crawler-traps

-----------
NUTCH
-----------
https://stackoverflow.com/questions/35449673/nutch-and-solr-indexing-blacklist-domain
	https://nutch.apache.org/apidocs/apidocs-1.6/org/apache/nutch/urlfilter/domainblacklist/DomainBlacklistURLFilter.html
	
https://lucene.472066.n3.nabble.com/blacklist-for-crawling-td618343.html
https://lucene.472066.n3.nabble.com/Content-of-size-X-was-truncated-to-Y-td4003517.html


Google: nutch mirror web site
https://stackoverflow.com/questions/33354460/nutch-clone-website
[https://stackoverflow.com/questions/35714897/nutch-not-crawling-entire-website
fetch -all seems to be a nutch v2 thing?]

Google: nutch performance tuning
* https://stackoverflow.com/questions/24383212/apache-nutch-performance-tuning-for-whole-web-crawling
* https://stackoverflow.com/questions/4871972/how-to-speed-up-crawling-in-nutch

Nutch v2 installation and set up:
https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781783286850/1/ch01lvl1sec09/installing-and-configuring-apache-nutch


SOLR:
* Query syntax: http://www.solrtutorial.com/solr-query-syntax.html
* Deleting a core: https://factorpad.com/tech/solr/reference/solr-delete.html