Changeset 33604 for gs3-extensions/maori-lang-detection/conf/sites-too-big-to-exhaustively-crawl.txt

Timestamp:

2019-10-24T23:22:30+13:00 (5 years ago)

Author:

ak19

Message:

1. Better output into possible-product-sites.txt including the overseas country code prefix to help decide whether the site is worth keeping or not. 2. Updated whitelisting and top-sites filters to grab the /mi/ subsections of sites that don't appear to be autotranslated. This is done in preparation for blocking out product sites hereafter

File:

• gs3-extensions/maori-lang-detection/conf/sites-too-big-to-exhaustively-crawl.txt (modified) (1 diff)

gs3-extensions/maori-lang-detection/conf/sites-too-big-to-exhaustively-crawl.txt

@@ -61 +61 @@
 # special case
 mi.centr-zashity.ru,SINGLEPAGE
+
+martinvrijland.nl,martinvrijland.nl/mi/
+csunplugged.org,csunplugged.org/mi/
+gpedia.com,gpedia.com/mi/
 
 # TOP SITE BUT NOT TOP 500