Changeset 33604 for gs3-extensions/maori-lang-detection/conf/url-whitelist-filter.txt

Timestamp:

2019-10-24T23:22:30+13:00 (4 years ago)

Author:

ak19

Message:

1. Better output into possible-product-sites.txt including the overseas country code prefix to help decide whether the site is worth keeping or not. 2. Updated whitelisting and top-sites filters to grab the /mi/ subsections of sites that don't appear to be autotranslated. This is done in preparation for blocking out product sites hereafter

File:

• gs3-extensions/maori-lang-detection/conf/url-whitelist-filter.txt (modified) (2 diffs)

gs3-extensions/maori-lang-detection/conf/url-whitelist-filter.txt

@@ -2 +2 @@
 # whitelist overrides blacklist and greylist.
 # FORMAT:
-# precede URL by ^ to greylist urls that match the given prefix
-# succeed URL by $ to greylist urls that match the given suffix
-# ^url$ will greylist urls that match the given url completely
-# Without either ^ or $ symbol, urls containing the given url will get greylisted
+# precede URL by ^ to whitelist urls that match the given prefix
+# succeed URL by $ to whitelist urls that match the given suffix
+# ^url$ will whitelist urls that match the given url completely
+# Without either ^ or $ symbol, urls containing the given url will get whitelisted
 
 # Special exception for this url on yale.edu, since we needed to blacklist
@@ -15 +15 @@
 http://www.krassotkin.ru/sites/prayer.su/maori/
 https://mi.centr-zashity.ru/
+
+
+
+# WHITELIST WEBSITES THAT HAVE NON-AUTOMATED /mi/ SUBSECTIONS
+# WE CONTROL WHAT PART OF THEM WILL BE DOWNLOADED (THE /mi SUBSECTION)
+# IN sites-too-big-to-exhaustively-crawl.txt
+#https://www.martinvrijland.nl/mi/te-mana-hinengaro/Ko-te-nuinga-ake-o-nga-tangata-kei-te-timata-ki-te-kite-kei-te-noho-tatou-i-roto-i-te-whakaata-ko-te-aha-tenei/
+#https://www.csunplugged.org/mi/principles/
+#http://www.gpedia.com/mi/gpedia/Reo_M%C4%81ori
+
+https://www.martinvrijland.nl
+https://www.csunplugged.org
+http://www.gpedia.com
+
+