Changeset 33531

Show
Ignore:
Timestamp:
26.09.2019 23:03:01 (3 weeks ago)
Author:
ak19
Message:

Added whitelist for mi.wikipedia.org, and updates to blacklist and whitelist based on alexa top sites. Alexa.com only makes the present top 50 publicly visible, but wikipedia at least lists some more from Alexa taken from end 2016.

Location:
gs3-extensions/maori-lang-detection/conf
Files:
1 added
2 modified

Legend:

Unmodified
Added
Removed
  • gs3-extensions/maori-lang-detection/conf/url-blacklist-filter.txt

    r33502 r33531  
    1717# sounds like some pirating site 
    1818^http://pirateguides.com/ 
     19 
     20# from alexa topsites at https://www.alexa.com/topsites 
     21livejasmin.com 
     22pornhub.com 
     23# listed as a similar topsite at https://en.wikipedia.org/wiki/List_of_most_popular_websites 
     24redtube.com 
     25xvideos.com 
     26xhamster.com 
     27xnxx.com 
  • gs3-extensions/maori-lang-detection/conf/url-greylist-filter.txt

    r33502 r33531  
    1 # URL 'greylist': save matching urls to one side, to eyeball later and confirm if they should 
    2 # be included or skipped 
     1# URL 'greylist': save matching urls to one side, to eyeball later and decide if 
     2# they should be included after all or whether it was okay to have skipped them 
    33# FORMAT: 
    44# precede URL by ^ to greylist urls that match the given prefix 
     
    1515 
    1616# Add alexa top sites to greylist 
     17 
     18youtube.com 
     19tmall.com 
     20baidu.com 
     21qq.com 
     22sohu.com 
     23facebook.com 
     24taobao.com 
     25#login.tmall.com 
     26wikipedia.org 
     27yahoo.com 
     28360.cn 
     29jd.com 
     30amazon.com 
     31Sina.com.cn 
     32weibo.com 
     33#pages.tmall.com 
     34live.com 
     35vk.com 
     36netflix.com 
     37alipay.com 
     38office.com 
     39okezone.com 
     40csdn.net 
     41instagram.com 
     42xinhuanet.com 
     43babytree.com 
     44twitter.com 
     45ebay.com 
     46stackoverflow.com 
     47naver.com 
     48aliexpress.com 
     49twitch.tv 
     50tribunnews.com 
     51apple.com 
     52soso.com 
     53tianya.cn 
     54microsoftonline.com 
     55yandex.ru 
     56 
     57# Remaining top sites from https://en.wikipedia.org/wiki/List_of_most_popular_websites 
     58 
     59ok.ru 
     60paypal.com 
     61t.co 
     62pinterest.com 
     63sogou.com 
     64espn.com 
     65walmart.com 
     66bitly.com 
     67ampproject.org 
     68sm.cn 
     69 
     70 
     71 
     72# UNSURE - what if these contain translated pages? 
     73google.com 
     74bing.com 
     75amazon.co 
     76msn.com 
     77microsoft.com 
     78accuweather.com 
     79 
     80#nasa.gov 
     81# w3schools.com 
     82# quora.com 
     83#reddit.com 
     84#blogspot.com 
     85#yahoo.co.