Changeset 33849

Timestamp:

2020-01-17T22:22:18+13:00 (4 years ago)

Author:

ak19

Message:

One less Australian site as it was an infographic containing Maori words in English captions and graph legend.

Location:

other-projects/maori-lang-detection

Files:

• MoreReading/mongodb.txt (modified) (6 diffs)
• journal-paper/writeup (modified) (1 diff)

other-projects/maori-lang-detection/MoreReading/mongodb.txt

@@ -1154 +1154 @@
 !!  https://www.kiwiproperty.com - e.g. https://www.kiwiproperty.com/the-base/mi/he-paepaki/ has some actual MRI sentences. [Not autotranslated]
 ?   http://fionajack.net - Wellington gallery of artist. A few occurrences of Kia Ora in a title like context (e.g. "Street Party Kia Ora! Kia Ora!")
-!!  https://infogram.com/te-marautanga-o-aotearoa-moe-pld-allocations-2012-1go502ygvn562jd  - site of individual pages (like docs.google.com). This one has a relevant infogram image.
+X!!     https://infogram.com/te-marautanga-o-aotearoa-moe-pld-allocations-2012-1go502ygvn562jd  - site of individual pages (like docs.google.com). This one has a relevant infogram image. But it's English with MRI in the image legend and captions.
 !!  https://koreromaori.com - some actual Maori language sentences
     http://theunderwaterworld.com/Galleries/Roimata/roimata-frame.html - placenames
@@ -1310 +1310 @@
 + http://www.unicode.org, [Universal declaration of Human Rights]
 + https://static-promote.weebly.com,
-+ http://aclhokiangarocks.blogspot.com, [often English, but COMMUNITY]
++ http://aclhokiangarocks.blogspot.com, [often English, but COMMUNITY. At least short or partial MRI sentences.]
 
 BIBLE/MOHAMMED/BAHAI TRANSLATIONS probably not auto translations:
@@ -1329 +1329 @@
 !! https://maorinews.com, 
 !! http://maaori.com, 
-!!+ http://kiaorahola.blogspot.com/
+!!+ http://kiaorahola.blogspot.com,
 + https://kjohnsonnz.blogspot.com, 
 + http://pumanawawhangara.blogspot.com, 
 + http://dannykahei.tripod.com,
-+ http://burkekm001.tripod.com
++ http://burkekm001.tripod.com,
 + http://tkkpipipaopao.blogspot.com, 
 + http://manateina.blogspot.com, 
@@ -1472 +1472 @@
 
 ---------------
+All sites except NZ or .nz TLD where containingMRI=true manually inspected. Includes overseas sites with mi in URL path. All NZ sites passed through without inspection.
 
 MANUAL - TOTAL NUM SITES WITH SOME MRI CONTENT BY COUNTRY
@@ -1489 +1490 @@
 NZ: 176
 US: 25+4 from US with mi in URL path = 29
-AU: 3
+AU: 2
 DE: 2
 DK: 2
@@ -1497 +1498 @@
 FR: 1
 IE: 1
-TOTAL: 213+4 from US with mi in URL path = 217
+TOTAL: 213+4 from US with mi in URL path = 216
 
 

other-projects/maori-lang-detection/journal-paper/writeup

@@ -34 +34 @@
 mri     0.0014          0.0017          0.0012
 
-Over 1400 sites were detected and CommonCrawl returned over 1400 unique site domain containing pages it detected as Maori in the twelve-month period from Sep 2018 to Aug 2019. The above percentages are for the 3 final crawls (June to Aug 2019). Of these 1400 sites, 213+3 = 216 sites appeared to contain actual Maori language sentences composed by humans when manually inspected. The percentage of the high-quality web content that is in Maori may therefore be almost an order of magnitude less.
+Over 1400 sites were detected and CommonCrawl returned over 1400 unique site domain containing pages it detected as Maori in the twelve-month period from Sep 2018 to Aug 2019. The above percentages are for the 3 final crawls (June to Aug 2019). Of these 1400 sites, 216 sites appeared to contain actual Maori language sentences composed by humans when manually inspected. The percentage of the high-quality web content that is in Maori may therefore be almost an order of magnitude less.