# # ChangeLog for other-projects/maori-lang-detection/mongodb-data # # Generated by Trac 1.4.2 # 2024-05-24T01:32:46+12:00 Thu, 13 Feb 2020 06:34:14 GMT ak19 [33918] * other-projects/maori-lang-detection/mongodb-data/ManualShortlisting2_afterMongoDBReingest.txt (modified) * other-projects/maori-lang-detection/mongodb-data/manualList_globalDomains_whereAPageContainsMRI.txt (modified) Country codes added to each domain's URL of the manual site/domain ... Thu, 13 Feb 2020 04:42:11 GMT ak19 [33916] * other-projects/maori-lang-detection/mongodb-data/ManualShortlisting2_afterMongoDBReingest.txt (modified) Updated the rest of the file after reingest Thu, 13 Feb 2020 04:12:06 GMT ak19 [33915] * other-projects/maori-lang-detection/mongodb-data/6counts_nonProductSites1_manualShortlist.json (added) * other-projects/maori-lang-detection/mongodb-data/ManualShortlisting2_afterMongoDBReingest.txt (moved) Forgot to add a (manual) counts file created last week, and am now ... Thu, 13 Feb 2020 04:09:07 GMT ak19 [33914] * other-projects/maori-lang-detection/MoreReading/mongodb.txt (modified) * other-projects/maori-lang-detection/mongodb-data/ManualShortlisting.txt (modified) * other-projects/maori-lang-detection/mongodb-data/ManualShortlisting2.txt (modified) * other-projects/maori-lang-detection/mongodb-data/manualList_globalDomains_whereAPageContainsMRI.txt (added) Shortlisted just the domain sites by country into ... Wed, 12 Feb 2020 08:27:02 GMT ak19 [33913] * other-projects/maori-lang-detection/MoreReading/mongodb.txt (modified) * other-projects/maori-lang-detection/hdfs-cc-work/GS_README.TXT (modified) * other-projects/maori-lang-detection/mongodb-data/tables.txt (modified) * other-projects/maori-lang-detection/src/org/greenstone/atea/MongoDBQueryer.java (modified) 1. Adjusted table mongodb query statements to be more exact, but same ... Wed, 05 Feb 2020 10:38:57 GMT ak19 [33907] * other-projects/maori-lang-detection/mongodb-data/ManualShortlisting2.txt (added) See previous commit message. This will be the file with the results ... Mon, 03 Feb 2020 10:20:53 GMT ak19 [33895] * other-projects/maori-lang-detection/mongodb-data/5b_counts_containsMRI_groupedByNZorOverseasNoFilter.json (moved) Minor rename Mon, 03 Feb 2020 10:20:33 GMT ak19 [33894] * other-projects/maori-lang-detection/mongodb-data/5b_count_containsMRI_groupedByNZorOverseasNoFilter.json (added) * other-projects/maori-lang-detection/mongodb-data/5b_geojson-features_containsMRI_groupedByNZorOverseasNoFilter.json (added) * other-projects/maori-lang-detection/mongodb-data/5b_map_containsMRI_groupedByNZorOverseasNoFilter.png (added) * other-projects/maori-lang-detection/mongodb-data/5b_multipoint_containsMRI_groupedByNZorOverseasNoFilter.json (added) * other-projects/maori-lang-detection/mongodb-data/6counts_sitesWithPagesContainingMRI_manualShortlist.json (moved) * other-projects/maori-lang-detection/mongodb-data/6geojson-features_sitesWithPagesContainingMRI_manualShortlist.json (moved) * other-projects/maori-lang-detection/mongodb-data/6map_sitesWithPagesContainingMRI_manualShortlist.png (moved) * other-projects/maori-lang-detection/mongodb-data/6multipoint_sitesWithPagesContainingMRI_manualShortlist.json (moved) * other-projects/maori-lang-detection/mongodb-data/tables.txt (modified) 1. Adding map, counts.json and geo-json files for 5b count of sites ... Mon, 03 Feb 2020 09:41:47 GMT ak19 [33893] * other-projects/maori-lang-detection/mongodb-data/8TableOfNumDetectedVsManualSITESWithMRI.ods (modified) * other-projects/maori-lang-detection/mongodb-data/8table_siteCountSummary.png (modified) 1. Left out region code column. 2. Two more sheets of work in ... Mon, 03 Feb 2020 09:28:44 GMT ak19 [33892] * other-projects/maori-lang-detection/mongodb-data/8TableOfNumDetectedVsManualSITESWithMRI.ods (moved) Sheets renamed and spreadsheet renamed Mon, 03 Feb 2020 09:27:37 GMT ak19 [33891] * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/8table_siteCountSummary.png (added) * other-projects/maori-lang-detection/mongodb-data/ManualShortlisting.txt (added) * other-projects/maori-lang-detection/mongodb-data/TableOfNumDetectedVsManualSITESWithMRI.ods (added) Site level detected vs manual inspected data: working shown in file ... Mon, 03 Feb 2020 07:31:33 GMT ak19 [33890] * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (modified) Finished going through NZ sites listing of numPagesContainingMRI > 0 ... Mon, 03 Feb 2020 02:48:40 GMT ak19 [33889] * other-projects/maori-lang-detection/mongodb-data/1a_table_miInUrlPath.csv (modified) * other-projects/maori-lang-detection/mongodb-data/1a_table_miInUrlPath.png (added) * other-projects/maori-lang-detection/mongodb-data/1b_table_noMiInUrlPath.csv (modified) * other-projects/maori-lang-detection/mongodb-data/1b_table_noMiInUrlPath.png (added) * other-projects/maori-lang-detection/mongodb-data/1table_allCrawledSites.csv (modified) * other-projects/maori-lang-detection/mongodb-data/1table_allCrawledSites.png (added) * other-projects/maori-lang-detection/mongodb-data/2table_sitesWithPagesInMRI.csv (modified) * other-projects/maori-lang-detection/mongodb-data/2table_sitesWithPagesInMRI.png (added) * other-projects/maori-lang-detection/mongodb-data/3table_sitesWithPagesContainingMRI.csv (modified) * other-projects/maori-lang-detection/mongodb-data/3table_sitesWithPagesContainingMRI.png (added) * other-projects/maori-lang-detection/mongodb-data/4table_tentativeNonProductSites.csv (modified) * other-projects/maori-lang-detection/mongodb-data/4table_tentativeNonProductSites.png (added) * other-projects/maori-lang-detection/mongodb-data/5b_table_containsMRI_groupedByNZorOverseasNoFilter.csv (added) * other-projects/maori-lang-detection/mongodb-data/5b_table_containsMRI_groupedByNZorOverseasNoFilter.png (added) * other-projects/maori-lang-detection/mongodb-data/5table_tentativeNonProductSites1.csv (modified) * other-projects/maori-lang-detection/mongodb-data/5table_tentativeNonProductSites1.png (added) * other-projects/maori-lang-detection/mongodb-data/tables.txt (modified) 1. Additional column: totalPagesAcrossMatchingSites. 2. Screengrab of ... Fri, 31 Jan 2020 10:17:47 GMT ak19 [33886] * other-projects/maori-lang-detection/mongodb-data/2table_sitesWithPagesInMRI.csv (moved) Minor. File rename Fri, 31 Jan 2020 09:21:40 GMT ak19 [33884] * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/src/org/greenstone/atea/WebPageURLsListing.java (modified) 0. Previous commit had lots of modifications, and only 2 files ... Fri, 31 Jan 2020 08:50:34 GMT ak19 [33883] * other-projects/maori-lang-detection/mongodb-data/5table_tentativeNonProductSites1.csv (modified) * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/src/org/greenstone/atea/MongoDBAccess.java (modified) * other-projects/maori-lang-detection/src/org/greenstone/atea/RandomURLsForDomainGenerator.java (modified) * other-projects/maori-lang-detection/src/org/greenstone/atea/WebPageURLsListing.java (modified) Clarifications Thu, 30 Jan 2020 07:18:09 GMT ak19 [33878] * other-projects/maori-lang-detection/mongodb-data/tables.txt (modified) Better comment Thu, 30 Jan 2020 07:07:59 GMT ak19 [33877] * other-projects/maori-lang-detection/mongodb-data/5counts_tentativeNonProductSites1.json (modified) Reordering to have proper descending order of counts Wed, 29 Jan 2020 06:18:29 GMT ak19 [33875] * other-projects/maori-lang-detection/mongodb-data/6b_geojson-features_manualShortlist_numPagesContainingMRI.json (moved) * other-projects/maori-lang-detection/mongodb-data/6b_multipoint_manualShortlist_numPagesContainingMRI.json (moved) Renaming 2 more files correctly Wed, 29 Jan 2020 06:15:29 GMT ak19 [33874] * other-projects/maori-lang-detection/mongodb-data/6a_geojson-features_manualShortlist_numPagesInMRI.json (moved) * other-projects/maori-lang-detection/mongodb-data/6a_multipoint_manualShortlist_numPagesInMRI.json (moved) Renaming 2 files correctly Fri, 24 Jan 2020 08:44:04 GMT ak19 [33872] * other-projects/maori-lang-detection/mongodb-data/4counts_tentativeNonProductSites.json (modified) * other-projects/maori-lang-detection/mongodb-data/5counts_tentativeNonProductSites1.json (modified) * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/random255_domainsNZ_IsMRI.txt (added) * other-projects/maori-lang-detection/mongodb-data/tables.txt (modified) 1. Added the file containing the 255 random NZ page URLs to sample. ... Thu, 23 Jan 2020 08:16:44 GMT ak19 [33868] * other-projects/maori-lang-detection/mongodb-data/6a_counts_geojson-features_manualShortlist_numPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6a_counts_multipoint_manualShortlist_numPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6a_map_numPagesInMRI_fromManualInspectedSites.png (added) * other-projects/maori-lang-detection/mongodb-data/6b_counts_geojson-features_manualShortlist_numPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6b_counts_multipoint_manualShortlist_numPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6b_map_numPagesContainingMRI_fromManualInspectedSites.png (added) * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (modified) With the updated code for generating the maps from 6a and 6b manual ... Tue, 21 Jan 2020 09:01:07 GMT ak19 [33854] * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (modified) Manually gone over around 150 webpages of sample size of 255 webpages ... Fri, 17 Jan 2020 09:38:24 GMT ak19 [33851] * other-projects/maori-lang-detection/mongodb-data/6a_map_manuallyInspected_numPagesInMRI.png (deleted) * other-projects/maori-lang-detection/mongodb-data/6b_map_manuallyInspected_numPagesContainingMRI.png (deleted) Deleting faulty maps. NZ numPages inMRI and containingMRI count is ... Fri, 17 Jan 2020 09:38:00 GMT ak19 [33850] * other-projects/maori-lang-detection/mongodb-data/6a_map_manuallyInspected_numPagesInMRI.png (moved) * other-projects/maori-lang-detection/mongodb-data/6b_map_manuallyInspected_numPagesContainingMRI.png (moved) Renames before deleting faulty maps. NZ numPages inMRI and ... Fri, 17 Jan 2020 09:21:14 GMT ak19 [33848] * other-projects/maori-lang-detection/mongodb-data/1a_counts_miInUrlPath.json (modified) * other-projects/maori-lang-detection/mongodb-data/1a_table_miInUrlPath.csv (added) * other-projects/maori-lang-detection/mongodb-data/1b_counts_noMiInUrlPath.json (modified) * other-projects/maori-lang-detection/mongodb-data/1b_table_noMiInUrlPath.csv (added) * other-projects/maori-lang-detection/mongodb-data/1table_allCrawledSites.csv (added) * other-projects/maori-lang-detection/mongodb-data/2table__sitesWithPagesInMRI.csv (added) * other-projects/maori-lang-detection/mongodb-data/3table_sitesWithPagesContainingMRI.csv (added) * other-projects/maori-lang-detection/mongodb-data/4table_tentativeNonProductSites.csv (added) * other-projects/maori-lang-detection/mongodb-data/5table_tentativeNonProductSites1.csv (added) * other-projects/maori-lang-detection/mongodb-data/6a_counts_manualShortlist_numPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6a_geojson-features_manualShortlist_numPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6a_manuallyInspected_numPagesInMRI.png (added) * other-projects/maori-lang-detection/mongodb-data/6a_multipoint_manualShortlist_numPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6b_counts_manualShortlist_numPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6b_geojson-features_manualShortlist_numPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6b_manuallyInspected_numPagesContainingMRI.png (added) * other-projects/maori-lang-detection/mongodb-data/6b_multipoint_manualShortlist_numPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/6counts_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/6geojson-features_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/6map_exclAutotranslatedSites1_manualShortlist.png (modified) * other-projects/maori-lang-detection/mongodb-data/6table_nonProductSites1_manualShortlist.json (added) * other-projects/maori-lang-detection/mongodb-data/tables.txt (added) Tables of mongodb counts (1-5 table) and manual counts (6table). ... Fri, 17 Jan 2020 06:32:16 GMT ak19 [33847] * other-projects/maori-lang-detection/MoreReading/mongodb.txt (modified) * other-projects/maori-lang-detection/mongodb-data/6counts_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/6geojson-features_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/6map_exclAutotranslatedSites1_manualShortlist.png (modified) indigenousblogs.com did have one page actually in Maori (an XML ... Fri, 17 Jan 2020 03:49:05 GMT ak19 [33846] * other-projects/maori-lang-detection/mongodb-data/1map_allCrawledSites.png (modified) * other-projects/maori-lang-detection/mongodb-data/2map_sitesWithPagesInMRI.png (modified) * other-projects/maori-lang-detection/mongodb-data/3map_sitesWithPagesContainingMRI.png (modified) * other-projects/maori-lang-detection/mongodb-data/4map_exclTentativeAutotranslatedSites.png (modified) * other-projects/maori-lang-detection/mongodb-data/5map_exclTentativeAutotranslatedSites1.png (modified) * other-projects/maori-lang-detection/mongodb-data/6map_exclAutotranslatedSites1_manualShortlist.png (modified) Cropped out the json portion Fri, 17 Jan 2020 03:34:11 GMT ak19 [33845] * other-projects/maori-lang-detection/mongodb-data/6map_exclAutotranslatedSites1_manualShortlist.png (modified) Cropped out the json portion Fri, 17 Jan 2020 03:33:24 GMT ak19 [33844] * other-projects/maori-lang-detection/mongodb-data/6counts_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/6geojson-features_nonProductSites1_manualShortlist.json (modified) * other-projects/maori-lang-detection/mongodb-data/6map_exclAutotranslatedSites1_manualShortlist.png (modified) * other-projects/maori-lang-detection/mongodb-data/7miInURLPath_exclNZ_byCountryCode.json (added) Regenerated Mon, 13 Jan 2020 06:45:21 GMT ak19 [33823] * other-projects/maori-lang-detection/MoreReading/mongodb.txt (modified) * other-projects/maori-lang-detection/conf/url-blacklist-filter.txt (modified) * other-projects/maori-lang-detection/mongodb-data (added) * other-projects/maori-lang-detection/mongodb-data/1a_counts_miInUrlPath.json (added) * other-projects/maori-lang-detection/mongodb-data/1a_geojson-features_miInUrlPath.json (added) * other-projects/maori-lang-detection/mongodb-data/1a_multipoint_miInUrlPath.json (added) * other-projects/maori-lang-detection/mongodb-data/1b_counts_noMiInUrlPath.json (added) * other-projects/maori-lang-detection/mongodb-data/1b_geojson-features_noMiInUrlPath.json (added) * other-projects/maori-lang-detection/mongodb-data/1b_multipoint_noMiInUrlPath.json (added) * other-projects/maori-lang-detection/mongodb-data/1counts_allCrawledSites.json (added) * other-projects/maori-lang-detection/mongodb-data/1geojson-features_allCrawledSites.json (added) * other-projects/maori-lang-detection/mongodb-data/1map_allCrawledSites.png (added) * other-projects/maori-lang-detection/mongodb-data/1multipoint_allCrawledSites.json (added) * other-projects/maori-lang-detection/mongodb-data/2counts_sitesWithPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/2geojson-features_sitesWithPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/2map_sitesWithPagesInMRI.png (added) * other-projects/maori-lang-detection/mongodb-data/2multipoint_sitesWithPagesInMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/3counts_sitesWithPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/3geojson-features_sitesWithPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/3map_sitesWithPagesContainingMRI.png (added) * other-projects/maori-lang-detection/mongodb-data/3multipoint_sitesWithPagesContainingMRI.json (added) * other-projects/maori-lang-detection/mongodb-data/4counts_tentativeNonProductSites.json (added) * other-projects/maori-lang-detection/mongodb-data/4geojson-features_tentativeNonProductSites.json (added) * other-projects/maori-lang-detection/mongodb-data/4map_exclTentativeAutotranslatedSites.png (added) * other-projects/maori-lang-detection/mongodb-data/4multipoint_tentativeNonProductSites.json (added) * other-projects/maori-lang-detection/mongodb-data/5counts_tentativeNonProductSites1.json (added) * other-projects/maori-lang-detection/mongodb-data/5geojson-features_tentativeNonProductSites1.json (added) * other-projects/maori-lang-detection/mongodb-data/5map_exclTentativeAutotranslatedSites1.png (added) * other-projects/maori-lang-detection/mongodb-data/5multipoint_tentativeNonProductSites1.json (added) * other-projects/maori-lang-detection/mongodb-data/6counts_nonProductSites1_manualShortlist.json (added) * other-projects/maori-lang-detection/mongodb-data/6geojson-features_nonProductSites1_manualShortlist.json (added) * other-projects/maori-lang-detection/mongodb-data/6map_exclAutotranslatedSites1_manualShortlist.png (added) * other-projects/maori-lang-detection/mongodb-data/6multipoint_nonProductSites1_manualShortlist.json (added) Recommitting mongo-data folder with renamed files with numbering.