Changeset 33623 for gs3-extensions/maori-lang-detection/MoreReading/crawling-Nutch.txt

Timestamp:

2019-11-05T21:04:09+13:00 (4 years ago)

Author:

ak19

Message:

1. Incorporated Dr Nichols earlier suggestion of storing page modified time and char-encoding metadata if present in the crawl dump output. Have done so, but neither modifiedTime nor fetchTime metadata of the dump file appear to be a webpage's actual modified time, as they're from 2019 and set around the period we've been crawling. 2. Moved getDomainFromURL() function from CCWETProcessor.java to Utility.java since it's been reused. 3. MongoDBAccess class successfully connects (at least, no exceptions) and uses the newly added properties in config.properties to make the connection.

File:

• gs3-extensions/maori-lang-detection/MoreReading/crawling-Nutch.txt (modified) (1 diff)

gs3-extensions/maori-lang-detection/MoreReading/crawling-Nutch.txt

@@ -492 +492 @@
 
 
+INSTALLATION MONGO-DB AND CLIENT
+FROM: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
+    wget -qO - https://www.mongodb.org/static/pgp/server-4.2.asc | sudo apt-key add -
+    echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.2.list
+    sudo apt-get update
+    sudo apt-get install -y mongodb-org
+
+UNINSTALLING
+    https://www.anintegratedworld.com/uninstall-mongodb-in-ubuntu-via-command-line-in-3-easy-steps/