| Line | |
|---|
| 1 |
|
|---|
| 2 | Vargrant provisioning files to spin up a modest (4 node) Hadoop
|
|---|
| 3 | cluster for experiments processing HTRC Extracted Feature JSON files
|
|---|
| 4 | suitable for ingesting into Solr.
|
|---|
| 5 |
|
|---|
| 6 | Top-level code Apache Spark, processing HDFS stored JSON files, hence
|
|---|
| 7 | the need for an underlying Hadoop cluster.
|
|---|
| 8 |
|
|---|
| 9 | Provisioning based on the following online resources, but updated to
|
|---|
| 10 | use newer versions of Ubuntu, Java, and Hadoop.
|
|---|
| 11 |
|
|---|
| 12 | http://cscarioni.blogspot.co.nz/2012/09/setting-up-hadoop-virtual-cluster-with.html
|
|---|
| 13 |
|
|---|
| 14 | https://github.com/calo81/vagrant-hadoop-cluster
|
|---|
| 15 |
|
|---|
| 16 |
|
|---|
| 17 | Useful documentation about setting up a Hadoop cluster, read:
|
|---|
| 18 |
|
|---|
| 19 | http://chaalpritam.blogspot.co.nz/2015/05/hadoop-270-single-node-cluster-setup-on.html
|
|---|
| 20 | then
|
|---|
| 21 | http://chaalpritam.blogspot.co.nz/2015/05/hadoop-270-multi-node-cluster-setup-on.html
|
|---|
| 22 |
|
|---|
| 23 | OR
|
|---|
| 24 |
|
|---|
| 25 | https://xuri.me/2015/03/09/setup-hadoop-on-ubuntu-single-node-cluster.html
|
|---|
| 26 | then
|
|---|
| 27 | https://xuri.me/2016/03/22/setup-hadoop-on-ubuntu-multi-node-cluster.html
|
|---|
| 28 |
|
|---|
| 29 |
|
|---|
| 30 |
|
|---|
Note:
See
TracBrowser
for help on using the repository browser.
