Changeset 30917 for other-projects

Show
Ignore:
Timestamp:
25.10.2016 14:20:26 (3 years ago)
Author:
davidb
Message:

Changes resulting from a fresh run at provisioning, which yielded the first successful run of RUN.bash on the cluster over HDFS. OS=Trusty as Xenial box doesn't appear to have puppet installed. JDK=1.7 because Trusty by default doesn't have apt-get for OpenJDK8.

Location:
other-projects/hathitrust/vagrant-spark-hdfs-cluster/trunk
Files:
7 added
4 modified

Legend:

Unmodified
Added
Removed
  • other-projects/hathitrust/vagrant-spark-hdfs-cluster/trunk/Vagrantfile

    r30903 r30917  
    55 
    66  config.vm.provider "virtualbox" do |v| 
    7     v.memory = 1024 
     7    v.memory = 2048 
    88    v.cpus = 2 
    99  end 
    1010 
    1111  config.vm.box = "ubuntu/trusty64" 
     12#  config.vm.box = "ubuntu/xenial64" 
    1213 
    1314  config.vm.provision :puppet do |puppet| 
  • other-projects/hathitrust/vagrant-spark-hdfs-cluster/trunk/manifests/base-hadoop.pp

    r30913 r30917  
    33$spark_home  = "${install_dir}/spark" 
    44 
    5 $user = "ubuntu" 
    6 $group = "ubuntu" 
     5$user = "htrc" 
     6$group = "hadoop" 
    77 
    88$hadoop_master = '10.10.0.52' 
  • other-projects/hathitrust/vagrant-spark-hdfs-cluster/trunk/modules/hadoop/manifests/init.pp

    r30913 r30917  
    22 
    33exec { "download_hadoop": 
    4   # Download from nearby mirror, otherwise task can time-out 
    5   command => "wget -O /tmp/hadoop-2.7.3.tar.gz http://apache.mirror.amaze.com.au/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz", 
    6   # command => "wget -O /tmp/hadoop-2.7.3.tar.gz http://www-eu.apache.org/dist/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz", 
     4  # Download from nearby mirror, otherwise task can time-out with default setting of '300' 
     5  # command => "wget -O /tmp/hadoop-2.7.3.tar.gz http://apache.mirror.amaze.com.au/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz", 
     6  command => "wget -O /tmp/hadoop-2.7.3.tar.gz   http://www-us.apache.org/dist/hadoop/common/hadoop-2.7.3/hadoop-2.7.3.tar.gz 
     7", 
     8  timeout => 0, 
    79  path => $path, 
    8   user => $user, 
    9   group => $user, 
     10#  user => $user, 
     11#  group => $group, 
    1012  unless => "ls ${install_dir} | grep hadoop-2.7.3", 
    1113  require => Package["openjdk-7-jdk"] 
  • other-projects/hathitrust/vagrant-spark-hdfs-cluster/trunk/modules/hadoop/templates/hadoop-env.sh

    r30903 r30917  
    2323 
    2424# The java implementation to use. 
    25 export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64/ 
     25export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 
    2626 
    2727# The jsvc implementation to use. Jsvc is required to run secure datanodes