Software engineering notes

notes from cloudera basic training > installing the cloudera hadoop distro locally and on ec2

ref: http://www.cloudera.com/hadoop
installing cloudera distro on a cluster
– motivation: hadoop is complocated to install
– cloudera uses Alternatives to manage a|b testing
– cloudera has created a “configurator” that will generate an rpm customized to your cluster
— generates the configuration files and a custom installer. Each can optionally be used together or separately
– alternatively, you can install an unconfigured distro
– for large scale deployment, use puppet, bcfg2, cfengine, etc. to manage the cluster
— cloudera’s tool can still be used to generate config scripts
– storing data in ebs takes advantage of locality and is much faster than s3
— ebs is more performant than normal hard drives

Written by Erik

June 11, 2009 at 3:38 pm

Posted in notes

Tagged with ,

%d bloggers like this: