Friday, May 30, 2014

Hadoop : Single Node Setup/Pseudo-Distributed Operation (cygwin)

http://hadoop.apache.org/docs/current1/single_node_setup.html#PseudoDistributed
https://gist.github.com/tariqmislam/2159173
http://neckuv.blogspot.com/2012/09/trying-out-hadoop-pt1-hadoop-103-on.html
http://alans.se/blog/2010/hadoop-hbase-cygwin-windows-7-x64/#.Uw2CJfldVx0
http://archive.apache.org/dist/hadoop/core/hadoop-0.20.2/
http://wiki.apache.org/hadoop/QuickStart

On cygwin:

1. Setup sshd , ssh-copy-id to localhost (actually, seems just "ssh-host-config -y" will also work for 0.20.2)






















2. conf/hadoop-env.sh (JAVA_HOME=/cygdrive/c/Java/jdk1.7.0_51)
3. conf/core-site.xml, conf/hdfs-site.xml, conf/mapred-site.xml
4. bin/hadoop namenode -format, bin/start-all.sh
5. check 50070/50030





























6. run grep example




























7. collect output
8. bin/stop-all.sh



[Note] actually the example got stuck at 0%, seems need to use hadoop 0.20.2
[Note] mklink /D Java "C:\Program Files\Java\jdk1.7.0_51"

No comments:

Related Posts Plugin for WordPress, Blogger...