Wednesday, June 03, 2015

MapReduce in Hadoop

After installing Hadoop you will go for MapReduce operations or Hive NoSQL implementation. Here I am going to demonstrate two mapreduce application already provided in hadoop to test the architecture which will help you getting familiar with it. First one is calculating value of Pi and other one is running Terasort test.

1. Calculate value of Pi.

Hadoop jar /root/ Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar pi 2 10




2. Running TeraSort test.

Run teragen to generate rows of random data to sort.

yarn jar /root/Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar teragen 2 /root/Downloads/hadoop


Run terasort to sort the database.

yarn jar /root/Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar terasort /root/Downloads/hadoop /root/Downloads/hadoop/terasort


Run teravalidate to validate the sorted Teragen.

yarn jar /root/Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar teravalidate /root/Downloads/hadoop/terasort /root/Downloads/hadoop/teravalidate