Wednesday, June 03, 2015

MapReduce in Hadoop

After installing Hadoop you will go for MapReduce operations or Hive NoSQL implementation. Here I am going to demonstrate two mapreduce application already provided in hadoop to test the architecture which will help you getting familiar with it. First one is calculating value of Pi and other one is running Terasort test.

1. Calculate value of Pi.

Hadoop jar /root/ Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar pi 2 10

2. Running TeraSort test.

Run teragen to generate rows of random data to sort.

yarn jar /root/Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar teragen 2 /root/Downloads/hadoop

Run terasort to sort the database.

yarn jar /root/Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar terasort /root/Downloads/hadoop /root/Downloads/hadoop/terasort

Run teravalidate to validate the sorted Teragen.

yarn jar /root/Downloads/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar teravalidate /root/Downloads/hadoop/terasort /root/Downloads/hadoop/teravalidate

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.