Before starting Map Reduce Program, two locations are to be created in HDFS. The first location is for the input, where MapReduce reads it and the second one for the output for storing it.

Creating the directories

Give the commands,

·         Hadoopfs –mkdir test1

·         Hadoopfs –mkdir test2

Here to write the Map Reduce Program, we are going to use Java - Eclipse SDK is considered.

To start with,

In the ‘Project Explorer’, Select‘DFS Location’ and then ‘mylocation’

Go to ‘define Hadoop Location’ and select the first tab, ‘General’.

o   Give, Location name as ‘xyz’

o   Give the Host as localhost,

o   The port is displayed by default. If you want, you can set it as something like, 54311 under MapReduce Master. Give the port number as 54310 under DFD Master.

Now, keep the input file into the input folder. To do it,

·         Go to the ‘input folder’.

·         Right click on it.

·         Select ‘Upload files to DFS’

·         Go to the Directory, where the file is stored

·         Select the file

·         Double click the file to view it. It does not allow you to edit the contents in this HDFS.

To create a program

o   Click on ‘File’.

o   Select ‘New’

o   Select ‘Other’

o   In the new window, select, ‘Map/Reduce Project’

o   Click on ‘Next’

o   Give the project name, like ‘Word Count’

o   When you do it for the first time, you need to configure Hadoop install directory. So, click on ‘Configure Hadoop install directory’ in the same window. The default location, where Hadoop is installed would be located, if not enter the specific location of Hadoop Home.

o   Click on ‘Next’

o   Click on ‘Finish’

It then loads all the Hadoop dependencies now.

To run the program,

·         Select ‘src’

·         Select ‘New’

·         Then select ‘Other’

·         Select ‘Map/Reduce folder’

·         Right click on the ‘Map/Reduce Driver’

·         Select ‘Run’

·         Select on ‘Run on Hadoop’.

Select the location of ‘mylocation’ and click on ‘Finish’.

Map/Reducer Driver: It is a driver, where the Map/Reduce program gets started. It is the macro program, where the process of Mapping, Reduce, Shuffle and sort are performed. This is a huge program that can be written as your own. The Map/Reduce program can be triggered from the web too, through the browser.

Once the job is completed, it shows the result and output.Once the output is created, it shows the result of the MapReduce.

For example, when a text file is taken with the names of the fruits,it is shown in the output file that how many of each fruit are repeated in the given text file. The text file is considered from the file added in the input folder.

The corresponding in detailed information can be accessed from the browser with ‘localhost: 50030/jobtracker.jsp’ and change it according to the information you want to explore.

All the information individually for the Mapper, Reducer, Shuffle, Sort can be accessed from it. The program can be edited according to the results we seek.

earning Map Reduce – Intellipaat is market leader in providing hadoop training across the globe. Trained more than 20000 professionals on Bigdata Training and online Hadoop training. Online Cassandra training at Intellipaat has advantages like you will getting recorded session of previous batch as soon as enrolled and all the sessions will be recorded for which you will have life time access.Trainers training on online Cassandra training are Industry architects having 18 yrs of experience.