edu.rice.hj.runtime.mapreduce2

Class WordCount

java.lang.Object

edu.rice.hj.runtime.mapreduce2.MapReduceTask<Void,edu.rice.hj.runtime.mapreduce2.KeyString,edu.rice.hj.runtime.mapreduce2.KeyString,Integer,Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>>

edu.rice.hj.runtime.mapreduce2.WordCount

public class WordCount
extends MapReduceTask<Void,edu.rice.hj.runtime.mapreduce2.KeyString,edu.rice.hj.runtime.mapreduce2.KeyString,Integer,Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>>

WordCount class.

Author:

Shams Imam (shams@rice.edu)

Field Summary

Fields inherited from class edu.rice.hj.runtime.mapreduce2.MapReduceTask

numMapTasks, numReduceTasks, printTimes

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	WordCount(int numMapTasks, int numReduceTasks, boolean printTimes, String fileName) Constructor for WordCount.

Method Summary

Modifier and Type	Method and Description
static void	main(String[] args) main.
void	map(Void inKey, edu.rice.hj.runtime.mapreduce2.KeyString inValue, Aggregator<edu.rice.hj.runtime.mapreduce2.KeyString,Integer> aggregator) The MAP function.
Map<Void,List<edu.rice.hj.runtime.mapreduce2.KeyString>>	partition(int partitionId) PARTITION the input to be divided into the MAP tasks.
Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>	reduce(edu.rice.hj.runtime.mapreduce2.KeyString inKey, List<Integer> inValue, Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer> initialValue) The REDUCE function.
Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>	reduceIdentity(edu.rice.hj.runtime.mapreduce2.KeyString inKey) reduceIdentity.

Methods inherited from class edu.rice.hj.runtime.mapreduce2.MapReduceTask

results, run

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WordCount

protected WordCount(int numMapTasks,
                    int numReduceTasks,
                    boolean printTimes,
                    String fileName)

Constructor for WordCount.

Parameters:

numMapTasks - a int.

numReduceTasks - a int.

printTimes - a boolean.

fileName - a String object.

Method Detail

main

public static void main(String[] args)

main.

Parameters:

args - an array of String objects.

partition

public Map<Void,List<edu.rice.hj.runtime.mapreduce2.KeyString>> partition(int partitionId)

PARTITION the input to be divided into the MAP tasks. This should be a constant time operation to avoid serialization bottlenecks.

Specified by:

partition in class MapReduceTask<Void,edu.rice.hj.runtime.mapreduce2.KeyString,edu.rice.hj.runtime.mapreduce2.KeyString,Integer,Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>>

Parameters:

partitionId - The index of partition to create.

Returns:

array representing each of the partitions.

map

public void map(Void inKey,
                edu.rice.hj.runtime.mapreduce2.KeyString inValue,
                Aggregator<edu.rice.hj.runtime.mapreduce2.KeyString,Integer> aggregator)

The MAP function.

Specified by:

map in class MapReduceTask<Void,edu.rice.hj.runtime.mapreduce2.KeyString,edu.rice.hj.runtime.mapreduce2.KeyString,Integer,Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>>

Parameters:

inKey - The input key.

inValue - The input value.

aggregator - Stores the intermediates.

reduceIdentity

public Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer> reduceIdentity(edu.rice.hj.runtime.mapreduce2.KeyString inKey)

reduceIdentity.

Specified by:

reduceIdentity in class MapReduceTask<Void,edu.rice.hj.runtime.mapreduce2.KeyString,edu.rice.hj.runtime.mapreduce2.KeyString,Integer,Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>>

Parameters:

inKey - The input key.

Returns:

the identity value for the reducer.

reduce

public Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer> reduce(edu.rice.hj.runtime.mapreduce2.KeyString inKey,
                                                                     List<Integer> inValue,
                                                                     Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer> initialValue)

The REDUCE function.

Specified by:

reduce in class MapReduceTask<Void,edu.rice.hj.runtime.mapreduce2.KeyString,edu.rice.hj.runtime.mapreduce2.KeyString,Integer,Pair<edu.rice.hj.runtime.mapreduce2.KeyString,Integer>>

Parameters:

inKey - The input key.

inValue - The input value.

initialValue - The current value representing the given key.

Returns:

the reduced value.