'Pseudocode to Calculate average using MapReduce
Hi I want to write a MapReduce algorithm in pseudo code to solve the following problem:
Given input records in the following format:
address, zip, city, house_value,
please calculate the average house value for each zip code.
I would really appreciate if you could help me with this..
Solution 1:[1]
The easiest would be to use Apache Pig, here is an example of finding an average:
inpt = load 'data.txt' as (address:chararray, zip:chararray, city:chararray, house_value:long);
grp = group inpt by zip;
average = foreach grp generate FLATTEN(group) as (zip), AVG(inpt.house_value) as average_price;
dump average;
For Pseudo Map Reduce code you would need one MAPPER, COMBINER and a REDUCER
MAPPER(record):
zip_code_key = record['zip'];
value = {1, record['house_value']};
emit(zip_code_key, value);
COMBINER(zip_code_key, value_list):
record_num = 0;
value_sum = 0;
foreach (value : value_list) {
record_num += value[0];
value_sum += value[1];
}
value_out = {record_num, value_sum};
emit(zip_code_key, value_out);
REDUCER(zip_code_key, value_list):
record_num = 0;
value_sum = 0;
foreach (value : value_list) {
record_num += value[0];
value_sum += value[1];
}
avg = value_sum / record_num;
emit(zip_code_key, avg);
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 |