2018年安徽省大数据比赛MapReduce题目解答第三题

3、求每年最高气温（10分）

根据/opt/ahdsjjs/Temperature.txt编写MapperReduce脚本 获得每年的最高气温，并输出

数据格式如下：  

2014010114 2014010216 2014010317 2014010410 2014010506 2012010609 2012010732 2012010812 2012010919 2012011023 2001010116 2001010212 2001010310 2001010411 2001010529 2013010619 2013010722 2013010812 2013010929 2013011023 2008010105 2008010216 2008010337 2008010414 2008010516 2007010619 2007010712 2007010812 2007010999 2007011023 2010010114 2010010216 2010010317 2010010410 2010010506 2015010649 2015010722 2015010812 2015010999 2015011023

比如：2010012325表示在2010年01月23日的气温为25度。

现在要求使用MapReduce，计算每一年出现过的最大气温

思路：其实就是字符串的截取，那到年，以年为key进行分组，天气为value

主函数：

import java.io.IOException; import java.util.Arrays; import java.util.HashMap; import java.util.TreeMap; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Reducer; public class Myred extends Reducer<Text, IntWritable, Text, IntWritable> { @Override protected void reduce(Text key, Iterable<IntWritable> values, Reducer<Text, IntWritable, Text, IntWritable>.Context context) throws IOException, InterruptedException { /** * 2014 14 2014 16 2014 17 2014 10 2014 06 */ int maxval = Integer.MIN_VALUE; int max2val = -10000000; for (IntWritable val : values) { maxval = Math.max(maxval, val.get()); } context.write(new Text(key+" "+maxval), new IntWritable(max2val)); } }

Map函数：

import java.io.IOException; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; import tq.TQ; public class Mymap extends Mapper<LongWritable, Text, Text, IntWritable>{ @Override protected void map(LongWritable key, Text value, Mapper<LongWritable, Text, Text, IntWritable>.Context context) throws IOException, InterruptedException { // 2014010506 // 2012010609 String line = value.toString(); String yera = line.substring(0,6); int Tem = Integer.parseInt(line.substring(8,10)); context.write(new Text(yera), new IntWritable(Tem)); } }

Reduce函数：

import java.io.IOException; import java.util.Arrays; import java.util.HashMap; import java.util.TreeMap; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Reducer; public class Myred extends Reducer<Text, IntWritable, Text, IntWritable> { @Override protected void reduce(Text key, Iterable<IntWritable> values, Reducer<Text, IntWritable, Text, IntWritable>.Context context) throws IOException, InterruptedException { /** * 2014 14 2014 16 2014 17 2014 10 2014 06 */ int maxval = Integer.MIN_VALUE; int max2val = -10000000; for (IntWritable val : values) { maxval = Math.max(maxval, val.get()); } context.write(new Text(key+" "+maxval), new IntWritable(max2val)); } }