最大值的话小编没有什么可以说的,我相信大家都明白,这次我就直接上案列和代码了。
数据如下: 案列:编写MapReduce代码获得每年的最高气温。
代码:
package demo; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; //2014010114 public class MaxTmp { public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException { Configuration conf=new Configuration(); Job job = Job.getInstance(conf); job.setJarByClass(demo.class); job.setMapperClass(MMapper.class); job.setReducerClass(MReduce.class); job.setMapOutputKeyClass(Text.class); job.setMapOutputValueClass(IntWritable.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(LongWritable.class); FileInputFormat.setInputPaths(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); } public static class MMapper extends Mapper<LongWritable, Text, Text, IntWritable>{ protected void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { //转换数据类型 String line = value.toString(); //截取数据中的0-6位。 String year = line.substring(0,6); //转换int类型并截取数据中的8-10位 int tmp = Integer.parseInt(line.substring(8,10)); //直接写入 context.write(new Text(year), new IntWritable(tmp)); //201401 14 } } public static class MReduce extends Reducer<Text, IntWritable, Text, IntWritable>{ private IntWritable v = new IntWritable(); protected void reduce(Text key, Iterable<IntWritable> value, Context context) throws IOException, InterruptedException { // 14 int maxv = Integer.MIN_VALUE; for (IntWritable v : value) { maxv=Math.max(maxv, v.get()); } v.set(maxv); context.write(key, v); } } }运行结果: 下面我附上全部数据: 2014010114 2014010216 2014010317 2014010410 2014010506 2012010609 2012010732 2012010812 2012010919 2012011023 2001010116 2001010212 2001010310 2001010411 2001010529 2013010619 2013010722 2013010812 2013010929 2013011023 2008010105 2008010216 2008010337 2008010414 2008010516 2007010619 2007010712 2007010812 2007010999 2007011023 2010010114 2010010216 2010010317 2010010410 2010010506 2015010649 2015010722 2015010812 2015010999 2015011023
本次教程就到此结束了,博主会把关于MapReduce数据分析的各种操作讲解给大家听。有什么不懂的下方评论,博主看到都会去解答的。还有不要忘了多多支持博主呦,你们的支持就是我的动力。