Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Binary output in MR job

$
0
0
Hi,

I'm writing a MR 2 job in which I'm reading plain text as input and
producing avro output. On running the job in local mode, the output is
being serialized into json format. What can I do so that the output uses
binary encoding. Following is my job definition:

Job job = new Job(getConf(), "Post convertor");
job.setJarByClass(getClass());

AvroJob.setOutputKeySchema(job, Post.getClassSchema());
AvroJob.setMapOutputKeySchema(job, Schema.create(Schema.Type.LONG));
AvroJob.setMapOutputValueSchema(job, Post.getClassSchema());

FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));

job.setMapperClass(PostMapper.class);
job.setReducerClass(PostReducer.class);

Regards.
Anand

Viewing all articles
Browse latest Browse all 5648

Trending Articles