-M/R, Strange behavior with multiple Gzip files
x6i4uybz labs 2012-12-05, 16:02
I have a M/R job which does a bulk import to hbase.
I have to process many gzip files (2800 x ~ 100mb)
I don't understand why my job instanciates 80 maps but runs each map
sequentialy like if there is only one big gz file.
Is there a problem in my driver ? Or maybe I miss something.
I use "FileInputFormat.addInputPath(job, new Path(args))" where args
is a directory.
Can you help me, please ?
Harsh J 2012-12-05, 17:33
x6i4uybz labs 2012-12-06, 16:25
Harsh J 2012-12-06, 16:39