Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Uber Job!


Thanks I was more interested to know about the use case of uber task

On Monday, May 6, 2013, Mohammad Tariq wrote:

> Split creation is primarily InputForma's responsibility, IMHO. It's good
> if splits overlap with the block, but it's not always true.
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Mon, May 6, 2013 at 8:15 PM, Rahul Bhattacharjee <
> [EMAIL PROTECTED] <javascript:_e({}, 'cvml',
> '[EMAIL PROTECTED]');>> wrote:
>
>> Hi,
>>
>> I was going through the definition of Uber Job of Hadoop.
>>
>> A job is considered uber when it has 10 or less maps , one reducer and
>> the complete data is less than one dfs block size.
>>
>> I have some doubts here-
>>
>> Splits are created as per the dfs block size.Creating 10 mappers are
>> possible from one block of data by some settings change (changing the max
>> split size). But trying to understand , why would some job need to run
>> around 10 maps for 64 MB of data.
>> One thing may be that the job is immensely CUP intensive. Will it be a
>> correct assumption? or is there is any other reason for this.
>>
>> Thanks,
>> Rahul
>>
>>
>>
>

--
Sent from Gmail Mobile