Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Why a sql only use one map task?


Copy link to this message
-
Re: RE: Why a sql only use one map task?
wd 2011-08-24, 10:19
What about your total Map Task Capacity?
you may check it from http://your_jobtracker:50030/jobtracker.jsp

2011/8/24 Daniel,Wu <[EMAIL PROTECTED]>:
> I checked my setting, all are with the default value.So per the book of
> "Hadoop the definitive guide", the split size should be 64M. And the file
> size is about 500M, so that's about 8 splits. And from the map job
> information (after the map job is done), I can see it gets 8 split from one
> node. But anyhow it starts only one map task.
>
>
>
> At 2011-08-24 02:28:18,"Aggarwal, Vaibhav" <[EMAIL PROTECTED]> wrote:
>
> If you actually have splittable files you can set the following setting to
> create more splits:
>
>
>
> mapred.max.split.size appropriately.
>
>
>
> Thanks
>
> Vaibhav
>
>
>
> From: Daniel,Wu [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, August 23, 2011 6:51 AM
> To: hive
> Subject: Why a sql only use one map task?
>
>
>
>   I run the following simple sql
> select count(*) from sales;
> And the job information shows it only uses one map task.
>
> The underlying hadoop has 3 data/data nodes. So I expect hive should kick
> off 3 map tasks, one on each task nodes. What can make hive only run one map
> task? Do I need to set something to kick off multiple map task?  in my
> config, I didn't change hive config.
>
>
>
>