Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Why a sql only use one map task?


Copy link to this message
-
Re: RE: Why a sql only use one map task?
What about your total Map Task Capacity?
you may check it from http://your_jobtracker:50030/jobtracker.jsp

2011/8/24 Daniel,Wu <[EMAIL PROTECTED]>:
> I checked my setting, all are with the default value.So per the book of
> "Hadoop the definitive guide", the split size should be 64M. And the file
> size is about 500M, so that's about 8 splits. And from the map job
> information (after the map job is done), I can see it gets 8 split from one
> node. But anyhow it starts only one map task.
>
>
>
> At 2011-08-24 02:28:18,"Aggarwal, Vaibhav" <[EMAIL PROTECTED]> wrote:
>
> If you actually have splittable files you can set the following setting to
> create more splits:
>
>
>
> mapred.max.split.size appropriately.
>
>
>
> Thanks
>
> Vaibhav
>
>
>
> From: Daniel,Wu [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, August 23, 2011 6:51 AM
> To: hive
> Subject: Why a sql only use one map task?
>
>
>
>   I run the following simple sql
> select count(*) from sales;
> And the job information shows it only uses one map task.
>
> The underlying hadoop has 3 data/data nodes. So I expect hive should kick
> off 3 map tasks, one on each task nodes. What can make hive only run one map
> task? Do I need to set something to kick off multiple map task?  in my
> config, I didn't change hive config.
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB