Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase bulk loader doing speculative execution when it set to false in mapred-site.xml


Copy link to this message
-
Re: HBase bulk loader doing speculative execution when it set to false in mapred-site.xml
Thanks for the suggestion, Sandy. I wil let you know the outcome once i run
the job.

On Mon, Apr 2, 2012 at 3:26 PM, Sandy Pratt <[EMAIL PROTECTED]> wrote:

> It might work to set the property as final on the server side, so that
> clients can't override it:
>
>        <property>
>                <name>mapred.reduce.tasks.speculative.execution</name>
>                <value>false</value>
>                <final>true</final>
>                <description>If true, then multiple instances of some
> reduce tasks
>                        may be executed in parallel.
>                </description>
>        </property>
>
> Seems like having it set as final at the JT would prevent speculative
> execution no matter what the client says, but I could be mistaken.
>
> Sandy
>
> > -----Original Message-----
> > From: anil gupta [mailto:[EMAIL PROTECTED]]
> > Sent: Friday, March 30, 2012 14:53
> > To: [EMAIL PROTECTED]
> > Subject: HBase bulk loader doing speculative execution when it set to
> false in
> > mapred-site.xml
> >
> > Hi All,
> >
> > I am using cdh3u2. I ran HBase bulk loading with property
> > "mapred.reduce.tasks.speculative.execution" set to "false" in mapred-
> > site.xml. Still, i can see 6 killed task in Bulk Loading job and after
> short analysis
> > i realized that these jobs are killed because another worker node
> completed
> > the task, hence it means that speculative execution is still on. Why the
> HBase
> > Bulk loader is doing speculative execution when i have set it to false in
> > mapred-site.xml? Please let me know if i am missing something over here.
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
>

--
Thanks & Regards,
Anil Gupta
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB