Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - HBase bulk loader doing speculative execution when it set to false in mapred-site.xml


Copy link to this message
-
Re: HBase bulk loader doing speculative execution when it set to false in mapred-site.xml
anil gupta 2012-04-02, 22:56
Thanks for the suggestion, Sandy. I wil let you know the outcome once i run
the job.

On Mon, Apr 2, 2012 at 3:26 PM, Sandy Pratt <[EMAIL PROTECTED]> wrote:

> It might work to set the property as final on the server side, so that
> clients can't override it:
>
>        <property>
>                <name>mapred.reduce.tasks.speculative.execution</name>
>                <value>false</value>
>                <final>true</final>
>                <description>If true, then multiple instances of some
> reduce tasks
>                        may be executed in parallel.
>                </description>
>        </property>
>
> Seems like having it set as final at the JT would prevent speculative
> execution no matter what the client says, but I could be mistaken.
>
> Sandy
>
> > -----Original Message-----
> > From: anil gupta [mailto:[EMAIL PROTECTED]]
> > Sent: Friday, March 30, 2012 14:53
> > To: [EMAIL PROTECTED]
> > Subject: HBase bulk loader doing speculative execution when it set to
> false in
> > mapred-site.xml
> >
> > Hi All,
> >
> > I am using cdh3u2. I ran HBase bulk loading with property
> > "mapred.reduce.tasks.speculative.execution" set to "false" in mapred-
> > site.xml. Still, i can see 6 killed task in Bulk Loading job and after
> short analysis
> > i realized that these jobs are killed because another worker node
> completed
> > the task, hence it means that speculative execution is still on. Why the
> HBase
> > Bulk loader is doing speculative execution when i have set it to false in
> > mapred-site.xml? Please let me know if i am missing something over here.
> >
> > --
> > Thanks & Regards,
> > Anil Gupta
>

--
Thanks & Regards,
Anil Gupta