Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: When speculative execution is true, there is a data loss issue with multpleoutputs


Copy link to this message
-
Re: When speculative execution is true, there is a data loss issue with multpleoutputs
Thanks Radim.

Yes, as you said we are not writing into sub-directory of main job. I will
try by making them as sub-directories of output dir.

But one question, when I turn of speculative execution then it is working
fine with same multiple output directory structure. May I know, how exactly
it working in this case?

When we change the speculative execution flag, why exactly there is a
difference in output data?

Thanks,
B Anil Kumar.

On Wed, Nov 21, 2012 at 8:01 PM, Radim Kolar <[EMAIL PROTECTED]> wrote:

> its not data loss, problem is caused that multipleoutputs do not work with
> standard committer if you do not write into subdirectory of main job output.
>
+
Radim Kolar 2012-11-21, 15:44
+
Radim Kolar 2012-11-21, 15:31
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB