Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: When speculative execution is true, there is a data loss issue with multpleoutputs


+
AnilKumar B 2012-11-21, 15:07
Copy link to this message
-
Re: When speculative execution is true, there is a data loss issue with multpleoutputs
this is another problem with fileoutputformat committer, its related to
your.

https://issues.apache.org/jira/browse/MAPREDUCE-3772

it works like this: if multipleoutput is relative to job output, then
there is a workaround to make it work with commiter and outputs from
multiple tasks do not clash with each other, problem mentioned in ticket
cheats that relative vs absolute output path detection and all output is
lost on task commit.

But if output is absolute path, then its written directly to output file
which fails because writers from multiple attempts crash together.
+
Radim Kolar 2012-11-21, 15:31