Thanks Radim.

Yes, as you said we are not writing into sub-directory of main job. I will
try by making them as sub-directories of output dir.

But one question, when I turn of speculative execution then it is working
fine with same multiple output directory structure. May I know, how exactly
it working in this case?

When we change the speculative execution flag, why exactly there is a
difference in output data?

Thanks,
B Anil Kumar.

On Wed, Nov 21, 2012 at 8:01 PM, Radim Kolar <[EMAIL PROTECTED]> wrote:

> its not data loss, problem is caused that multipleoutputs do not work with
> standard committer if you do not write into subdirectory of main job output.
>

this is another problem with fileoutputformat committer, its related to
your.

https://issues.apache.org/jira/browse/MAPREDUCE-3772

it works like this: if multipleoutput is relative to job output, then
there is a workaround to make it work with commiter and outputs from
multiple tasks do not clash with each other, problem mentioned in ticket
cheats that relative vs absolute output path detection and all output is
lost on task commit.

But if output is absolute path, then its written directly to output file
which fails because writers from multiple attempts crash together.

Dne 21.11.2012 16:07, AnilKumar B napsal(a):
> Thanks Radim.
>
> Yes, as you said we are not writing into sub-directory of main job. I
> will try by making them as sub-directories of output dir.
>
> But one question, when I turn of speculative execution then it is
> working fine with same multiple output directory structure. May I
> know, how exactly it working in this case?
>
> When we change the speculative execution flag, why exactly there is a
> difference in output data?
because if you are not using multipleoutput then you are not writing to
real file, but to file with name generated from its task attempt in tmp
subdirectory. They do not overwrite each other. In HDFS you can have
only one writer per file.