Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - _SUCCESS file -> _FAILURE file?


+
Zebeljan, Nebojsa 2012-10-18, 07:43
+
Dmitriy Ryaboy 2012-10-18, 21:03
Copy link to this message
-
Re: _SUCCESS file -> _FAILURE file?
Prashant Kommireddi 2012-10-18, 21:10
One way could be to handle the failure of a Pig job in a wrapper script. Or
use the Java APIs to be able to see if the job completed successfully.

http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#store%28java.lang.String,%20java.lang.String%29
http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#executeBatch%28%29
On Thu, Oct 18, 2012 at 2:03 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> That's a Hadoop mapreduce feature, not a Pig feature, so that request
> should go there.
>
> Can't really do the _failure thing though, if you think about it --
> programs can fail by crashing, in which case they might not be able to
> write a file. Or maybe they are not crashing, but there is a problem
> talking to HDFS, still can't write the failure.
>
>
>
> D
>
>
> On Thu, Oct 18, 2012 at 12:43 AM, Zebeljan, Nebojsa
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> > Pig writes a 0 byte _SUCCESS file, when the STORE has been successfully
> done. Is there something like a _FAILURE file that indicates that the STORE
> failed?
> >
> > I need to determine with a another application (java), if the pig run
> has finished and if it was successful or not.
> >
> > Any approach is welcome – Thanks for your help!
> >
> > Regards,
> > Nebo
>
+
Alejandro Abdelnur 2012-10-18, 22:59
+
Gianmarco De Francisci Mo... 2012-10-19, 16:00