Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> _SUCCESS file -> _FAILURE file?


+
Zebeljan, Nebojsa 2012-10-18, 07:43
+
Dmitriy Ryaboy 2012-10-18, 21:03
Copy link to this message
-
Re: _SUCCESS file -> _FAILURE file?
One way could be to handle the failure of a Pig job in a wrapper script. Or
use the Java APIs to be able to see if the job completed successfully.

http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#store%28java.lang.String,%20java.lang.String%29
http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#executeBatch%28%29
On Thu, Oct 18, 2012 at 2:03 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> That's a Hadoop mapreduce feature, not a Pig feature, so that request
> should go there.
>
> Can't really do the _failure thing though, if you think about it --
> programs can fail by crashing, in which case they might not be able to
> write a file. Or maybe they are not crashing, but there is a problem
> talking to HDFS, still can't write the failure.
>
>
>
> D
>
>
> On Thu, Oct 18, 2012 at 12:43 AM, Zebeljan, Nebojsa
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> > Pig writes a 0 byte _SUCCESS file, when the STORE has been successfully
> done. Is there something like a _FAILURE file that indicates that the STORE
> failed?
> >
> > I need to determine with a another application (java), if the pig run
> has finished and if it was successful or not.
> >
> > Any approach is welcome – Thanks for your help!
> >
> > Regards,
> > Nebo
>
+
Alejandro Abdelnur 2012-10-18, 22:59
+
Gianmarco De Francisci Mo... 2012-10-19, 16:00
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB