Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> _SUCCESS file -> _FAILURE file?


Copy link to this message
-
Re: _SUCCESS file -> _FAILURE file?
that will only work if the interested party in the output fo the pig
script is the same one running the pig script.

On Thu, Oct 18, 2012 at 2:10 PM, Prashant Kommireddi
<[EMAIL PROTECTED]> wrote:
> One way could be to handle the failure of a Pig job in a wrapper script. Or
> use the Java APIs to be able to see if the job completed successfully.
>
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#store%28java.lang.String,%20java.lang.String%29
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#executeBatch%28%29
>
>
> On Thu, Oct 18, 2012 at 2:03 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
>> That's a Hadoop mapreduce feature, not a Pig feature, so that request
>> should go there.
>>
>> Can't really do the _failure thing though, if you think about it --
>> programs can fail by crashing, in which case they might not be able to
>> write a file. Or maybe they are not crashing, but there is a problem
>> talking to HDFS, still can't write the failure.
>>
>>
>>
>> D
>>
>>
>> On Thu, Oct 18, 2012 at 12:43 AM, Zebeljan, Nebojsa
>> <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> > Pig writes a 0 byte _SUCCESS file, when the STORE has been successfully
>> done. Is there something like a _FAILURE file that indicates that the STORE
>> failed?
>> >
>> > I need to determine with a another application (java), if the pig run
>> has finished and if it was successful or not.
>> >
>> > Any approach is welcome – Thanks for your help!
>> >
>> > Regards,
>> > Nebo
>>

--
Alejandro
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB