Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> _SUCCESS file -> _FAILURE file?


+
Zebeljan, Nebojsa 2012-10-18, 07:43
+
Dmitriy Ryaboy 2012-10-18, 21:03
+
Prashant Kommireddi 2012-10-18, 21:10
Copy link to this message
-
Re: _SUCCESS file -> _FAILURE file?
that will only work if the interested party in the output fo the pig
script is the same one running the pig script.

On Thu, Oct 18, 2012 at 2:10 PM, Prashant Kommireddi
<[EMAIL PROTECTED]> wrote:
> One way could be to handle the failure of a Pig job in a wrapper script. Or
> use the Java APIs to be able to see if the job completed successfully.
>
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#store%28java.lang.String,%20java.lang.String%29
> http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#executeBatch%28%29
>
>
> On Thu, Oct 18, 2012 at 2:03 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
>> That's a Hadoop mapreduce feature, not a Pig feature, so that request
>> should go there.
>>
>> Can't really do the _failure thing though, if you think about it --
>> programs can fail by crashing, in which case they might not be able to
>> write a file. Or maybe they are not crashing, but there is a problem
>> talking to HDFS, still can't write the failure.
>>
>>
>>
>> D
>>
>>
>> On Thu, Oct 18, 2012 at 12:43 AM, Zebeljan, Nebojsa
>> <[EMAIL PROTECTED]> wrote:
>> > Hi,
>> > Pig writes a 0 byte _SUCCESS file, when the STORE has been successfully
>> done. Is there something like a _FAILURE file that indicates that the STORE
>> failed?
>> >
>> > I need to determine with a another application (java), if the pig run
>> has finished and if it was successful or not.
>> >
>> > Any approach is welcome – Thanks for your help!
>> >
>> > Regards,
>> > Nebo
>>

--
Alejandro
+
Gianmarco De Francisci Mo... 2012-10-19, 16:00