Hi,

I am using Pig 0.7.0. Is there a good way to have Pig assign an informative name to each of the MR Job generated in Pig Physical plan ? Maybe name of the relation itself.

Also I am not able to customize my job name using PigServer.setJobName(). I am calling it just before registering my pig script. Am I missing something ?

Thanks,
-Rakesh
     

On 11/16/10 2:53 PM, "rakesh kothari" <[EMAIL PROTECTED]> wrote:

>
>
> Hi,
>
> I am using Pig 0.7.0. Is there a good way to have Pig assign an informative
> name to each of the MR Job generated in Pig Physical plan ? Maybe name of the
> relation itself.
>

This is a good suggestion. In current versions, your pig query is in a file,
the file name is part of the job name. But all MR jobs for the query have
this same name. Having relation name as part of job name might not be very
useful as the mapping of relation to MR job is not 1:1 .
I think a name of the form "PigLatin:name:mrid" will be useful. Where name
is the name of the query file, or 'set job.name' or PigServer.setJobName() .
And mrid is an id corresponding to each MR job, and this id would be printed
in output of explain command (which also shows what relations are
corresponding to the MR job).
Can you please open a jira for this feature ?

> Also I am not able to customize my job name using PigServer.setJobName(). I am
> calling it just before registering my pig script. Am I missing something ?
>

This is a bug. Can you please open a jira? This might be related to -
https://issues.apache.org/jira/browse/PIG-1674

Thanks,
Thejas

I used to assign different number of reducers with the PARALLEL
keyword. It may not be as accurate as relation name, but works in most
of the time.

Xiaomeng

On Wed, Nov 17, 2010 at 8:22 AM, Thejas M Nair <[EMAIL PROTECTED]> wrote:
>
>
>
> On 11/16/10 2:53 PM, "rakesh kothari" <[EMAIL PROTECTED]> wrote:
>
>>
>>
>> Hi,
>>
>> I am using Pig 0.7.0. Is there a good way to have Pig assign an informative
>> name to each of the MR Job generated in Pig Physical plan ? Maybe name of the
>> relation itself.
>>
>
> This is a good suggestion. In current versions, your pig query is in a file,
> the file name is part of the job name. But all MR jobs for the query have
> this same name. Having relation name as part of job name might not be very
> useful as the mapping of relation to MR job is not 1:1 .
> I think a name of the form "PigLatin:name:mrid" will be useful. Where name
> is the name of the query file, or 'set job.name' or PigServer.setJobName() .
> And mrid is an id corresponding to each MR job, and this id would be printed
> in output of explain command (which also shows what relations are
> corresponding to the MR job).
> Can you please open a jira for this feature ?
>
>> Also I am not able to customize my job name using PigServer.setJobName(). I am
>> calling it just before registering my pig script. Am I missing something ?
>>
>
> This is a bug. Can you please open a jira? This might be related to -
> https://issues.apache.org/jira/browse/PIG-1674
>
> Thanks,
> Thejas
>
>

Here is the JIRA for this feature: https://issues.apache.org/jira/browse/PIG-1746

> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED]
> Date: Wed, 17 Nov 2010 07:22:45 -0800
> Subject: Re: Debugging Pig Job
>
>
>
>
> On 11/16/10 2:53 PM, "rakesh kothari" <[EMAIL PROTECTED]> wrote:
>
> >
> >
> > Hi,
> >
> > I am using Pig 0.7.0. Is there a good way to have Pig assign an informative
> > name to each of the MR Job generated in Pig Physical plan ? Maybe name of the
> > relation itself.
> >
>
> This is a good suggestion. In current versions, your pig query is in a file,
> the file name is part of the job name. But all MR jobs for the query have
> this same name. Having relation name as part of job name might not be very
> useful as the mapping of relation to MR job is not 1:1 .
> I think a name of the form "PigLatin:name:mrid" will be useful. Where name
> is the name of the query file, or 'set job.name' or PigServer.setJobName() .
> And mrid is an id corresponding to each MR job, and this id would be printed
> in output of explain command (which also shows what relations are
> corresponding to the MR job).
> Can you please open a jira for this feature ?
>
> > Also I am not able to customize my job name using PigServer.setJobName(). I am
> > calling it just before registering my pig script. Am I missing something ?
> >
>
> This is a bug. Can you please open a jira? This might be related to -
> https://issues.apache.org/jira/browse/PIG-1674
>
> Thanks,
> Thejas
>