Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Debugging Pig Job


+
rakesh kothari 2010-11-16, 22:53
Copy link to this message
-
Re: Debugging Pig Job

On 11/16/10 2:53 PM, "rakesh kothari" <[EMAIL PROTECTED]> wrote:

>
>
> Hi,
>
> I am using Pig 0.7.0. Is there a good way to have Pig assign an informative
> name to each of the MR Job generated in Pig Physical plan ? Maybe name of the
> relation itself.
>

This is a good suggestion. In current versions, your pig query is in a file,
the file name is part of the job name. But all MR jobs for the query have
this same name. Having relation name as part of job name might not be very
useful as the mapping of relation to MR job is not 1:1 .
I think a name of the form "PigLatin:name:mrid" will be useful. Where name
is the name of the query file, or 'set job.name' or PigServer.setJobName() .
And mrid is an id corresponding to each MR job, and this id would be printed
in output of explain command (which also shows what relations are
corresponding to the MR job).
Can you please open a jira for this feature ?

> Also I am not able to customize my job name using PigServer.setJobName(). I am
> calling it just before registering my pig script. Am I missing something ?
>

This is a bug. Can you please open a jira? This might be related to -
https://issues.apache.org/jira/browse/PIG-1674

Thanks,
Thejas
+
Xiaomeng Wan 2010-11-17, 17:01
+
rakesh kothari 2010-11-24, 03:58
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB