Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - missing error log


Copy link to this message
-
RE: missing error log
william.dowling@... 2013-03-25, 21:47
Thanks Johnny for your reply.  Working backwards: I am using MRv1.  I did try the logging suggestion you made, but did not get any other info.

So I did it the old-fashioned way with code bisection, commenting out swaths of my script to localize the error. It turned out it was this statement:

%default analysis_func = positive_features

When my code had that statement, it did not run productively, nor did it show any useful error message.  When I changed that to

%default analysis_func  positive_features

then the script ran as expected, and when I intentionally introduced an error, the line
  lkl lkl
then that error was logged with a good message

Pig Stack Trace
---------------
ERROR 1000: Error during parsing. Encountered " <IDENTIFIER> "lkl "" at line 100, column 1.
Was expecting one of:
    <EOF>
    "cat" ...
    "fs" ...
    "sh" ...
    "cd" ...

So the problem was the '=' character in the %default statement, but I was not able to find that out through an error message.

Thanks again,

Will
William F Dowling
Senior Technologist
Thomson Reuters

-----Original Message-----
From: Johnny Zhang [mailto:[EMAIL PROTECTED]]
Sent: Monday, March 25, 2013 4:10 PM
To: [EMAIL PROTECTED]
Subject: Re: missing error log

Hi, Will:
Can you try to check below make sure the logger is property configured:

1. Edit conf/log4j.properties as follows:

log4j.logger.org.apache.hadoop.conf.Configuration=ERROR, A

2. Edit conf/pig.properties as follows:

# log4jconf log4j configuration file
log4jconf=<path_to_log4j.properties>/log4j.properties

If there is still no error message, bump the level to INFO to make sure you
can see the logger output in the log file.

BTW, are you using MRv1 or MRv2 ?

Johnny
On Mon, Mar 25, 2013 at 12:32 PM, <[EMAIL PROTECTED]>wrote:

> Dear pig users,
>
> What does it mean when pig [Cloudera Pig version 0.10.0-cdh4.1.2] reports
>
> 2013-03-25 14:46:31,186 [main] INFO  org.apache.pig.Main - Logging error
> messages to: /proj/ac/acComponents/blocker/pig_1364237191181.log
>
> but that file is not created? I think there are errors in my pig script or
> something it depends on because it has STORE statements (and I tried DUMP
> too) yet there was no output. Also the runtime is too quick for any real
> processing to have happened.  But without an error log it’s hard to track
> down. How can I make pig reporting more verbose, and force it to the
> console?
>  Thanks,
>
> Will
>
> William F Dowling
> Senior Technologist
>
> Thomson Reuters
>
>
>
>