Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> UDFContext NULL JobConf


+
Henning Kropp 2013-10-30, 16:57
+
Cheolsoo Park 2013-10-30, 17:36
Copy link to this message
-
Re: UDFContext NULL JobConf
Hi,

thanks for your reply. I read about the expected behavior on the front-end
and I am getting the NPE on the back-end. The Mappers log the Exception
during Execution.

I am currently digging through debug messages. What to look out for? There
are bunch of

[main] DEBUG org.apache.hadoop.conf.Configuration  - java.io.IOException:
config()

log messages. But I recall them as being "normal" for reasons I don't
remember.

Regards
2013/10/30 Cheolsoo Park <[EMAIL PROTECTED]>

> Hi,
>
> Are you getting NPE on the front-end or the back-end? Sounds like jobConf
> is not added to UDFContext, which is expected on the front-end. Please see
> the comments in getJobConf() and addJobConf() in the source code:
>
>
> https://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/impl/util/UDFContext.java
>
> Thanks,
> Cheolsoo
>
>
> On Wed, Oct 30, 2013 at 9:57 AM, Henning Kropp <[EMAIL PROTECTED]
> >wrote:
>
> > Hi,
> >
> > I am stuck. In my UDF (Java) extends EvalFunc the following code throws
> and
> > NPE in exec(), when executed in -x mapred mode:
> >
> > Configuration jobConf = UDFContext.getUDFContext().getJobConf();
> > System.err.println(jobConf.toString());
> >
> > I did not find any useful information as why my JobConf is always null.
> All
> > I find is that this is the right way to get the JobConf in a UDF and that
> > the behavior of what is returned when running locally (jira issue).
> >
> > Any ideas? I am running it on a very old Hadoop version 0.20.2 Are there
> > some known issues? I use Pig 0.11.1
> >
> > Many thanks in advanced
> >
> > PS: Just found someone with the same issue
> > http://stackoverflow.com/questions/18795008/accessing-hdfs-from-pig-udf
> >
>
+
Pradeep Gollakota 2013-10-30, 17:58
+
Henning Kropp 2013-10-30, 18:34
+
Cheolsoo Park 2013-10-30, 19:09
+
Henning Kropp 2013-10-31, 07:49
+
Henning Kropp 2013-10-30, 16:43
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB