Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Re: Nullpointer Exception when using UDF

Copy link to this message
Re: Nullpointer Exception when using UDF
Hi Dean,

Thanks for your response.  I reviewed the stack trace. As you mentioned the error shows up at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:137)

But this probably is  a java class that comes with hadoop 1.1.1 and it untouched.  Do you think there is a bug with this Java Class for hadoop 1.1.1?

or is the way the UDF I have created using the file downloaded from - https://issues.apache.org/jira/browse/HIVE-2361, may be causing the issue?
I read few forums that indicate the class may not be in the classpath for hadoop. Although I confirmed that is not the case.

Any further pointers or advise is appreciated.


 From: Dean Wampler <[EMAIL PROTECTED]>
Sent: Thursday, February 14, 2013 6:41 AM
Subject: Re:

According to your stack trace, you have NullPointerException on line 137 of your UDF.
On Thu, Feb 14, 2013 at 2:28 AM, neelesh gadhia <[EMAIL PROTECTED]> wrote:

>I am a Newbie to using UDF's on hive. But
implemented these GenericJDF ( https://issues.apache.org/jira/browse/HIVE-2361 ) on hive 0.9.0 and hadoop 1.1.1. Was able to add jar to hive
>hive> select * from emp;
>1    10    1000
>2    10    1200
>3    12    1500
>4    12    300
>5    12    1800
>6    20    5000
>7    20    7000
>8    20    10000
>Time taken: 0.191 seconds
>hive> add jar /usr/local/Cellar/hive/0.9.0/libexec/lib/GenUDF.jar;                        
>Added /usr/local/Cellar/hive/0.9.0/libexec/lib/GenUDF.jar to class path
>Added resource: /usr/local/Cellar/hive/0.9.0/libexec/lib/GenUDF.jar
>hive> create temporary function nexr_sum as 'com.nexr.platform.analysis.udf.GenericUDFSum';
>Time taken: 0.012 seconds
>and kicked the sample sql shown below. 
>SELECT t.empno, t.deptno, t.sal, nexr_sum(hash(t.deptno),t.sal) as sal_sum
>select a.empno, a.deptno, a.sal from emp a
>distribute by hash(a.deptno)
>sort BY a.deptno, a.empno
>) t;
>The sql failed with errors. Any pointers or advise towards resolving this is much appreciated.
>2013-02-13 23:30:18,925 INFO org.apache.hadoop.mapred.JobTracker:
Adding task (REDUCE) 'attempt_201302132324_0002_r_000000_3' to tip
task_201302132324_0002_r_000000, for tracker
>2013-02-13 23:30:18,925 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201302132324_0002_r_000000_2'
>2013-02-13 23:30:26,484 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from attempt_201302132324_0002_r_000000_3:
java.lang.RuntimeException: Error in configuring object
>at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
>at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
>at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
>at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:486)
>at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:421)
>at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>at java.security.AccessController.doPrivileged(Native Method)
>at javax.security.auth.Subject.doAs(Subject.java:396)
>at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
>at org.apache.hadoop.mapred.Child.main(Child.java:249)
>Caused by: java.lang.reflect.InvocationTargetException
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>at java.lang.reflect.Method.invoke(Method.java:597)
>at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
>... 9 more
>Caused by: java.lang.RuntimeException: Reduce operator initialization failed
>at org.apache.hadoop.hive.ql.exec.ExecReducer.configure(ExecReducer.java:157)
Dean Wampler, Ph.D.
Nitin Pawar 2013-02-14, 16:54
neelesh gadhia 2013-02-14, 17:04
Christopher Wirt 2013-02-15, 17:46