Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Hive UDF error : numberformat exception (String to Integer) conversion
praveenesh kumar
2012-05-30, 14:40
Edward Capriolo
2012-05-30, 14:44
praveenesh kumar
2012-05-30, 16:14
Edward Capriolo
2012-05-30, 16:16
Copy link to this message
-
Re: Hive UDF error : numberformat exception (String to Integer) conversion
Nitin Pawar 2012-05-30, 14:47
i won't tell the error but I would recommend to write a main function in
your udf and try with sample inputs which you are expecting in your query.

You will know whats the error you are committing

On Wed, May 30, 2012 at 8:14 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> You should to try catch and return NULL on bad data. The issue is if
> you have a single bad row the UDF will throw a exception up the chain.
> It will try again, it will fail again, ultimately the job will fail.
>
> On Wed, May 30, 2012 at 10:40 AM, praveenesh kumar <[EMAIL PROTECTED]>
> wrote:
> > Hello Hive Users,
> >
> > There is a strange situation I am facing.
> >
> > I have a string column in my Hive table ( its IP address). I am creating
> a
> > UDF where I am taking this string column and converting it into Long
> value.
> > Its a simple UDF. Following is my code :
> >
> > package com.practice.hive.udf;
> > public class IPtoINT extends UDF {
> > public static LongWritable execute(Text addr) {
> >
> >      String[] addrArray = addr.toString().split("\\.");
> >
> >      long num = 0;
> >
> >      for (int i=0;i<addrArray.length;i++) {
> >           int power = 3-i;
> >             num += ((Integer.parseInt(addrArray[i])%256 *
> > Math.pow(256,power)));
> >      }
> >      return new LongWritable(num);
> >   }
> > }
> >
> > After creating jar, I am running the following commands:
> >
> > $ hive
> > hive > add jar /home/hadoop/Desktop/HiveData/IPtoINT.jar;
> > hive > create temporary function ip2int as
> 'com.practice.hive.udf.IPtoINT';
> > hive > select ip2int(ip1) from sample_data;
> >
> > But running the above, is giving me the following error:
> >
> > java.lang.RuntimeException:
> > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
> while
> > processing row
> > {"ip1":"1.0.144.36","ip2":16814116,"country":"Thailand","key":null}
> >                 at
> > org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
> >                 at
> org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> >                 at
> > org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)
> >                 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> >                 at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >                 at java.security.AccessController.doPrivileged(Native
> > Method)
> >                 at javax.security.auth.Subject.doAs(Subject.java:415)
> >                 at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> >                 at org.apache.hadoop.mapred.Child.main(Child.java:249)
> > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> > Error while processing row
> > {"ip1":"1.0.144.36","ip2":16814116,"country":"Thailand","key":null}
> >                 at
> > org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550)
> >                 at
> > org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
> >                 ... 8 more
> > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to
> > execute method public org.apache.hadoop.io.Text
> > com.musigma.hive.udf.ip2int.evaluate(org.apache.hadoop.io.Text)  on
> object
> > com.musigma.hive.udf.ip2int@19a4d79 of class com.musigma.hive.udf.ip2int
> > with arguments {1.0.144.36:org.apache.hadoop.io.Text} of size 1
> >                 at
> >
> org.apache.hadoop.hive.ql.exec.FunctionRegistry.invoke(FunctionRegistry.java:848)
> >                 at
> >
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.evaluate(GenericUDFBridge.java:181)
> >                 at
> >
> org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.evaluate(ExprNodeGenericFuncEvaluator.java:163)
> >                 at
> >
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:76)
> >                 at
> > org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> >                 at
> > org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)

Nitin Pawar