Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - How to wrie the user custemed Load Funtion


Copy link to this message
-
Re: Re: How to wrie the user custemed Load Funtion
Serega Sheypak 2014-01-26, 13:47
Can you:
1. provide source code
2. provide stacktrace
I've seen smilar stacktrace with could not instantiate
'my.pig.stuff.SomeClass' with arguments 'null', sbut the root cause was:
2.1. missing jar with class used in UDF/Load func
2.2. not correcntly handled exception.
2014-01-26 [EMAIL PROTECTED] <[EMAIL PROTECTED]>

> I write a simple LOAD UDF  according to the link and packaged it in the
> jar.
>
> register tracking-0.0.1-SNAPSHOT.jar;
> DEFINE PvDataLoader com.agrantsem.tracking.hadoop.udf.PvDataLoader();
> data = LOAD
> '/user/tracking/pv/log/hourly/[EMAIL PROTECTED]'
> USING PvDataLoader();
> dump data;
>
> pig script failed to validate: java.lang.RuntimeException: could not
> instantiate 'com.agrantsem.tracking.hadoop.udf.PvDataLoader' with arguments
> 'null'
>         at
> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:835)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3236)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1315)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
>         at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
>
> However, there's no arguments in my udf.
> Any insight?
>
> Thanks,
> Lei
>
>
>
>
> [EMAIL PROTECTED]
>
> From: Serega Sheypak
> Date: 2014-01-26 20:04
> To: user
> Subject: Re: How to wrie the user custemed Load Funtion
> Try to use this one as start point:
>
> https://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/PigStorage.java
>
>
> 2014-01-26 [EMAIL PROTECTED] <[EMAIL PROTECTED]>
>
> >
> > Hi,
> >    I want to parse some text file data compressed with .gz format. The
> > data is not neat. The seperator is not uniq and some records are not
> > imcomplete.
> >     Anyone can give an examle of how to write the pig Load UDF?
> >
> > Thanks,
> > Lei
> >
> >
> >
> >
> > [EMAIL PROTECTED]
>