Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> How to wrie the user custemed Load Funtion


Copy link to this message
-
Re: Re: How to wrie the user custemed Load Funtion
Can you:
1. provide source code
2. provide stacktrace
I've seen smilar stacktrace with could not instantiate
'my.pig.stuff.SomeClass' with arguments 'null', sbut the root cause was:
2.1. missing jar with class used in UDF/Load func
2.2. not correcntly handled exception.
2014-01-26 [EMAIL PROTECTED] <[EMAIL PROTECTED]>

> I write a simple LOAD UDF  according to the link and packaged it in the
> jar.
>
> register tracking-0.0.1-SNAPSHOT.jar;
> DEFINE PvDataLoader com.agrantsem.tracking.hadoop.udf.PvDataLoader();
> data = LOAD
> '/user/tracking/pv/log/hourly/[EMAIL PROTECTED]'
> USING PvDataLoader();
> dump data;
>
> pig script failed to validate: java.lang.RuntimeException: could not
> instantiate 'com.agrantsem.tracking.hadoop.udf.PvDataLoader' with arguments
> 'null'
>         at
> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:835)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3236)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1315)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
>         at
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
>         at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
>
> However, there's no arguments in my udf.
> Any insight?
>
> Thanks,
> Lei
>
>
>
>
> [EMAIL PROTECTED]
>
> From: Serega Sheypak
> Date: 2014-01-26 20:04
> To: user
> Subject: Re: How to wrie the user custemed Load Funtion
> Try to use this one as start point:
>
> https://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/PigStorage.java
>
>
> 2014-01-26 [EMAIL PROTECTED] <[EMAIL PROTECTED]>
>
> >
> > Hi,
> >    I want to parse some text file data compressed with .gz format. The
> > data is not neat. The seperator is not uniq and some records are not
> > imcomplete.
> >     Anyone can give an examle of how to write the pig Load UDF?
> >
> > Thanks,
> > Lei
> >
> >
> >
> >
> > [EMAIL PROTECTED]
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB