Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> AvroStorage Issue - Possibly version related


+
j.barrett Strausser 2013-09-19, 16:23
+
Mark Wagner 2013-09-19, 18:24
+
j.barrett Strausser 2013-09-19, 19:14
+
j.barrett Strausser 2013-09-19, 20:10
+
Rohini Palaniswamy 2013-10-01, 00:54
+
j.barrett Strausser 2013-10-01, 01:09
+
Rohini Palaniswamy 2013-10-01, 14:00
Copy link to this message
-
Re: AvroStorage Issue - Possibly version related
Thanks for the heads up and the solution.

I was trying to get pig working with avro in a prototype fashion so I
didn't have any version constraints.

On Tue, Oct 1, 2013 at 10:00 AM, Rohini Palaniswamy <[EMAIL PROTECTED]
> wrote:

> 0.23 has json-simple jar in the hadoop installation, so there is no
> problem. When you go to hadoop 2.x, you will hit the same problem as
> json-simple is not in 2.x hadoop installation. I actually hit the error you
> mentioned with 1.x while trying 2.x.
>
> Regards,
> Rohini
>
>
> On Mon, Sep 30, 2013 at 6:09 PM, j.barrett Strausser <
> [EMAIL PROTECTED]> wrote:
>
> > I ended up just using the .23.9 hadoop release without any issue.
> >
> >
> >
> >
> > On Mon, Sep 30, 2013 at 8:54 PM, Rohini Palaniswamy <
> > [EMAIL PROTECTED]
> > > wrote:
> >
> > >  It hits this error when json-simple-1.1.jar is not in classpath. You
> can
> > > get around that by adding it to PIG_CLASSPATH apart from registering
> the
> > > jar.  The problem is with java classloading where it fails to load the
> > > exception class(ParseException) thrown by a constructor of the class(
> > > AvroStorage) from the custom classloader which includes the registered
> > > jars. Can you file a jira for this? I had spent some time earlier
> trying
> > to
> > > find solution but couldn't. If fixing classloading is not possible,
> easy
> > > thing would be to change AvroStorage constructor to throw
> > RunTimeException
> > > instead of ParseException
> > >
> > > Regards,
> > > Rohini
> > >
> > >
> > > On Thu, Sep 19, 2013 at 1:10 PM, j.barrett Strausser <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > > > Not my day I guess.
> > > >
> > > > Trying with Hadoop 1.2.x
> > > >
> > > > Getting :
> > > >
> > > >
> > > > Caused by: java.lang.RuntimeException: could not instantiate
> > > > 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments
> > 'null'
> > > >         at
> > > >
> > >
> >
> org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:618)
> > > >
> > > > Caused by: java.lang.NoClassDefFoundError:
> > > > org/json/simple/parser/ParseException
> > > >         at java.lang.Class.getDeclaredConstructors0(Native Method)
> > > >
> > > >
> > > >
> > > > When I attempt to load the relation using
> > > >
> > > > Load 'path' USING
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> > > >
> > > > I've registered  :  json-simple-1.1.jar
> > > >
> > > >
> > > > On Thu, Sep 19, 2013 at 3:14 PM, j.barrett Strausser <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > Are the releases from the download page not compatible with 23.x?
> or
> > > 2.X
> > > > >
> > > > > Says they are -
> > > > >
> > > >
> > >
> >
> http://pig.apache.org/releases.html#1+April%2C+2013%3A+release+0.11.1+available
> > > > >
> > > > > In any case I tried it with .23.9 and received a different error:
> > > > >
> > > > > 2013-09-19 15:13:56,044 [main] WARN
> > > > > org.apache.pig.backend.hadoop20.PigJobControl - falling back to
> > default
> > > > > JobControl (not using hadoop 0.20 ?)
> > > > > java.lang.NoSuchFieldException: runnerState
> > > > >     at java.lang.Class.getDeclaredField(Class.java:1938)
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Sep 19, 2013 at 2:24 PM, Mark Wagner <
> > [EMAIL PROTECTED]
> > > > >wrote:
> > > > >
> > > > >> It sounds like you're using a version of Pig that wasn't compiled
> > for
> > > > >> Hadoop 2.x/.23. Try recompiling with 'ant clean jar
> > > > >> -Dhadoopversion=23'.
> > > > >>
> > > > >> -Mark
> > > > >>
> > > > >> On Thu, Sep 19, 2013 at 9:23 AM, j.barrett Strausser
> > > > >> <[EMAIL PROTECTED]> wrote:
> > > > >> > Running
> > > > >> >
> > > > >> > Hadoop-2.1.0-Beta
> > > > >> > Pig-0.11.1
> > > > >> > Hive-0.11.1
> > > > >> >
> > > > >> > 1. Created Avro backed table in Hive.
> > > > >> > 2. Loaded the table in Pig - records = Load '/path' USING
> > > > >> > org.apache.pig.piggybank.storage.avro.AvroStorage();

https://github.com/bearrito
@deepbearrito
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB