Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> perl code generated from thrift


Copy link to this message
-
Re: perl code generated from thrift
Thanks Ed.  I'll be glad to bark up another tree - just want to make sure
its the most relevant tree! :)   In my case i'm not sure if its a hive or
thrift domain issue i have.

Ultimately though like you suggest if "thrift -gen perl" and "thrift -gen
py" _given the same input_ don't quite generate the analogous code it sure
wouldn't hurt to ask 'em.  :)

Thanks again for your help on this - much appreciated.

Cheers,
Stephen

On Tue, May 29, 2012 at 1:40 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> I never tried to this exactly but are you sure your thrift version
> matches what hive is using? If not I would consider talking to the
> #thrift guys because thrift SHOULD_BE generating bindings for every
> language roughly the same. (How is that for passing the buck?)
>
> Edward
>
> On Tue, May 29, 2012 at 1:34 PM, Stephen Sprague <[EMAIL PROTECTED]>
> wrote:
> > thanks Ed.  Something just isn't quite right.
> >
> > in .$HOME/downloads/hive-0.9.0/src/metastore/if/hive_metastore.thrift
> >  there is a 'struct Schema' entry but all attempts to generate the perl
> > code for that seem to come up empty.
> >
> > looking at -gen py and the associated code i see this code frag in
> > ThriftHive.py
> >
> >        if ftype == TType.STRUCT:
> >          self.success = hive_metastore.ttypes.Schema()
> >          self.success.read(iprot)
> >        else:
> >
> > for perl i only have this:
> >
> >     if ($ftype == TType::STRUCT) {
> >        $self->{success} = new Schema();
> >        $xfer += $self->{success}->read($input);
> >
> > and hence "new Schema" throws the exception of not found.
> >
> >
> > comparing to the generated python code -- it has a whole big ttype.py
> under
> > hive_metastore while the generated perl code has zilch. the perl stuff
> only
> > has ThriftHiveMetastore.pm but no subclasses. :(
> >
> > thoughts, ideas, ruminations?
> >
> > Thanks again Ed.
> >
> > Cheers,
> > Stephen
> >
> > On Tue, May 29, 2012 at 9:00 AM, Edward Capriolo <[EMAIL PROTECTED]
> >wrote:
> >
> >> Hive uses thrift in two places 1) HiveServer 2)Metastore . Search in
> >> the metastore directory for the thrift interface file and compile
> >> that.
> >>
> >> Then you should get a class generated for hive_metastore.Schema
> >>
> >> Edward
> >>
> >>
> >>
> >> On Tue, May 29, 2012 at 11:46 AM, Stephen Sprague <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi Good People,
> >> > I am totally stumped on this one. Wondered if a "perl" bindings chap
> on
> >> > this list might be able to help me.
> >> >
> >> > I generated the ThrftHive.pm class as follows (see below) from my
> >> download
> >> > area.  I've tested some basic examples of querying - and it works
> well.
> >> >  The one thing that i can't get to work, though, is a call to
> >> > "$client->getSchema()" in order to ascertain the schema of a result
> set.
> >> >
> >> > * this built ThriftHIve.pm
> >> >
> >> > $ thrift -r \
> >> >   -I $HOME/downloads/hive-0.9.0/src/ql \
> >> >   -I $HOME/downloads/hive-0.9.0/src \
> >> >   -I $HOME/downloads/thrift-0.8.0 \
> >> > \
> >> >  -gen perl \
> >> > \
> >> >  $HOME/downloads/hive-0.9.0/src/service/if/hive_service.thrift
> >> >
> >> >
> >> >
> >> >
> >> > * in hive_service.thrift here are the relevant lines (that i can tell)
> >> >
> >> > #the includes
> >> >  include "contrib/fb303/if/fb303.thrift"
> >> >  include "metastore/if/hive_metastore.thrift"
> >> >  include "ql/if/queryplan.thrift"
> >> >  namespace java org.apache.hadoop.hive.service
> >> >  namespace cpp Apache.Hadoop.Hive
> >> >
> >> >  <snip>
> >> > #the getSchema() reference
> >> >   service ThriftHive extends hive_metastore.ThriftHiveMetastore {
> >> >   <snip>
> >> >     # Get a schema object with fields represented with native Hive
> types
> >> >     hive_metastore.Schema getSchema() throws(1:HiveServerException ex)
> >> >
> >> >
> >> > * the relevant line in ThriftHive.pm that throws an undefined method
> >> call:
> >> >
> >> > Undefined subroutine &ThriftHive_getSchema_result::Schema called at