Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> A major addition to Pig. Working with spatial data


Copy link to this message
-
Re: A major addition to Pig. Working with spatial data
You can give them all the same label or tag and filter on that later on.
2013/5/6 Ahmed Eldawy <[EMAIL PROTECTED]>

> Thanks all for taking the time to respond. Danial, I didn't know that Solr
> uses JTS. This is a good finding and we can definitely ask them to see if
> there is a work around we can do. Jonathan, I thought of the same idea of
> serializing/deserializing a bytearray each time a UDF is called. The
> deserialization part is good for letting Pig auto detect spatial types if
> not set explicitly in the schema. What is the best way to start this? I
> want to add an initial set of JIRA issues and start working on them but I
> also need to keep the work grouped in some sense just for organization.
>
> Thanks
> Ahmed
>
> Best regards,
> Ahmed Eldawy
>
>
> On Sat, May 4, 2013 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
>
> > I agree that this is cool, and if other projects are using JTS it is
> worth
> > talking them to see how. I also agree that licensing is very frustrating.
> >
> > In the short term, however, while it is annoying to have to manage the
> > serialization and deserialization yourself, you can have the geometry
> type
> > be passed around as a bytearray type. Your UDF's will have to know this
> and
> > treat it accordingly, but if you did this then all of the tools could be
> in
> > an external project on github instead of a branch in Pig. Then, if we can
> > get the licensing done, we could add the Geometry type to Pig. Adding
> > types, honestly, is kind of tedious but not super difficult, so once the
> > rest is done, that shouldn't be too difficult.
> >
> >
> > 2013/5/4 Russell Jurney <[EMAIL PROTECTED]>
> >
> > > If a way could be found, this would be an awesome addition to Pig.
> > >
> > > Russell Jurney http://datasyndrome.com
> > >
> > > On May 3, 2013, at 4:09 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
> > >
> > > > I am not sure how other Apache projects dealing with it? Seems Solr
> > also
> > > > has some connector to JTS?
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > >
> > > > On Thu, May 2, 2013 at 11:59 AM, Ahmed Eldawy <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > >> Thanks Alan for your interest. It's too bad that an open source
> > > licensing
> > > >> issue is holding me back from doing some open source work. I
> > understand
> > > the
> > > >> issue and your workarounds make sense. However, as I mentioned in
> the
> > > >> beginning, I don't want to have my own branch of Pig because it
> makes
> > my
> > > >> extension less portable. I'll think of another way to do it. I'll
> ask
> > > vivid
> > > >> solutions if they can double license their code although I think the
> > > answer
> > > >> will be no. I'll also think of a way to ship my extension as a set
> of
> > > jar
> > > >> files without the need to change the core of Pig. This way, it can
> be
> > > >> easily ported to newer versions of Pig.
> > > >>
> > > >> Thanks
> > > >> Ahmed
> > > >>
> > > >> Best regards,
> > > >> Ahmed Eldawy
> > > >>
> > > >>
> > > >> On Thu, May 2, 2013 at 12:33 PM, Alan Gates <[EMAIL PROTECTED]>
> > > wrote:
> > > >>
> > > >>> I know this is frustrating, but the different licenses do have
> > > different
> > > >>> requirements that make it so that Apache can't ship GPL code.  A
> > legal
> > > >>> explanation is at
> > > http://www.apache.org/licenses/GPL-compatibility.htmlFor additional
> info
> > > on the LGPL specific questions see
> > > >>> http://www.apache.org/legal/3party.html
> > > >>>
> > > >>> As far as pulling it in via ivy, the issue isn't so much where the
> > code
> > > >>> lives as much as what code we are requiring to make Pig work.  If
> > > >> something
> > > >>> that is [L]GPL is required for Pig it violates Apache rules as
> > outlined
> > > >>> above.  It also would be a show stopper for a lot of companies that
> > > >>> redistribute Pig and that are allergic to GPL software.
> > > >>>
> > > >>> So, as I said before, if you wanted to continue with that library
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB