Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # dev - A major addition to Pig. Working with spatial data


+
Ahmed Eldawy 2013-05-02, 00:40
+
Alan Gates 2013-05-02, 00:50
+
Ahmed Eldawy 2013-05-02, 01:08
+
Alan Gates 2013-05-02, 17:33
+
Ahmed Eldawy 2013-05-02, 18:59
+
Daniel Dai 2013-05-03, 23:09
+
Russell Jurney 2013-05-04, 09:14
+
Jonathan Coveney 2013-05-04, 21:47
+
Ahmed Eldawy 2013-05-05, 23:44
Copy link to this message
-
Re: A major addition to Pig. Working with spatial data
Jonathan Coveney 2013-05-06, 14:53
You can give them all the same label or tag and filter on that later on.
2013/5/6 Ahmed Eldawy <[EMAIL PROTECTED]>

> Thanks all for taking the time to respond. Danial, I didn't know that Solr
> uses JTS. This is a good finding and we can definitely ask them to see if
> there is a work around we can do. Jonathan, I thought of the same idea of
> serializing/deserializing a bytearray each time a UDF is called. The
> deserialization part is good for letting Pig auto detect spatial types if
> not set explicitly in the schema. What is the best way to start this? I
> want to add an initial set of JIRA issues and start working on them but I
> also need to keep the work grouped in some sense just for organization.
>
> Thanks
> Ahmed
>
> Best regards,
> Ahmed Eldawy
>
>
> On Sat, May 4, 2013 at 4:47 PM, Jonathan Coveney <[EMAIL PROTECTED]>
> wrote:
>
> > I agree that this is cool, and if other projects are using JTS it is
> worth
> > talking them to see how. I also agree that licensing is very frustrating.
> >
> > In the short term, however, while it is annoying to have to manage the
> > serialization and deserialization yourself, you can have the geometry
> type
> > be passed around as a bytearray type. Your UDF's will have to know this
> and
> > treat it accordingly, but if you did this then all of the tools could be
> in
> > an external project on github instead of a branch in Pig. Then, if we can
> > get the licensing done, we could add the Geometry type to Pig. Adding
> > types, honestly, is kind of tedious but not super difficult, so once the
> > rest is done, that shouldn't be too difficult.
> >
> >
> > 2013/5/4 Russell Jurney <[EMAIL PROTECTED]>
> >
> > > If a way could be found, this would be an awesome addition to Pig.
> > >
> > > Russell Jurney http://datasyndrome.com
> > >
> > > On May 3, 2013, at 4:09 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:
> > >
> > > > I am not sure how other Apache projects dealing with it? Seems Solr
> > also
> > > > has some connector to JTS?
> > > >
> > > > Thanks,
> > > > Daniel
> > > >
> > > >
> > > > On Thu, May 2, 2013 at 11:59 AM, Ahmed Eldawy <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > >> Thanks Alan for your interest. It's too bad that an open source
> > > licensing
> > > >> issue is holding me back from doing some open source work. I
> > understand
> > > the
> > > >> issue and your workarounds make sense. However, as I mentioned in
> the
> > > >> beginning, I don't want to have my own branch of Pig because it
> makes
> > my
> > > >> extension less portable. I'll think of another way to do it. I'll
> ask
> > > vivid
> > > >> solutions if they can double license their code although I think the
> > > answer
> > > >> will be no. I'll also think of a way to ship my extension as a set
> of
> > > jar
> > > >> files without the need to change the core of Pig. This way, it can
> be
> > > >> easily ported to newer versions of Pig.
> > > >>
> > > >> Thanks
> > > >> Ahmed
> > > >>
> > > >> Best regards,
> > > >> Ahmed Eldawy
> > > >>
> > > >>
> > > >> On Thu, May 2, 2013 at 12:33 PM, Alan Gates <[EMAIL PROTECTED]>
> > > wrote:
> > > >>
> > > >>> I know this is frustrating, but the different licenses do have
> > > different
> > > >>> requirements that make it so that Apache can't ship GPL code.  A
> > legal
> > > >>> explanation is at
> > > http://www.apache.org/licenses/GPL-compatibility.htmlFor additional
> info
> > > on the LGPL specific questions see
> > > >>> http://www.apache.org/legal/3party.html
> > > >>>
> > > >>> As far as pulling it in via ivy, the issue isn't so much where the
> > code
> > > >>> lives as much as what code we are requiring to make Pig work.  If
> > > >> something
> > > >>> that is [L]GPL is required for Pig it violates Apache rules as
> > outlined
> > > >>> above.  It also would be a show stopper for a lot of companies that
> > > >>> redistribute Pig and that are allergic to GPL software.
> > > >>>
> > > >>> So, as I said before, if you wanted to continue with that library
+
Ahmed Eldawy 2013-05-06, 17:12
+
Nick Dimiduk 2013-05-06, 20:03
+
Jonathan Coveney 2013-05-06, 20:09
+
Ahmed Eldawy 2013-05-24, 04:40
+
Ahmed Eldawy 2013-05-29, 15:50
+
Russell Jurney 2013-05-29, 17:17
+
Ahmed Eldawy 2013-06-03, 17:26
+
Russell Jurney 2013-06-03, 17:52