Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - A major addition to Pig. Working with spatial data


Copy link to this message
-
Re: A major addition to Pig. Working with spatial data
Russell Jurney 2013-06-03, 17:52
Those JIRAs do best that are completed by one person driving them.
On Mon, Jun 3, 2013 at 10:26 AM, Ahmed Eldawy <[EMAIL PROTECTED]> wrote:

> I've just created a new JIRA issue for the spatial functionality.
> https://issues.apache.org/jira/browse/PIG-3344
> This issue is all about the new datatype which is the only thing that needs
> to be changed internally in Pig in this phase. Pigeon is already working
> with the ESRI library but it converts between binary representation and
> Geometry class back and forth. Once the new datatype is added, we can
> change Pigeon to work with this datatype too. We can still keep the current
> conversion functionality as it allows the system to automatically perform
> the conversion from the bytearray datatype as it adds the autodetect
> functionality when a column is not given a type in the schema.
>
> I don't know if I should provide a patch to this issue myself or there is
> someone else who can work on it. I can of course do it but I think it will
> take me some time to finish as I'm not yet familiar with the internals of
> Pig. Someone who is familiar with the parser would definitely make a better
> job here. I can focus on Pigeon and add more spatial functions there so
> that we can have a plenty of functions once the new datatype is added. I'm
> open to both solutions but I'm just checking with you.
>
> Thanks
> Ahmed
>
> Best regards,
> Ahmed Eldawy
>
>
> On Wed, May 29, 2013 at 12:17 PM, Russell Jurney
> <[EMAIL PROTECTED]>wrote:
>
> > Awesome. This would be a great addition to Pig. Please create a JIRA.
> >
> > Russell Jurney http://datasyndrome.com
> >
> > On May 29, 2013, at 8:51 AM, Ahmed Eldawy <[EMAIL PROTECTED]> wrote:
> >
> > > Hi all,
> > >
> > > Nick has pointed out to me an alternative GIS package that can replace
> > JTS.
> > > ESRI has recently released a GIS
> > > package<https://github.com/Esri/geometry-api-java>under Apache
> > > license. I changed Pigeon to work with that new package. I
> > > think it could be easier now to integrate this work with main branch of
> > > Apache Pig. I will go on with the current project and add more spatial
> > > functionality. We can then add a new datatype to Apache and link it to
> > > those functions.
> > >
> > > ESRI package contains a class OGCGeometry
> > > <
> >
> http://esri.github.io/geometry-api-java/javadoc/com/esri/core/geometry/ogc/OGCGeometry.html
> > >which
> > > can be linked to a new datatype 'Geometry'. Do you think we can rely on
> > the
> > > new package and integrate the work with Apache Pig?
> > >
> > > On May 23, 2013 11:40 PM, "Ahmed Eldawy" <[EMAIL PROTECTED]> wrote:
> > >
> > >> Hi all,
> > >>  Thanks for your help. I've started the project with a minimal
> > >> functionality as a start. It's currently hosted in github. It is
> > licensed
> > >> under the Apache public license to make it easier to merge with Pig.
> > >> Currently it has only a very few functions. I implemented a function
> > from
> > >> different types of functions (e.g., Aggregate and create). I'll keep
> > adding
> > >> functions and any contributions to the project are welcome. As a
> > beginning,
> > >> I need an ANT build file that runs the tests, compiles and generates a
> > jar
> > >> file. I'm not familiar with ANT so any help in this is encouraged.
> > >> Here's the project home page
> > >> https://github.com/aseldawy/pigeon
> > >>
> > >>
> > >> If you have any comments or suggestion please contact me.
> > >>
> > >>
> > >> Best regards,
> > >> Ahmed Eldawy
> > >>
> > >>
> > >> On Mon, May 6, 2013 at 3:09 PM, Jonathan Coveney <[EMAIL PROTECTED]
> > >wrote:
> > >>
> > >>> Nick: the only issue is that the way types are implemented in Pig
> don't
> > >>> allow us to easily "plug-in" types externally. Adding support for
> that
> > >>> would be cool, but a fair bit of work.
> > >>>
> > >>>
> > >>> 2013/5/6 Nick Dimiduk <[EMAIL PROTECTED]>
> > >>>
> > >>>> I'm to a lawyer, but I see no reason why this cannot be an external
>
Russell Jurney twitter.com/rjurney [EMAIL PROTECTED] datasyndrome.com