Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> A major addition to Pig. Working with spatial data


+
Ahmed Eldawy 2013-05-02, 00:40
+
Alan Gates 2013-05-02, 00:50
+
Ahmed Eldawy 2013-05-02, 01:08
+
Alan Gates 2013-05-02, 17:33
+
Ahmed Eldawy 2013-05-02, 18:59
+
Daniel Dai 2013-05-03, 23:09
+
Russell Jurney 2013-05-04, 09:14
+
Jonathan Coveney 2013-05-04, 21:47
+
Ahmed Eldawy 2013-05-05, 23:44
+
Jonathan Coveney 2013-05-06, 14:53
+
Ahmed Eldawy 2013-05-06, 17:12
+
Nick Dimiduk 2013-05-06, 20:03
+
Jonathan Coveney 2013-05-06, 20:09
+
Ahmed Eldawy 2013-05-24, 04:40
+
Ahmed Eldawy 2013-05-29, 15:50
+
Russell Jurney 2013-05-29, 17:17
+
Ahmed Eldawy 2013-06-03, 17:26
Copy link to this message
-
Re: A major addition to Pig. Working with spatial data
Those JIRAs do best that are completed by one person driving them.
On Mon, Jun 3, 2013 at 10:26 AM, Ahmed Eldawy <[EMAIL PROTECTED]> wrote:

> I've just created a new JIRA issue for the spatial functionality.
> https://issues.apache.org/jira/browse/PIG-3344
> This issue is all about the new datatype which is the only thing that needs
> to be changed internally in Pig in this phase. Pigeon is already working
> with the ESRI library but it converts between binary representation and
> Geometry class back and forth. Once the new datatype is added, we can
> change Pigeon to work with this datatype too. We can still keep the current
> conversion functionality as it allows the system to automatically perform
> the conversion from the bytearray datatype as it adds the autodetect
> functionality when a column is not given a type in the schema.
>
> I don't know if I should provide a patch to this issue myself or there is
> someone else who can work on it. I can of course do it but I think it will
> take me some time to finish as I'm not yet familiar with the internals of
> Pig. Someone who is familiar with the parser would definitely make a better
> job here. I can focus on Pigeon and add more spatial functions there so
> that we can have a plenty of functions once the new datatype is added. I'm
> open to both solutions but I'm just checking with you.
>
> Thanks
> Ahmed
>
> Best regards,
> Ahmed Eldawy
>
>
> On Wed, May 29, 2013 at 12:17 PM, Russell Jurney
> <[EMAIL PROTECTED]>wrote:
>
> > Awesome. This would be a great addition to Pig. Please create a JIRA.
> >
> > Russell Jurney http://datasyndrome.com
> >
> > On May 29, 2013, at 8:51 AM, Ahmed Eldawy <[EMAIL PROTECTED]> wrote:
> >
> > > Hi all,
> > >
> > > Nick has pointed out to me an alternative GIS package that can replace
> > JTS.
> > > ESRI has recently released a GIS
> > > package<https://github.com/Esri/geometry-api-java>under Apache
> > > license. I changed Pigeon to work with that new package. I
> > > think it could be easier now to integrate this work with main branch of
> > > Apache Pig. I will go on with the current project and add more spatial
> > > functionality. We can then add a new datatype to Apache and link it to
> > > those functions.
> > >
> > > ESRI package contains a class OGCGeometry
> > > <
> >
> http://esri.github.io/geometry-api-java/javadoc/com/esri/core/geometry/ogc/OGCGeometry.html
> > >which
> > > can be linked to a new datatype 'Geometry'. Do you think we can rely on
> > the
> > > new package and integrate the work with Apache Pig?
> > >
> > > On May 23, 2013 11:40 PM, "Ahmed Eldawy" <[EMAIL PROTECTED]> wrote:
> > >
> > >> Hi all,
> > >>  Thanks for your help. I've started the project with a minimal
> > >> functionality as a start. It's currently hosted in github. It is
> > licensed
> > >> under the Apache public license to make it easier to merge with Pig.
> > >> Currently it has only a very few functions. I implemented a function
> > from
> > >> different types of functions (e.g., Aggregate and create). I'll keep
> > adding
> > >> functions and any contributions to the project are welcome. As a
> > beginning,
> > >> I need an ANT build file that runs the tests, compiles and generates a
> > jar
> > >> file. I'm not familiar with ANT so any help in this is encouraged.
> > >> Here's the project home page
> > >> https://github.com/aseldawy/pigeon
> > >>
> > >>
> > >> If you have any comments or suggestion please contact me.
> > >>
> > >>
> > >> Best regards,
> > >> Ahmed Eldawy
> > >>
> > >>
> > >> On Mon, May 6, 2013 at 3:09 PM, Jonathan Coveney <[EMAIL PROTECTED]
> > >wrote:
> > >>
> > >>> Nick: the only issue is that the way types are implemented in Pig
> don't
> > >>> allow us to easily "plug-in" types externally. Adding support for
> that
> > >>> would be cool, but a fair bit of work.
> > >>>
> > >>>
> > >>> 2013/5/6 Nick Dimiduk <[EMAIL PROTECTED]>
> > >>>
> > >>>> I'm to a lawyer, but I see no reason why this cannot be an external
>
Russell Jurney twitter.com/rjurney [EMAIL PROTECTED] datasyndrome.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB