Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Pig developer meeting in February


+
Olga Natkovich 2011-01-24, 18:14
+
Julien Le Dem 2011-01-26, 22:41
+
Dmitriy Ryaboy 2011-01-26, 23:55
+
Daniel Dai 2011-01-27, 01:59
+
Dmitriy Ryaboy 2011-01-27, 02:04
Copy link to this message
-
RE: Pig developer meeting in February
While there is a lively discussion on this thread, I have not actually gotten any responses to having the meeting with exception of 1 person :).

Please, let me know by the end of the week if you are planning to attend. If we don't get at least a few more responses I suggest we postpone the meeting.

Thanks,

Olga

-----Original Message-----
From: Dmitriy Ryaboy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, January 26, 2011 6:04 PM
To: [EMAIL PROTECTED]
Subject: Re: Pig developer meeting in February

Right, we do partition filtering, but not true predicate pushdown.

On Wed, Jan 26, 2011 at 5:59 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Are you talking about LoadMetadata.setPartitionFilter?
> PartitionFilterOptimizer will do that.
>
> Daniel
>
>
> Dmitriy Ryaboy wrote:
>
>> I may be wrong but I think predicate pushdown is designed for, but not
>> actually implemented in the current LoadPushdown interface (you can only
>> push projections). If I am wrong, that's great.. but if not, that would be
>> an important feature to add, as people are trying to connect Pig to
>> "smart"
>> storage systems like rdbmses, HBase, and Cassandra more and more.  I think
>> we only kind of simulate this with partition keys info, which is not
>> always
>> sufficient
>>
>> D
>>
>> On Wed, Jan 26, 2011 at 2:41 PM, Julien Le Dem <[EMAIL PROTECTED]>
>> wrote:
>>
>>
>>
>>> If making Pig Thread safe (i.e.: two threads running a different pig
>>> script) is important then we need to change some of the APIs from static
>>> singleton access to a dependency injection pattern.
>>> In that case, this should probably be done before 1.0
>>> For example: UDFContext should be passed to the UDF after construction
>>> (similar to the SevrletContext in Servlet or the way Hadoop passes the
>>> context to tasks)
>>> Also a clearly separated API that does not depend on the Pig
>>> implementation
>>> would help.
>>> For example UDFContext is in org.apache.pig.impl.util when it would be
>>> better in org.apache.pig.api (Or at least an interface defining it)
>>>
>>> Julien
>>>
>>> On 1/24/11 10:14 AM, "Olga Natkovich" <[EMAIL PROTECTED]> wrote:
>>>
>>> Hi Guys,
>>>
>>> I think it is time for us to have another meeting. Yahoo would be happy
>>> to
>>> host if this works for everybody. How about Wednesday, 2/9 4-6 pm.
>>> Please,
>>> let us know if you are planning to attend and if the date/time works for
>>> you.
>>>
>>> Things that come to mind to discuss and as always feel free to suggest
>>> others:
>>>
>>> -          Error handling proposal - this might be easier to finalize
>>> face-to-face
>>> -          Pig 0.9 plan
>>> -          Pig Roadmap beyond 0.9
>>> o        What do we want to do in Pig.next?
>>> o        Are we ready for Pig 1.0
>>>
>>> Olga
>>>
>>>
>>>
>>>
>>
>
+
Dmitriy Ryaboy 2011-01-28, 00:09
+
Julien Le Dem 2011-01-28, 01:21
+
Ashutosh Chauhan 2011-01-28, 10:35
+
Olga Natkovich 2011-01-28, 20:58
+
Santhosh Srinivasan 2011-01-28, 23:35
+
Romain Rigaux 2011-02-04, 18:44
+
Ashutosh Chauhan 2011-01-27, 18:02
+
Dmitriy Ryaboy 2011-01-28, 00:15
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB