Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # dev - Pig developer meeting in February


+
Olga Natkovich 2011-01-24, 18:14
+
Julien Le Dem 2011-01-26, 22:41
+
Dmitriy Ryaboy 2011-01-26, 23:55
+
Daniel Dai 2011-01-27, 01:59
+
Dmitriy Ryaboy 2011-01-27, 02:04
Copy link to this message
-
RE: Pig developer meeting in February
Olga Natkovich 2011-01-27, 23:17
While there is a lively discussion on this thread, I have not actually gotten any responses to having the meeting with exception of 1 person :).

Please, let me know by the end of the week if you are planning to attend. If we don't get at least a few more responses I suggest we postpone the meeting.

Thanks,

Olga

-----Original Message-----
From: Dmitriy Ryaboy [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, January 26, 2011 6:04 PM
To: [EMAIL PROTECTED]
Subject: Re: Pig developer meeting in February

Right, we do partition filtering, but not true predicate pushdown.

On Wed, Jan 26, 2011 at 5:59 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Are you talking about LoadMetadata.setPartitionFilter?
> PartitionFilterOptimizer will do that.
>
> Daniel
>
>
> Dmitriy Ryaboy wrote:
>
>> I may be wrong but I think predicate pushdown is designed for, but not
>> actually implemented in the current LoadPushdown interface (you can only
>> push projections). If I am wrong, that's great.. but if not, that would be
>> an important feature to add, as people are trying to connect Pig to
>> "smart"
>> storage systems like rdbmses, HBase, and Cassandra more and more.  I think
>> we only kind of simulate this with partition keys info, which is not
>> always
>> sufficient
>>
>> D
>>
>> On Wed, Jan 26, 2011 at 2:41 PM, Julien Le Dem <[EMAIL PROTECTED]>
>> wrote:
>>
>>
>>
>>> If making Pig Thread safe (i.e.: two threads running a different pig
>>> script) is important then we need to change some of the APIs from static
>>> singleton access to a dependency injection pattern.
>>> In that case, this should probably be done before 1.0
>>> For example: UDFContext should be passed to the UDF after construction
>>> (similar to the SevrletContext in Servlet or the way Hadoop passes the
>>> context to tasks)
>>> Also a clearly separated API that does not depend on the Pig
>>> implementation
>>> would help.
>>> For example UDFContext is in org.apache.pig.impl.util when it would be
>>> better in org.apache.pig.api (Or at least an interface defining it)
>>>
>>> Julien
>>>
>>> On 1/24/11 10:14 AM, "Olga Natkovich" <[EMAIL PROTECTED]> wrote:
>>>
>>> Hi Guys,
>>>
>>> I think it is time for us to have another meeting. Yahoo would be happy
>>> to
>>> host if this works for everybody. How about Wednesday, 2/9 4-6 pm.
>>> Please,
>>> let us know if you are planning to attend and if the date/time works for
>>> you.
>>>
>>> Things that come to mind to discuss and as always feel free to suggest
>>> others:
>>>
>>> -          Error handling proposal - this might be easier to finalize
>>> face-to-face
>>> -          Pig 0.9 plan
>>> -          Pig Roadmap beyond 0.9
>>> o        What do we want to do in Pig.next?
>>> o        Are we ready for Pig 1.0
>>>
>>> Olga
>>>
>>>
>>>
>>>
>>
>
+
Dmitriy Ryaboy 2011-01-28, 00:09
+
Julien Le Dem 2011-01-28, 01:21
+
Ashutosh Chauhan 2011-01-28, 10:35
+
Olga Natkovich 2011-01-28, 20:58
+
Santhosh Srinivasan 2011-01-28, 23:35
+
Romain Rigaux 2011-02-04, 18:44
+
Ashutosh Chauhan 2011-01-27, 18:02
+
Dmitriy Ryaboy 2011-01-28, 00:15