Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill, mail # dev - B[yi]teSize execwork tasks someone could potentially help out with...


Copy link to this message
-
Re: B[yi]teSize execwork tasks someone could potentially help out with...
Jacques Nadeau 2013-04-27, 19:51
Great news!  Thanks for running that down.

J

On Sat, Apr 27, 2013 at 8:54 AM, kishore g <[EMAIL PROTECTED]> wrote:
> Good news, the author of larray got back and he will add the apache license
> to the source.
>  On Apr 26, 2013 11:13 AM, "kishore g" <[EMAIL PROTECTED]> wrote:
>
>> I have interacted with the Author, let me know if you want me to check.
>> Good thing was that he is responsive and even added few things for me.
>>
>>
>> On Fri, Apr 26, 2013 at 10:27 AM, Timothy Chen <[EMAIL PROTECTED]> wrote:
>>
>>> Ya, just bringing that up again that. Doubt it will be a blocker.
>>>
>>> Tim
>>>
>>>
>>> On Fri, Apr 26, 2013 at 10:12 AM, David Alves <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>> > good point, i'll try and ask the author.
>>> > it's a pretty recent lib so that might be an oversight…
>>> >
>>> > -david
>>> >
>>> > On Apr 26, 2013, at 12:04 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:
>>> >
>>> > > Jacques I think this is the one I emailed you before that has no
>>> > licensing info.
>>> > >
>>> > > Tim
>>> > >
>>> > > Sent from my iPhone
>>> > >
>>> > > On Apr 26, 2013, at 9:30 AM, David Alves <[EMAIL PROTECTED]>
>>> wrote:
>>> > >
>>> > >> i've looked through it and looks like it can leverage shared memory,
>>> > which I was looking for anyway.
>>> > >> I also like the way garbage collection works (gc in java also clears
>>> > off-heap).
>>> > >> I'll take a deeper look during the weekend.
>>> > >>
>>> > >> -david
>>> > >>
>>> > >> On Apr 26, 2013, at 11:25 AM, Jacques Nadeau <[EMAIL PROTECTED]>
>>> > wrote:
>>> > >>
>>> > >>> I've looked at that in the past and think the idea of using here is
>>> > very
>>> > >>> good.  It seems like ByteBuf is nice as it has things like endianess
>>> > >>> capabilities, reference counting and management and Netty direct
>>> > support.
>>> > >>> On the flipside, larray is nice for its large array capabilities and
>>> > >>> better input/output interfaces.  The best approach might be to
>>> define
>>> > a new
>>> > >>> ByteBuf implementation that leverages LArray.  I'll take a look at
>>> > this in
>>> > >>> a few days if someone else doesn't want to.
>>> > >>>
>>> > >>> j
>>> > >>>
>>> > >>> On Fri, Apr 26, 2013 at 8:39 AM, kishore g <[EMAIL PROTECTED]>
>>> > wrote:
>>> > >>>
>>> > >>>> Fort *ByteBuf Improvements*, Have you looked at LArrayJ
>>> > >>>> https://github.com/xerial/larray. It has those wrappers and I
>>> found
>>> > it
>>> > >>>> quite useful. The same person has also written java version for
>>> snappy
>>> > >>>> compression. Not sure if you guys have plan to add compression, but
>>> > one of
>>> > >>>> the nice things I could do was use the memory offsets for
>>> > source(compressed
>>> > >>>> data) and dest(uncompressed array) and do the decompression
>>> off-heap.
>>> > It
>>> > >>>> supports the need for looking up by index and has wrappers for most
>>> > of the
>>> > >>>> primitive data types.
>>> > >>>>
>>> > >>>> Are you looking at something like this?
>>> > >>>>
>>> > >>>> thanks,
>>> > >>>> Kishore G
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> On Fri, Apr 26, 2013 at 7:53 AM, Jacques Nadeau <
>>> [EMAIL PROTECTED]>
>>> > >>>> wrote:
>>> > >>>>
>>> > >>>>> They are on the list but the list is long :)
>>> > >>>>>
>>> > >>>>> Have a good weekend.
>>> > >>>>>
>>> > >>>>> On Thu, Apr 25, 2013 at 9:51 PM, Timothy Chen <[EMAIL PROTECTED]>
>>> > wrote:
>>> > >>>>>
>>> > >>>>>> So if no one picks anything up you will be done with all the
>>> work in
>>> > >>>> the
>>> > >>>>>> next couple of days? :)
>>> > >>>>>>
>>> > >>>>>> Would like to help out but I'm traveling to la over the weekend.
>>> > >>>>>>
>>> > >>>>>> I'll sync with you Monday to see how I can help then.
>>> > >>>>>>
>>> > >>>>>> Tim
>>> > >>>>>>
>>> > >>>>>> Sent from my iPhone
>>> > >>>>>>
>>> > >>>>>> On Apr 25, 2013, at 9:06 PM, Jacques Nadeau <[EMAIL PROTECTED]>
>>> > >>>> wrote:
>>> > >>>>>>
>>> > >>>>>>> I'm working on the execwork stuff and if someone would like to
>>> help
>>> > >>>>> out,