Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # dev >> B[yi]teSize execwork tasks someone could potentially help out with...


Copy link to this message
-
Re: B[yi]teSize execwork tasks someone could potentially help out with...
Ya, just bringing that up again that. Doubt it will be a blocker.

Tim
On Fri, Apr 26, 2013 at 10:12 AM, David Alves <[EMAIL PROTECTED]> wrote:

> good point, i'll try and ask the author.
> it's a pretty recent lib so that might be an oversight…
>
> -david
>
> On Apr 26, 2013, at 12:04 PM, Timothy Chen <[EMAIL PROTECTED]> wrote:
>
> > Jacques I think this is the one I emailed you before that has no
> licensing info.
> >
> > Tim
> >
> > Sent from my iPhone
> >
> > On Apr 26, 2013, at 9:30 AM, David Alves <[EMAIL PROTECTED]> wrote:
> >
> >> i've looked through it and looks like it can leverage shared memory,
> which I was looking for anyway.
> >> I also like the way garbage collection works (gc in java also clears
> off-heap).
> >> I'll take a deeper look during the weekend.
> >>
> >> -david
> >>
> >> On Apr 26, 2013, at 11:25 AM, Jacques Nadeau <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> I've looked at that in the past and think the idea of using here is
> very
> >>> good.  It seems like ByteBuf is nice as it has things like endianess
> >>> capabilities, reference counting and management and Netty direct
> support.
> >>> On the flipside, larray is nice for its large array capabilities and
> >>> better input/output interfaces.  The best approach might be to define
> a new
> >>> ByteBuf implementation that leverages LArray.  I'll take a look at
> this in
> >>> a few days if someone else doesn't want to.
> >>>
> >>> j
> >>>
> >>> On Fri, Apr 26, 2013 at 8:39 AM, kishore g <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> Fort *ByteBuf Improvements*, Have you looked at LArrayJ
> >>>> https://github.com/xerial/larray. It has those wrappers and I found
> it
> >>>> quite useful. The same person has also written java version for snappy
> >>>> compression. Not sure if you guys have plan to add compression, but
> one of
> >>>> the nice things I could do was use the memory offsets for
> source(compressed
> >>>> data) and dest(uncompressed array) and do the decompression off-heap.
> It
> >>>> supports the need for looking up by index and has wrappers for most
> of the
> >>>> primitive data types.
> >>>>
> >>>> Are you looking at something like this?
> >>>>
> >>>> thanks,
> >>>> Kishore G
> >>>>
> >>>>
> >>>>
> >>>> On Fri, Apr 26, 2013 at 7:53 AM, Jacques Nadeau <[EMAIL PROTECTED]>
> >>>> wrote:
> >>>>
> >>>>> They are on the list but the list is long :)
> >>>>>
> >>>>> Have a good weekend.
> >>>>>
> >>>>> On Thu, Apr 25, 2013 at 9:51 PM, Timothy Chen <[EMAIL PROTECTED]>
> wrote:
> >>>>>
> >>>>>> So if no one picks anything up you will be done with all the work in
> >>>> the
> >>>>>> next couple of days? :)
> >>>>>>
> >>>>>> Would like to help out but I'm traveling to la over the weekend.
> >>>>>>
> >>>>>> I'll sync with you Monday to see how I can help then.
> >>>>>>
> >>>>>> Tim
> >>>>>>
> >>>>>> Sent from my iPhone
> >>>>>>
> >>>>>> On Apr 25, 2013, at 9:06 PM, Jacques Nadeau <[EMAIL PROTECTED]>
> >>>> wrote:
> >>>>>>
> >>>>>>> I'm working on the execwork stuff and if someone would like to help
> >>>>> out,
> >>>>>>> here are a couple of things that need doing.  I figured I'd drop
> them
> >>>>>> here
> >>>>>>> and see if anyone wants to work on them in the next couple of days.
> >>>> If
> >>>>>> so,
> >>>>>>> let me know otherwise I'll be picking them up soon.
> >>>>>>>
> >>>>>>> *RPC*
> >>>>>>> - RPC Layer Handshakes: Currently, I haven't implemented the
> >>>> handshake
> >>>>>> that
> >>>>>>> should happen in either the User <> Bit or the Bit <> Bit layer.
>  The
> >>>>>> plan
> >>>>>>> was to use an additional inserted event handler that removed itself
> >>>>> from
> >>>>>>> the event pipeline after a successful handshake or disconnected the
> >>>>>> channel
> >>>>>>> on a failed handshake (with appropriate logging).  The main
> >>>> validation
> >>>>> at
> >>>>>>> this point will be simply confirming that both endpoints are
> running
> >>>> on
> >>>>>> the
> >>>>>>> same protocol version.   The only other information that is
> currently