Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # dev - Discuss: End of static, thread local


+
Edward Capriolo 2013-08-10, 17:32
+
Brock Noland 2013-08-10, 22:37
+
Navis류승우 2013-08-11, 04:25
+
Brock Noland 2013-08-11, 17:03
+
Prasad Mujumdar 2013-08-11, 19:34
Copy link to this message
-
Re: Discuss: End of static, thread local
Gunther Hagleitner 2013-08-12, 04:20
+1

That's definitely the right way forward. These thread local variables
should be verboten.

I also agree with Brock's assessment that this will need a design doc
first. My guess is that this will be quite a bit of work.

Thanks,
Gunther.
On Sun, Aug 11, 2013 at 12:34 PM, Prasad Mujumdar <[EMAIL PROTECTED]>wrote:

>    +1 !
> That's should be the right approach IMO too.
>
> thanks
> Prasad
>
>
>
> On Sun, Aug 11, 2013 at 10:03 AM, Brock Noland <[EMAIL PROTECTED]> wrote:
>
> > HIVE-4226 is a very welcome patch but in the long run I think we want to
> > move away from using static thread local everywhere which that patch
> uses.
> > Rightfully so at this juncture.
> >
> > Meaning I think that HIVE-4226 is a good interim solution but long term I
> > think we want to be passing around some kind of context object as Edward
> > suggests in his original mail. This will make the code cleaner and more
> > testable.
> >
> > This is a big effort and will require some analysis and planning before
> > execution.
> >
> >
> > On Sat, Aug 10, 2013 at 11:25 PM, Navis류승우 <[EMAIL PROTECTED]> wrote:
> >
> > > https://issues.apache.org/jira/browse/HIVE-4226 seemed addressing
> this.
> > >
> > > 2013/8/11 Brock Noland <[EMAIL PROTECTED]>:
> > > > I would love to get rid of the static thread local stuff. It was
> > required
> > > > to make hive work in a server model but isn't the correct solution to
> > > this
> > > > problem.
> > > >
> > > > I do think it will be a large amount of work so it'd be great to see
> > > > whoever leads this effort to have a high level plan as opposed to an
> > > adhoc
> > > > effort.
> > > >
> > > >
> > > > On Sat, Aug 10, 2013 at 12:32 PM, Edward Capriolo <
> > [EMAIL PROTECTED]
> > > >wrote:
> > > >
> > > >> I just committed https://issues.apache.org/jira/browse/HIVE-3772.
> > > >>
> > > >> For hive-server2 Carl and others did a lot of work to clean up un
> > thread
> > > >> safe things from hive.
> > > >>
> > > >> Hive was originally build as a fat client so it is not surprising
> that
> > > many
> > > >> such constructs exist. Now since we have retrofitted
> > multi-threaded-ness
> > > >> onto the project we have a number of edge case bugs.
> > > >>
> > > >> My suggestions here would be for that the next release 0.13 we make
> a
> > > push
> > > >> to remove all possible non thread safe code and explicitly pass
> > context
> > > >> objects or serialized structures everywhere thread safety is needed.
> > > >>
> > > >> I can see this would start with something like the Function
> Registry,
> > > this
> > > >> would be a per session object passed around rather then a global
> > object
> > > >> with static hashmap instances in it.
> > > >>
> > > >> I know that this probably will not be as simple as removing all
> static
> > > >> members from our codebase, but does anyone know of specific
> challenges
> > > that
> > > >> will be intrinsically hard to solve?
> > > >>
> > > >> Please comment.
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
> > >
> >
> >
> >
> > --
> > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
> >
>