Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Variable support in Amazon's Elastic MapReduce version of Hive


Copy link to this message
-
Re: Variable support in Amazon's Elastic MapReduce version of Hive
Thanks guys!

When I think about it, it may be good enough to do this at the CLI level as
that is probably the most common use case for this (in most of the other
"API" style modes the apps can dynamically generate queries as they need).
That way the parser does not have to be overloaded with too many assumptions
around this concept.

We should take a look at current "Parameter
Substitution<http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#Parameter+Substitution>"
feature in Pig. It seems pretty comprehensive. I'm not familiar enough with
the code to even venture a guess as to how much of that code would be
reusable.

On Wed, Dec 30, 2009 at 7:31 AM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> I see two ways to do this. we can do the variable substitution at the
> CLI level. Or we can do this at the query processor level.
>
> In each case the variables would be set into the SessionState and the
> respective component could do the substitution.
>
> I think having the query processor handle this would be better.
>
> If we don't here back in a few I will gladly do this as I can leverage
> this as well.
>
> Edward
> On Tue, Dec 29, 2009 at 8:57 PM, Zheng Shao <[EMAIL PROTECTED]> wrote:
> > Hi Vijay,
> >
> > I sent out an inquiry to the guys at aws on 12/21. There is no reply
> > yet. It might be that people are on vacation.
> > Let's wait a bit to see if they can contribute that back to open-source.
> >
> > Zheng
> >
> > On Tue, Dec 29, 2009 at 5:10 PM, Vijay <[EMAIL PROTECTED]> wrote:
> >> Sorry to bump the thread again. I thought this was lost during the
> holidays.
> >> Anybody have any ideas about this?
> >>
> >> On Sat, Dec 19, 2009 at 6:29 PM, Vijay <[EMAIL PROTECTED]> wrote:
> >>>
> >>> Amazon Elastic MapReduce version of Hive seems to have a nice feature
> >>> called "Variables." Basically you can define a variable via
> command-line
> >>> while invoking hive with -d DT=2009-12-09 and then refer to the
> variable via
> >>> ${DT} within the hive queries. This could be extremely useful. I can't
> seem
> >>> to find this feature even on trunk. Is this feature currently anywhere
> in
> >>> the roadmap?
> >>
> >>
> >
> >
> >
> > --
> > Yours,
> > Zheng
> >
>