Hive, mail # user - Re: Passing date as command line arguments

Re: Passing date as command line arguments

Matt Tucker 2012-08-04, 02:15

Hi,

In the command line, you want to wrap 20120709 in double-quotes, as they get stripped when being passed into the hiveconf variable.

Matt

On Aug 3, 2012, at 6:48 PM, Techy Teck <[EMAIL PROTECTED]> wrote:

> I have my below query in test1.hql file. I am trying to pass the date (dt) as the command line argument.
>
>
> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>
>
> So whenever I try to run the above test1.hql file like this-
>
> hive -f hivetest1.hql -hiveconf start_date=20120709
>
> I get zero records back. But the data is there in that table for that date. Why is it so? Something wrong I am doing?
>
> Can anyone help me out here? I was following Bejoy's Article here http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>
>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 06:01

I tried that. But it's not working for me. I didn't got any result.

Any other suggestion?
On Fri, Aug 3, 2012 at 7:41 PM, Matt Tucker <[EMAIL PROTECTED]> wrote:

> Yes, that should do the trick for you.
>
>
>
> On Aug 3, 2012, at 10:17 PM, Techy Teck <[EMAIL PROTECTED]> wrote:
>
> So you are saying, I should invoke like this below?
>
> * hive -f hivetest1.hql -hiveconf start_date="20120709"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 7:15 PM, Matt Tucker <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> In the command line, you want to wrap 20120709 in double-quotes, as they
>> get stripped when being passed into the hiveconf variable.
>>
>> Matt
>>
>>
>>
>> On Aug 3, 2012, at 6:48 PM, Techy Teck <[EMAIL PROTECTED]> wrote:
>>
>> I have my below query in *test1.hql file. *I am trying to pass the date *
>> (dt)* as the command line argument.
>>
>>
>> *select * from lip_data_quality where dt = '${hiveconf: start_date}';*
>>
>>
>> So whenever I try to run the above *test1.hql file like this-*
>> *
>> *
>> * hive -f hivetest1.hql -hiveconf start_date=20120709*
>> *
>> *
>> I get zero records back. But the data is there in that table for that
>> date. Why is it so? Something wrong I am doing?
>>
>> Can anyone help me out here? I was following Bejoy's Article here
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>
>>
>>
>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 06:06

Can anyone help me out on this?

I am stuck on this and I have no clue what wrong I am doing?
On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]> wrote:

> I have my below query in *test1.hql file. *I am trying to pass the date *
> (dt)* as the command line argument.
>
>
> *select * from lip_data_quality where dt = '${hiveconf: start_date}';*
>
>
> So whenever I try to run the above *test1.hql file like this-*
> *
> *
> * hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> I get zero records back. But the data is there in that table for that
> date. Why is it so? Something wrong I am doing?
>
> Can anyone help me out here? I was following Bejoy's Article here
> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>
>
>

Re: Passing date as command line arguments

Vijay 2012-08-04, 06:32

There can't be a space in the variable name. Try this: ${hiveconf:start_date}

On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]> wrote:
> Can anyone help me out on this?
>
> I am stuck on this and I have no clue what wrong I am doing?
>
>
>
>
> On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]> wrote:
>>
>> I have my below query in test1.hql file. I am trying to pass the date (dt)
>> as the command line argument.
>>
>>
>> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>
>>
>> So whenever I try to run the above test1.hql file like this-
>>
>> hive -f hivetest1.hql -hiveconf start_date=20120709
>>
>> I get zero records back. But the data is there in that table for that
>> date. Why is it so? Something wrong I am doing?
>>
>> Can anyone help me out here? I was following Bejoy's Article here
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>
>>
>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 06:41

Thanks Vijay for the suggestion. I also tried that and it still didn't
worked for me.

Any thoughts why it is not working?
On Fri, Aug 3, 2012 at 11:32 PM, Vijay <[EMAIL PROTECTED]> wrote:

> There can't be a space in the variable name. Try this:
> ${hiveconf:start_date}
>
> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]>
> wrote:
> > Can anyone help me out on this?
> >
> > I am stuck on this and I have no clue what wrong I am doing?
> >
> >
> >
> >
> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]>
> wrote:
> >>
> >> I have my below query in test1.hql file. I am trying to pass the date
> (dt)
> >> as the command line argument.
> >>
> >>
> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
> >>
> >>
> >> So whenever I try to run the above test1.hql file like this-
> >>
> >> hive -f hivetest1.hql -hiveconf start_date=20120709
> >>
> >> I get zero records back. But the data is there in that table for that
> >> date. Why is it so? Something wrong I am doing?
> >>
> >> Can anyone help me out here? I was following Bejoy's Article here
> >>
> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
> >>
> >>
> >
>

Re: Passing date as command line arguments

Bejoy Ks 2012-08-04, 06:52

Hi Techy

Are you getting the results for the same query directly from CLI, I mean does the below query return correct results
select * from lip_data_quality where dt = '20120709';
If the above query returns results, can you check the hive logs and see what is the actual query being executed?

Regards,
Bejoy KS
________________________________
From: Techy Teck <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc: Vijay Dirisala <[EMAIL PROTECTED]>
Sent: Saturday, August 4, 2012 12:11 PM
Subject: Re: Passing date as command line arguments

Thanks Vijay for the suggestion. I also tried that and it still didn't worked for me.

Any thoughts why it is not working?

On Fri, Aug 3, 2012 at 11:32 PM, Vijay <[EMAIL PROTECTED]> wrote:

There can't be a space in the variable name. Try this: ${hiveconf:start_date}
>
>
>On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]> wrote:
>> Can anyone help me out on this?
>>
>> I am stuck on this and I have no clue what wrong I am doing?
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]> wrote:
>>>
>>> I have my below query in test1.hql file. I am trying to pass the date (dt)
>>> as the command line argument.
>>>
>>>
>>> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>>>
>>>
>>> So whenever I try to run the above test1.hql file like this-
>>>
>>> hive -f hivetest1.hql -hiveconf start_date=20120709
>>>
>>> I get zero records back. But the data is there in that table for that
>>> date. Why is it so? Something wrong I am doing?
>>>
>>> Can anyone help me out here? I was following Bejoy's Article here
>>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>>>
>>>
>>
>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 06:58

Yes that query gives me the result back correctly.

How can I see the hive log Bejoy? Basically I am not sure which hive log
and what is the path where I can see that thing?
On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:

> Hi Techy
>
> Are you getting the results for the same query directly from CLI, I mean
> does the below query return correct results
> select * from lip_data_quality where dt = '20120709';
>
> If the above query returns results, can you check the hive logs and see
> what is the actual query being executed?
>
> Regards,
> Bejoy KS
>
> ------------------------------
> *From:* Techy Teck <[EMAIL PROTECTED]>
> *To:* [EMAIL PROTECTED]
> *Cc:* Vijay Dirisala <[EMAIL PROTECTED]>
> *Sent:* Saturday, August 4, 2012 12:11 PM
> *Subject:* Re: Passing date as command line arguments
>
> Thanks Vijay for the suggestion. I also tried that and it still didn't
> worked for me.
>
> Any thoughts why it is not working?
>
>
>
>
> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <[EMAIL PROTECTED]> wrote:
>
> There can't be a space in the variable name. Try this:
> ${hiveconf:start_date}
>
> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]>
> wrote:
> > Can anyone help me out on this?
> >
> > I am stuck on this and I have no clue what wrong I am doing?
> >
> >
> >
> >
> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]>
> wrote:
> >>
> >> I have my below query in test1.hql file. I am trying to pass the date
> (dt)
> >> as the command line argument.
> >>
> >>
> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
> >>
> >>
> >> So whenever I try to run the above test1.hql file like this-
> >>
> >> hive -f hivetest1.hql -hiveconf start_date=20120709
> >>
> >> I get zero records back. But the data is there in that table for that
> >> date. Why is it so? Something wrong I am doing?
> >>
> >> Can anyone help me out here? I was following Bejoy's Article here
> >>
> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
> >>
> >>
> >
>
>
>
>
>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 07:07

I found this thing from the Hive log. This is what you are looking for?-

*SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
*QueryStart QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
TIME="1344063597488"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344063597501"*
*QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
*7503"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344063597504"*
*
*
*
*

On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]> wrote:

> Yes that query gives me the result back correctly.
>
> How can I see the hive log Bejoy? Basically I am not sure which hive log
> and what is the path where I can see that thing?
>
>
>
>
>
>
> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
>
>> Hi Techy
>>
>> Are you getting the results for the same query directly from CLI, I mean
>> does the below query return correct results
>> select * from lip_data_quality where dt = '20120709';
>>
>> If the above query returns results, can you check the hive logs and see
>> what is the actual query being executed?
>>
>> Regards,
>> Bejoy KS
>>
>> ------------------------------
>> *From:* Techy Teck <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]
>> *Cc:* Vijay Dirisala <[EMAIL PROTECTED]>
>> *Sent:* Saturday, August 4, 2012 12:11 PM
>> *Subject:* Re: Passing date as command line arguments
>>
>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>> worked for me.
>>
>> Any thoughts why it is not working?
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <[EMAIL PROTECTED]> wrote:
>>
>> There can't be a space in the variable name. Try this:
>> ${hiveconf:start_date}
>>
>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]>
>> wrote:
>> > Can anyone help me out on this?
>> >
>> > I am stuck on this and I have no clue what wrong I am doing?
>> >
>> >
>> >
>> >
>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >> I have my below query in test1.hql file. I am trying to pass the date
>> (dt)
>> >> as the command line argument.
>> >>
>> >>
>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>> >>
>> >>
>> >> So whenever I try to run the above test1.hql file like this-
>> >>
>> >> hive -f hivetest1.hql -hiveconf start_date=20120709
>> >>
>> >> I get zero records back. But the data is there in that table for that
>> >> date. Why is it so? Something wrong I am doing?
>> >>
>> >> Can anyone help me out here? I was following Bejoy's Article here
>> >>
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>> >>
>> >>
>> >
>>
>>
>>
>>
>>
>

Re: Passing date as command line arguments

Bejoy KS 2012-08-04, 07:16

Yes.

From the logs the query being executed is
select * from lip_data_quality where dt=20120709

But here the dt is not in quotes. It should be like
select * from lip_data_quality where dt='20120709';

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <[EMAIL PROTECTED]>
Date: Sat, 4 Aug 2012 00:07:07
To: <[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Passing date as command line arguments

I found this thing from the Hive log. This is what you are looking for?-

*SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
*QueryStart QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
TIME="1344063597488"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344063597501"*
*QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
*7503"*
*Counters
plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
* from lip_data_quality where dt=20120709"},"queryCo*
*unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344063597504"*
*
*
*
*

On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]> wrote:

> Yes that query gives me the result back correctly.
>
> How can I see the hive log Bejoy? Basically I am not sure which hive log
> and what is the path where I can see that thing?
>
>
>
>
>
>
> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
>
>> Hi Techy
>>
>> Are you getting the results for the same query directly from CLI, I mean
>> does the below query return correct results
>> select * from lip_data_quality where dt = '20120709';
>>
>> If the above query returns results, can you check the hive logs and see
>> what is the actual query being executed?
>>
>> Regards,
>> Bejoy KS
>>
>> ------------------------------
>> *From:* Techy Teck <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]
>> *Cc:* Vijay Dirisala <[EMAIL PROTECTED]>
>> *Sent:* Saturday, August 4, 2012 12:11 PM
>> *Subject:* Re: Passing date as command line arguments
>>
>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>> worked for me.
>>
>> Any thoughts why it is not working?
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <[EMAIL PROTECTED]> wrote:
>>
>> There can't be a space in the variable name. Try this:
>> ${hiveconf:start_date}
>>
>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]>
>> wrote:
>> > Can anyone help me out on this?
>> >
>> > I am stuck on this and I have no clue what wrong I am doing?
>> >
>> >
>> >
>> >
>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >> I have my below query in test1.hql file. I am trying to pass the date
>> (dt)
>> >> as the command line argument.
>> >>
>> >>
>> >> select * from lip_data_quality where dt = '${hiveconf: start_date}';
>> >>
>> >>
>> >> So whenever I try to run the above test1.hql file like this-
>> >>
>> >> hive -f hivetest1.hql -hiveconf start_date=20120709
>> >>
>> >> I get zero records back. But the data is there in that table for that
>> >> date. Why is it so? Something wrong I am doing?
>> >>
>> >> Can anyone help me out here? I was following Bejoy's Article here
>> >>
>> http://kickstarthadoop.blogspot.com/2011/10/include-values-during-execution-time-in.html
>> >>
>> >>
>> >
>>
>>
>>
>>
>>
>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 07:21

I was not able to understand. So what changes I need to make? This is my
below *hivetest1.hql-*
*
*
*select * from lip_data_quality where dt = '${hiveconf:start_date}';*
*
*
And by this I am executing the above hql file-

*hive -f hivetest1.hql -hiveconf start_date=20120709*
*
*
*
*
Any thoughts what changes I need to make?

On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Yes.
>
> From the logs the query being executed is
> select * from lip_data_quality where dt=20120709
>
> But here the dt is not in quotes. It should be like
> select * from lip_data_quality where dt='20120709';
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I found this thing from the Hive log. This is what you are looking for?-
>
> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
> *QueryStart QUERY_STRING="select * from lip_data_quality where
> dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> TIME="1344063597488"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344063597501"*
> *QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
> *7503"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344063597504"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]>wrote:
>
>> Yes that query gives me the result back correctly.
>>
>> How can I see the hive log Bejoy? Basically I am not sure which hive log
>> and what is the path where I can see that thing?
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Techy
>>>
>>> Are you getting the results for the same query directly from CLI, I mean
>>> does the below query return correct results
>>> select * from lip_data_quality where dt = '20120709';
>>>
>>> If the above query returns results, can you check the hive logs and see
>>> what is the actual query being executed?
>>>
>>> Regards,
>>> Bejoy KS
>>>
>>> ------------------------------
>>> *From:* Techy Teck <[EMAIL PROTECTED]>
>>> *To:* [EMAIL PROTECTED]
>>> *Cc:* Vijay Dirisala <[EMAIL PROTECTED]>
>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>> *Subject:* Re: Passing date as command line arguments
>>>
>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>> worked for me.
>>>
>>> Any thoughts why it is not working?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <[EMAIL PROTECTED]> wrote:
>>>
>>> There can't be a space in the variable name. Try this:
>>> ${hiveconf:start_date}
>>>
>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]>
>>> wrote:
>>> > Can anyone help me out on this?
>>> >
>>> > I am stuck on this and I have no clue what wrong I am doing?
>>> >
>>> >
>>> >
>>> >
>>> > On Fri, Aug 3, 2012 at 3:48 PM, Techy Teck <[EMAIL PROTECTED]>
>>> wrote:
>>> >>
>>> >> I have my below query in test1.hql file. I am trying to pass the date
>>> (dt)
>>> >> as the command line argument.
>>> >>

Re: Passing date as command line arguments

Bejoy KS 2012-08-04, 07:24

I tried the same query on my end, It is working fine for me without any issues.

By de way the data type for 'dt' is String itself right?

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <[EMAIL PROTECTED]>
Date: Sat, 4 Aug 2012 00:21:37
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Passing date as command line arguments

I was not able to understand. So what changes I need to make? This is my
below *hivetest1.hql-*
*
*
*select * from lip_data_quality where dt = '${hiveconf:start_date}';*
*
*
And by this I am executing the above hql file-

*hive -f hivetest1.hql -hiveconf start_date=20120709*
*
*
*
*
Any thoughts what changes I need to make?

On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Yes.
>
> From the logs the query being executed is
> select * from lip_data_quality where dt=20120709
>
> But here the dt is not in quotes. It should be like
> select * from lip_data_quality where dt='20120709';
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I found this thing from the Hive log. This is what you are looking for?-
>
> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
> *QueryStart QUERY_STRING="select * from lip_data_quality where
> dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> TIME="1344063597488"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344063597501"*
> *QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
> *7503"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344063597504"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]>wrote:
>
>> Yes that query gives me the result back correctly.
>>
>> How can I see the hive log Bejoy? Basically I am not sure which hive log
>> and what is the path where I can see that thing?
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Techy
>>>
>>> Are you getting the results for the same query directly from CLI, I mean
>>> does the below query return correct results
>>> select * from lip_data_quality where dt = '20120709';
>>>
>>> If the above query returns results, can you check the hive logs and see
>>> what is the actual query being executed?
>>>
>>> Regards,
>>> Bejoy KS
>>>
>>> ------------------------------
>>> *From:* Techy Teck <[EMAIL PROTECTED]>
>>> *To:* [EMAIL PROTECTED]
>>> *Cc:* Vijay Dirisala <[EMAIL PROTECTED]>
>>> *Sent:* Saturday, August 4, 2012 12:11 PM
>>> *Subject:* Re: Passing date as command line arguments
>>>
>>> Thanks Vijay for the suggestion. I also tried that and it still didn't
>>> worked for me.
>>>
>>> Any thoughts why it is not working?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:32 PM, Vijay <[EMAIL PROTECTED]> wrote:
>>>
>>> There can't be a space in the variable name. Try this:
>>> ${hiveconf:start_date}
>>>
>>> On Fri, Aug 3, 2012 at 11:06 PM, Techy Teck <[EMAIL PROTECTED]>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 07:26

Yes it is String for me here.

This is the table structure by the way-

*create table lip_data_quality*
*( buyer_id bigint,*
* total_chkout bigint,*
* total_errpds bigint*
* )*
* partitioned by (dt string)*
*row format delimited fields terminated by '\t'*
*stored as sequencefile*
*location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
*;*

in which I am using dt in the where clause in my query. I am running *Hive
0.6.*

On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> I tried the same query on my end, It is working fine for me without any
> issues.
>
> By de way the data type for 'dt' is String itself right?
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I was not able to understand. So what changes I need to make? This is my
> below *hivetest1.hql-*
> *
> *
> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
> *
> *
> And by this I am executing the above hql file-
>
> *hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> *
> *
> Any thoughts what changes I need to make?
>
>
>
>
>
>
>
> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>> Yes.
>>
>> From the logs the query being executed is
>> select * from lip_data_quality where dt=20120709
>>
>> But here the dt is not in quotes. It should be like
>> select * from lip_data_quality where dt='20120709';
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> I found this thing from the Hive log. This is what you are looking for?-
>>
>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> TIME="1344063597488"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344063597501"*
>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>> *7503"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344063597504"*
>> *
>> *
>> *
>> *
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]>wrote:
>>
>>> Yes that query gives me the result back correctly.
>>>
>>> How can I see the hive log Bejoy? Basically I am not sure which hive
>>> log and what is the path where I can see that thing?
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi Techy
>>>>
>>>> Are you getting the results for the same query directly from CLI, I
>>>> mean does the below query return correct results
>>>> select * from lip_data_quality where dt = '20120709';
>>>>
>>>> If the above query returns results, can you check the hive logs and see
>>>> what is the actual query being executed?

Re: Passing date as command line arguments

Bejoy KS 2012-08-04, 07:27

Hi Techy

Which version of hive are you on?

I'm on hive 0.9 and I'm sure I have executed similar scripts in hive 0.7 as well.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: "Bejoy KS" <[EMAIL PROTECTED]>
Date: Sat, 4 Aug 2012 07:24:37
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Passing date as command line arguments

I tried the same query on my end, It is working fine for me without any issues.

By de way the data type for 'dt' is String itself right?

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <[EMAIL PROTECTED]>
Date: Sat, 4 Aug 2012 00:21:37
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Passing date as command line arguments

I was not able to understand. So what changes I need to make? This is my
below *hivetest1.hql-*
*
*
*select * from lip_data_quality where dt = '${hiveconf:start_date}';*
*
*
And by this I am executing the above hql file-

*hive -f hivetest1.hql -hiveconf start_date=20120709*
*
*
*
*
Any thoughts what changes I need to make?

On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Yes.
>
> From the logs the query being executed is
> select * from lip_data_quality where dt=20120709
>
> But here the dt is not in quotes. It should be like
> select * from lip_data_quality where dt='20120709';
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I found this thing from the Hive log. This is what you are looking for?-
>
> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
> *QueryStart QUERY_STRING="select * from lip_data_quality where
> dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> TIME="1344063597488"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344063597501"*
> *QueryEnd QUERY_STRING="select * from lip_data_quality where dt=20120709"
> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
> *7503"*
> *Counters
> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
> * from lip_data_quality where dt=20120709"},"queryCo*
> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344063597504"*
> *
> *
> *
> *
>
>
>
> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]>wrote:
>
>> Yes that query gives me the result back correctly.
>>
>> How can I see the hive log Bejoy? Basically I am not sure which hive log
>> and what is the path where I can see that thing?
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Techy
>>>
>>> Are you getting the results for the same query directly from CLI, I mean
>>> does the below query return correct results
>>> select * from lip_data_quality where dt = '20120709';
>>>
>>> If the above query returns results, can you check the hive logs and see
>>> what is the actual query being executed?
>>>
>>> Regards,
>>> Bejoy KS
>>>
>>> ------------------------------
>>> *From:* Techy Teck <[EMAIL PROTECTED]>
>>> *To:* [EMAIL PROTECTED]
>>> *Cc:* Vijay Dirisala <[EMAIL PROTECTED]>
>>> *Sent:* Saturday, August 4, 2012 12:11 PM

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 07:29

I am on Hive 0.6.

And when I tried executing using like this (*hive -f hivetest1.hql
-hiveconf start_date=20120709*). And after that when I tried viewing the
log file. I found only the below thing that's all. Why? Any idea?

*SessionStart SESSION_ID="rjamal_201208041409" TIME="1344064163296"*
On Sat, Aug 4, 2012 at 12:27 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Hi Techy
>
> Which version of hive are you on?
>
> I'm on hive 0.9 and I'm sure I have executed similar scripts in hive 0.7
> as well.
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * "Bejoy KS" <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 07:24:37 +0000
> *To: *<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I tried the same query on my end, It is working fine for me without any
> issues.
>
> By de way the data type for 'dt' is String itself right?
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I was not able to understand. So what changes I need to make? This is my
> below *hivetest1.hql-*
> *
> *
> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
> *
> *
> And by this I am executing the above hql file-
>
> *hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> *
> *
> Any thoughts what changes I need to make?
>
>
>
>
>
>
>
> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>> Yes.
>>
>> From the logs the query being executed is
>> select * from lip_data_quality where dt=20120709
>>
>> But here the dt is not in quotes. It should be like
>> select * from lip_data_quality where dt='20120709';
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> I found this thing from the Hive log. This is what you are looking for?-
>>
>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> TIME="1344063597488"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344063597501"*
>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>> *7503"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344063597504"*
>> *
>> *
>> *
>> *
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]>wrote:
>>
>>> Yes that query gives me the result back correctly.
>>>
>>> How can I see the hive log Bejoy? Basically I am not sure which hive
>>> log and what is the path where I can see that thing?
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 3, 2012 at 11:52 PM, Bejoy Ks <[EMAIL PROTECTED]

Re: Passing date as command line arguments

Bejoy KS 2012-08-04, 07:29

Try it on a higher version of hive and let me know if that doesn't work still. 0.9 should be good.
Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <[EMAIL PROTECTED]>
Date: Sat, 4 Aug 2012 00:26:44
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Passing date as command line arguments

Yes it is String for me here.

This is the table structure by the way-

*create table lip_data_quality*
*( buyer_id bigint,*
* total_chkout bigint,*
* total_errpds bigint*
* )*
* partitioned by (dt string)*
*row format delimited fields terminated by '\t'*
*stored as sequencefile*
*location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
*;*

in which I am using dt in the where clause in my query. I am running *Hive
0.6.*

On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> I tried the same query on my end, It is working fine for me without any
> issues.
>
> By de way the data type for 'dt' is String itself right?
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I was not able to understand. So what changes I need to make? This is my
> below *hivetest1.hql-*
> *
> *
> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
> *
> *
> And by this I am executing the above hql file-
>
> *hive -f hivetest1.hql -hiveconf start_date=20120709*
> *
> *
> *
> *
> Any thoughts what changes I need to make?
>
>
>
>
>
>
>
> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>> Yes.
>>
>> From the logs the query being executed is
>> select * from lip_data_quality where dt=20120709
>>
>> But here the dt is not in quotes. It should be like
>> select * from lip_data_quality where dt='20120709';
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> I found this thing from the Hive log. This is what you are looking for?-
>>
>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> TIME="1344063597488"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344063597501"*
>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>> dt=20120709"
>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>> *7503"*
>> *Counters
>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>> * from lip_data_quality where dt=20120709"},"queryCo*
>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344063597504"*
>> *
>> *
>> *
>> *
>>
>>
>>
>> On Fri, Aug 3, 2012 at 11:58 PM, Techy Teck <[EMAIL PROTECTED]>wrote:
>>
>>> Yes that query gives me the result back correctly.
>>>
>>> How can I see the hive log Bejoy? Basically I am not sure which hive
>>> log and what is the path where I can see that thing?
>>

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 07:30

I cannot try that on Higher version of Hive as I am working in a company
where they have installed only Hive 0.6 on the Ares cluster.

:(
On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
>
> Try it on a higher version of hive and let me know if that doesn't work
> still. 0.9 should be good.
>
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> Yes it is String for me here.
>
> This is the table structure by the way-
>
> *create table lip_data_quality*
> *( buyer_id bigint,*
> * total_chkout bigint,*
> * total_errpds bigint*
> * )*
> * partitioned by (dt string)*
> *row format delimited fields terminated by '\t'*
> *stored as sequencefile*
> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
> *;*
>
> in which I am using dt in the where clause in my query. I am running *Hive
> 0.6.*
>
>
>
> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>> I tried the same query on my end, It is working fine for me without any
>> issues.
>>
>> By de way the data type for 'dt' is String itself right?
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> I was not able to understand. So what changes I need to make? This is my
>> below *hivetest1.hql-*
>> *
>> *
>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>> *
>> *
>> And by this I am executing the above hql file-
>>
>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>> *
>> *
>> *
>> *
>> Any thoughts what changes I need to make?
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>
>>> **
>>> Yes.
>>>
>>> From the logs the query being executed is
>>> select * from lip_data_quality where dt=20120709
>>>
>>> But here the dt is not in quotes. It should be like
>>> select * from lip_data_quality where dt='20120709';
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
>>> *ReplyTo: * [EMAIL PROTECTED]
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I found this thing from the Hive log. This is what you are looking for?-
>>>
>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>> dt=20120709"
>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>> TIME="1344063597488"*
>>> *Counters
>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>>> TIME="1344063597501"*
>>> *QueryEnd QUERY_STRING="select * from lip_data_quality where
>>> dt=20120709"
>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="134406359*
>>> *7503"*
>>> *Counters
>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 07:35

Hi Bejoy,

I found something very strange. When I actually viewed the Hive log file, I
found something like this- See the thing in red color. Why is it like so?

*SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"*
*QueryStart QUERY_STRING=" select * from lip_data_quality where dt = '
${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
TIME="1344065629740"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344065629743"*
*QueryEnd QUERY_STRING=" select * from lip_data_quality where dt '${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344065629745"*

On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <[EMAIL PROTECTED]> wrote:

> I cannot try that on Higher version of Hive as I am working in a company
> where they have installed only Hive 0.6 on the Ares cluster.
>
> :(
>
>
>
>
> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>>
>> Try it on a higher version of hive and let me know if that doesn't work
>> still. 0.9 should be good.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> Yes it is String for me here.
>>
>> This is the table structure by the way-
>>
>> *create table lip_data_quality*
>> *( buyer_id bigint,*
>> * total_chkout bigint,*
>> * total_errpds bigint*
>> * )*
>> * partitioned by (dt string)*
>> *row format delimited fields terminated by '\t'*
>> *stored as sequencefile*
>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>> *;*
>>
>> in which I am using dt in the where clause in my query. I am running *Hive
>> 0.6.*
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>
>>> **
>>> I tried the same query on my end, It is working fine for me without any
>>> issues.
>>>
>>> By de way the data type for 'dt' is String itself right?
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>>> *ReplyTo: * [EMAIL PROTECTED]
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I was not able to understand. So what changes I need to make? This is my
>>> below *hivetest1.hql-*
>>> *
>>> *
>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>> *
>>> *
>>> And by this I am executing the above hql file-
>>>
>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>>> *
>>> *
>>> *
>>> *
>>> Any thoughts what changes I need to make?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>>
>>>> **
>>>> Yes.
>>>>
>>>> From the logs the query being executed is
>>>> select * from lip_data_quality where dt=20120709
>>>>
>>>> But here the dt is not in quotes. It should be like
>>>> select * from lip_data_quality where dt='20120709';
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.

Re: Passing date as command line arguments

Bejoy KS 2012-08-04, 07:35

You can try it out on a local installation and test it against the later versions.

As I pointed out, I just tested on hive 0.9 and it was working good for me.

I guess you should recommend an upgrade of hive in your cluster as well. Hive has gone far too ahead after 0.6 . :)
Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <[EMAIL PROTECTED]>
Date: Sat, 4 Aug 2012 00:30:41
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Passing date as command line arguments

I cannot try that on Higher version of Hive as I am working in a company
where they have installed only Hive 0.6 on the Ares cluster.

:(
On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
>
> Try it on a higher version of hive and let me know if that doesn't work
> still. 0.9 should be good.
>
>
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> Yes it is String for me here.
>
> This is the table structure by the way-
>
> *create table lip_data_quality*
> *( buyer_id bigint,*
> * total_chkout bigint,*
> * total_errpds bigint*
> * )*
> * partitioned by (dt string)*
> *row format delimited fields terminated by '\t'*
> *stored as sequencefile*
> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
> *;*
>
> in which I am using dt in the where clause in my query. I am running *Hive
> 0.6.*
>
>
>
> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>> I tried the same query on my end, It is working fine for me without any
>> issues.
>>
>> By de way the data type for 'dt' is String itself right?
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> I was not able to understand. So what changes I need to make? This is my
>> below *hivetest1.hql-*
>> *
>> *
>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>> *
>> *
>> And by this I am executing the above hql file-
>>
>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>> *
>> *
>> *
>> *
>> Any thoughts what changes I need to make?
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>
>>> **
>>> Yes.
>>>
>>> From the logs the query being executed is
>>> select * from lip_data_quality where dt=20120709
>>>
>>> But here the dt is not in quotes. It should be like
>>> select * from lip_data_quality where dt='20120709';
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
>>> *ReplyTo: * [EMAIL PROTECTED]
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I found this thing from the Hive log. This is what you are looking for?-
>>>
>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>> dt=20120709"
>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"
>>> TIME="1344063597488"*
>>> *Counters
>>> plan="{"queryId":"rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486","queryType":null,"queryAttributes":{"queryString":"select
>>> * from lip_data_quality where dt=20120709"},"queryCo*
>>> *unters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"

Re: Passing date as command line arguments

Techy Teck 2012-08-04, 07:37

Yup, I will try that out. I send out the latest email. Can you check why is
it happening like this?
On Sat, Aug 4, 2012 at 12:35 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> You can try it out on a local installation and test it against the later
> versions.
>
> As I pointed out, I just tested on hive 0.9 and it was working good for me.
>
> I guess you should recommend an upgrade of hive in your cluster as well.
> Hive has gone far too ahead after 0.6 . :)
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:30:41 -0700
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> I cannot try that on Higher version of Hive as I am working in a company
> where they have installed only Hive 0.6 on the Ares cluster.
>
> :(
>
>
>
>
> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>>
>> Try it on a higher version of hive and let me know if that doesn't work
>> still. 0.9 should be good.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> Yes it is String for me here.
>>
>> This is the table structure by the way-
>>
>> *create table lip_data_quality*
>> *( buyer_id bigint,*
>> * total_chkout bigint,*
>> * total_errpds bigint*
>> * )*
>> * partitioned by (dt string)*
>> *row format delimited fields terminated by '\t'*
>> *stored as sequencefile*
>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>> *;*
>>
>> in which I am using dt in the where clause in my query. I am running *Hive
>> 0.6.*
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>
>>> **
>>> I tried the same query on my end, It is working fine for me without any
>>> issues.
>>>
>>> By de way the data type for 'dt' is String itself right?
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>>> *ReplyTo: * [EMAIL PROTECTED]
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I was not able to understand. So what changes I need to make? This is my
>>> below *hivetest1.hql-*
>>> *
>>> *
>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>> *
>>> *
>>> And by this I am executing the above hql file-
>>>
>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*
>>> *
>>> *
>>> *
>>> *
>>> Any thoughts what changes I need to make?
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:16 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>>
>>>> **
>>>> Yes.
>>>>
>>>> From the logs the query being executed is
>>>> select * from lip_data_quality where dt=20120709
>>>>
>>>> But here the dt is not in quotes. It should be like
>>>> select * from lip_data_quality where dt='20120709';
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ------------------------------
>>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>>> *Date: *Sat, 4 Aug 2012 00:07:07 -0700
>>>> *To: *<[EMAIL PROTECTED]>; Bejoy Ks<[EMAIL PROTECTED]>
>>>> *ReplyTo: * [EMAIL PROTECTED]
>>>> *Subject: *Re: Passing date as command line arguments
>>>>
>>>> I found this thing from the Hive log. This is what you are looking for?-
>>>>
>>>> *SessionStart SESSION_ID="rjamal_201208041359" TIME="1344063565655"*
>>>> *QueryStart QUERY_STRING="select * from lip_data_quality where
>>>> dt=20120709"
>>>> QUERY_ID="rjamal_20120804135959_58e245e8-086a-47c2-9ed3-6b952e4a5486"

Re: Passing date as command line arguments

Bejoy KS 2012-08-04, 07:39

Yes that is the right issue. Variable substitution is not happening. I can't say much here as I haven't tried out this on 0.6 . The code on my blog post is based on 0.7 or higher version I guess.

Regards
Bejoy KS

Sent from handheld, please excuse typos.

-----Original Message-----
From: Techy Teck <[EMAIL PROTECTED]>
Date: Sat, 4 Aug 2012 00:35:20
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Passing date as command line arguments

Hi Bejoy,

I found something very strange. When I actually viewed the Hive log file, I
found something like this- See the thing in red color. Why is it like so?

*SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"*
*QueryStart QUERY_STRING=" select * from lip_data_quality where dt = '
${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
TIME="1344065629740"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
TIME="1344065629743"*
*QueryEnd QUERY_STRING=" select * from lip_data_quality where dt '${hiveconf:start_date}'"
QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"*
*Counters
plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
select * from lip_data_quality where dt '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
TIME="1344065629745"*

On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <[EMAIL PROTECTED]> wrote:

> I cannot try that on Higher version of Hive as I am working in a company
> where they have installed only Hive 0.6 on the Ares cluster.
>
> :(
>
>
>
>
> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>
>> **
>>
>> Try it on a higher version of hive and let me know if that doesn't work
>> still. 0.9 should be good.
>>
>>
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ------------------------------
>> *From: * Techy Teck <[EMAIL PROTECTED]>
>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>> *ReplyTo: * [EMAIL PROTECTED]
>> *Subject: *Re: Passing date as command line arguments
>>
>> Yes it is String for me here.
>>
>> This is the table structure by the way-
>>
>> *create table lip_data_quality*
>> *( buyer_id bigint,*
>> * total_chkout bigint,*
>> * total_errpds bigint*
>> * )*
>> * partitioned by (dt string)*
>> *row format delimited fields terminated by '\t'*
>> *stored as sequencefile*
>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>> *;*
>>
>> in which I am using dt in the where clause in my query. I am running *Hive
>> 0.6.*
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>
>>> **
>>> I tried the same query on my end, It is working fine for me without any
>>> issues.
>>>
>>> By de way the data type for 'dt' is String itself right?
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>>> *ReplyTo: * [EMAIL PROTECTED]
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> I was not able to understand. So what changes I need to make? This is my
>>> below *hivetest1.hql-*
>>> *
>>> *
>>> *select * from lip_data_quality where dt = '${hiveconf:start_date}';*
>>> *
>>> *
>>> And by this I am executing the above hql file-
>>>
>>> *hive -f hivetest1.hql -hiveconf start_date=20120709*

Re: Passing date as command line arguments

Lefty Leverenz 2012-08-04, 07:58

Does Hive 0.6 support underscores in variable names?

Just a thought. The examples I've seen contain letters and dots (".") but
no other characters.

– Lefty
On Sat, Aug 4, 2012 at 12:39 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:

> **
> Yes that is the right issue. Variable substitution is not happening. I
> can't say much here as I haven't tried out this on 0.6 . The code on my
> blog post is based on 0.7 or higher version I guess.
> Regards
> Bejoy KS
>
> Sent from handheld, please excuse typos.
> ------------------------------
> *From: * Techy Teck <[EMAIL PROTECTED]>
> *Date: *Sat, 4 Aug 2012 00:35:20 -0700
> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *Re: Passing date as command line arguments
>
> Hi Bejoy,
>
> I found something very strange. When I actually viewed the Hive log file,
> I found something like this- See the thing in red color. Why is it like so?
>
> *SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"*
> *QueryStart QUERY_STRING=" select * from lip_data_quality where dt = '
> ${hiveconf:start_date}'"
> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
> TIME="1344065629740"*
> *Counters
> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
> select * from lip_data_quality where dt > '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
> TIME="1344065629743"*
> *QueryEnd QUERY_STRING=" select * from lip_data_quality where dt > '${hiveconf:start_date}'"
> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"*
> *Counters
> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
> select * from lip_data_quality where dt > '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
> TIME="1344065629745"*
>
> On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <[EMAIL PROTECTED]>wrote:
>
>> I cannot try that on Higher version of Hive as I am working in a company
>> where they have installed only Hive 0.6 on the Ares cluster.
>>
>> :(
>>
>>
>>
>>
>> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>
>>> **
>>>
>>> Try it on a higher version of hive and let me know if that doesn't work
>>> still. 0.9 should be good.
>>>
>>>
>>> Regards
>>> Bejoy KS
>>>
>>> Sent from handheld, please excuse typos.
>>> ------------------------------
>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>> *Date: *Sat, 4 Aug 2012 00:26:44 -0700
>>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>>> *ReplyTo: * [EMAIL PROTECTED]
>>> *Subject: *Re: Passing date as command line arguments
>>>
>>> Yes it is String for me here.
>>>
>>> This is the table structure by the way-
>>>
>>> *create table lip_data_quality*
>>> *( buyer_id bigint,*
>>> * total_chkout bigint,*
>>> * total_errpds bigint*
>>> * )*
>>> * partitioned by (dt string)*
>>> *row format delimited fields terminated by '\t'*
>>> *stored as sequencefile*
>>> *location '/apps/hdmi-technology/b_apdpds/lip-data-quality'*
>>> *;*
>>>
>>> in which I am using dt in the where clause in my query. I am running *Hive
>>> 0.6.*
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>>
>>>> **
>>>> I tried the same query on my end, It is working fine for me without any
>>>> issues.
>>>>
>>>> By de way the data type for 'dt' is String itself right?
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ------------------------------
>>>> *From: * Techy Teck <[EMAIL PROTECTED]>
>>>> *Date: *Sat, 4 Aug 2012 00:21:37 -0700
>>>> *To: *<[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>

Re: Passing date as command line arguments

Edward Capriolo 2012-08-04, 14:19

Variable support was not added until hive 0.7.0
On Sat, Aug 4, 2012 at 3:58 AM, Lefty Leverenz <[EMAIL PROTECTED]> wrote:
> Does Hive 0.6 support underscores in variable names?
>
> Just a thought. The examples I've seen contain letters and dots (".") but
> no other characters.
>
> – Lefty
>
>
>
>
> On Sat, Aug 4, 2012 at 12:39 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>
>> Yes that is the right issue. Variable substitution is not happening. I
>> can't say much here as I haven't tried out this on 0.6 . The code on my blog
>> post is based on 0.7 or higher version I guess.
>> Regards
>> Bejoy KS
>>
>> Sent from handheld, please excuse typos.
>> ________________________________
>> From: Techy Teck <[EMAIL PROTECTED]>
>> Date: Sat, 4 Aug 2012 00:35:20 -0700
>> To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>> ReplyTo: [EMAIL PROTECTED]
>> Subject: Re: Passing date as command line arguments
>>
>> Hi Bejoy,
>>
>> I found something very strange. When I actually viewed the Hive log file,
>> I found something like this- See the thing in red color. Why is it like so?
>>
>> SessionStart SESSION_ID="rjamal_201208041433" TIME="1344065620956"
>> QueryStart QUERY_STRING=" select * from lip_data_quality where dt >> '${hiveconf:start_date}'"
>> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
>> TIME="1344065629740"
>> Counters
>> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
>> select * from lip_data_quality where dt >> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"false","started":"true"}"
>> TIME="1344065629743"
>> QueryEnd QUERY_STRING=" select * from lip_data_quality where dt >> '${hiveconf:start_date}'"
>> QUERY_ID="rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d"
>> QUERY_RET_CODE="0" QUERY_NUM_TASKS="0" TIME="1344065629745"
>> Counters
>> plan="{"queryId":"rjamal_20120804143333_3719afde-117d-4c6b-a322-0cff6e4b4c5d","queryType":null,"queryAttributes":{"queryString":"
>> select * from lip_data_quality where dt >> '${hiveconf:start_date}'"},"queryCounters":"null","stageGraph":{"nodeType":"1","roots":"null","adjacencyList":"]"},"stageList":"]","done":"true","started":"true"}"
>> TIME="1344065629745"
>>
>> On Sat, Aug 4, 2012 at 12:30 AM, Techy Teck <[EMAIL PROTECTED]>
>> wrote:
>>>
>>> I cannot try that on Higher version of Hive as I am working in a company
>>> where they have installed only Hive 0.6 on the Ares cluster.
>>>
>>> :(
>>>
>>>
>>>
>>>
>>> On Sat, Aug 4, 2012 at 12:29 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>>>
>>>>
>>>> Try it on a higher version of hive and let me know if that doesn't work
>>>> still. 0.9 should be good.
>>>>
>>>>
>>>> Regards
>>>> Bejoy KS
>>>>
>>>> Sent from handheld, please excuse typos.
>>>> ________________________________
>>>> From: Techy Teck <[EMAIL PROTECTED]>
>>>> Date: Sat, 4 Aug 2012 00:26:44 -0700
>>>> To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
>>>> ReplyTo: [EMAIL PROTECTED]
>>>> Subject: Re: Passing date as command line arguments
>>>>
>>>> Yes it is String for me here.
>>>>
>>>> This is the table structure by the way-
>>>>
>>>> create table lip_data_quality
>>>> ( buyer_id bigint,
>>>> total_chkout bigint,
>>>> total_errpds bigint
>>>> )
>>>> partitioned by (dt string)
>>>> row format delimited fields terminated by '\t'
>>>> stored as sequencefile
>>>> location '/apps/hdmi-technology/b_apdpds/lip-data-quality'
>>>> ;
>>>>
>>>> in which I am using dt in the where clause in my query. I am running
>>>> Hive 0.6.
>>>>
>>>>
>>>>
>>>> On Sat, Aug 4, 2012 at 12:24 AM, Bejoy KS <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>> I tried the same query on my end, It is working fine for me without any
>>>>> issues.
>>>>>
>>>>> By de way the data type for 'dt' is String itself right?
>>>>> Regards
>>>>> Bejoy KS
>>>>>
>>>>> Sent from handheld, please excuse typos.
>