Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - need help with an error - script used to work and now it does not :-(


Copy link to this message
-
Re: need help with an error - script used to work and now it does not :-(
Ramki Palle 2013-05-18, 16:54
Sanjay,

The date type column handling is little bit delicate in Hive. You may try
to cast (
https://cwiki.apache.org/Hive/languagemanual-udf.html#LanguageManualUDF-DateFunctions)
 explicitly on both sides of
*sh.date_seller=h.header_date
*and check if it works. This is a guess but it does not hurt to try as it
is simple to test.

Regards,
Ramki.
On Fri, May 17, 2013 at 12:01 PM, Stephen Sprague <[EMAIL PROTECTED]>wrote:

> First off. Note to self (and anybody else out there) Hive is a far cry
> from betting the farm on.   You better have a Plan B when it crumples.
> Just sayin.
>
> Yeah, Sanjay, we'll have to wait for someone else to chime in on the java
> debugging as i'm not the domain expert on that aspect.
>
>
> On Fri, May 17, 2013 at 11:45 AM, Sanjay Subramanian <
> [EMAIL PROTECTED]> wrote:
>
>>  Thanks Stephen….I am really worried about this because now I have a
>> backlog of days piling up for this script to process…
>> A quick question….is there a way to do some remote debugging using some
>> Eclipse-ish tool…u know how we debug java applications remotely and the
>> debug point comes back and we can analyze line by line….so I was thinking
>> if we can do that same thing for Hive source code and I can walk thru that…
>>
>>  Thanks
>> sanjay
>>
>>   From: Stephen Sprague <[EMAIL PROTECTED]>
>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Date: Friday, May 17, 2013 11:36 AM
>>
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Subject: Re: need help with an error - script used to work and now it
>> does not :-(
>>
>>   ok. so it sounds like you are doing A/B testing then.   so if it works
>> in your sandbox but doesn't in prod then you can slowing transform your
>> sandbox - one component at time - to look like your prod system until it
>> breaks.  The last component you add then is an area of interest.
>>
>>  CTAS is short for "Create Table <blah> AS"
>>
>>
>> On Fri, May 17, 2013 at 11:25 AM, Sanjay Subramanian <
>> [EMAIL PROTECTED]> wrote:
>>
>>>  Hi
>>> I actually did all of the following
>>> - tested all UDFs…they return values correctly
>>> - tested left side of LEFT OUTER JOIN
>>> - tested right side of LEFT OUTER JOIN
>>>
>>>  But when I add that ON statement
>>>  *     sh.date_seller=h.header_date*
>>>
>>>  I start getting this error…and this script has had no change for 3
>>> weeks….used to run fine in production and we did 15 days of aggregations
>>> using this script.
>>> Two days back we installed LZO compression on the production
>>> servers….Circumstancial…but the script is failing after that LZO jar
>>> install…Maybe totally unrelated
>>>
>>>  As we speak I am testing this script on my sandbox which I am fairly
>>> sure will work since I don't have LZO compression on my sandbox but I want
>>> to verify
>>>
>>>  What is CTAS semantics ? I don't know so please tell me… But even if I
>>> create intermediate tables, I will eventually need to join them…
>>>
>>>  Thanks
>>> sanjay
>>>
>>>   From: Stephen Sprague <[EMAIL PROTECTED]>
>>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>> Date: Friday, May 17, 2013 11:18 AM
>>>
>>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>>> Subject: Re: need help with an error - script used to work and now it
>>> does not :-(
>>>
>>>   in the meantime why don't you breakup your single query into a series
>>> of queries (using CTAS semantics to create intermediate tables  ).
>>>
>>> The idea is narrow the problem down to a minimal size that _isolates the
>>> problem_  .  what you have there is overly complex to expect someone to
>>> troubleshoot for you.  try to minimize the failure case. take out your
>>> UDF's. Does it work then or fail?   strip it down to the bare necessities!
>>>
>>>
>>> On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>>  I am using Hive 0.9.0+155  that is bundled in Cloudera Manager
>>>> version 4.1.2
>>>> Still getting the errors  listed below :-(