Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> need help with an error - script used to work and now it does not :-(


+
Sanjay Subramanian 2013-05-17, 02:34
+
Edward Capriolo 2013-05-17, 02:51
+
Sanjay Subramanian 2013-05-17, 03:25
+
Sanjay Subramanian 2013-05-17, 04:42
+
Sanjay Subramanian 2013-05-17, 17:56
+
Stephen Sprague 2013-05-17, 18:18
+
Sanjay Subramanian 2013-05-17, 18:25
Copy link to this message
-
Re: need help with an error - script used to work and now it does not :-(
ok. so it sounds like you are doing A/B testing then.   so if it works in
your sandbox but doesn't in prod then you can slowing transform your
sandbox - one component at time - to look like your prod system until it
breaks.  The last component you add then is an area of interest.

CTAS is short for "Create Table <blah> AS"
On Fri, May 17, 2013 at 11:25 AM, Sanjay Subramanian <
[EMAIL PROTECTED]> wrote:

>  Hi
> I actually did all of the following
> - tested all UDFs…they return values correctly
> - tested left side of LEFT OUTER JOIN
> - tested right side of LEFT OUTER JOIN
>
>  But when I add that ON statement
>  *     sh.date_seller=h.header_date*
>
>  I start getting this error…and this script has had no change for 3
> weeks….used to run fine in production and we did 15 days of aggregations
> using this script.
> Two days back we installed LZO compression on the production
> servers….Circumstancial…but the script is failing after that LZO jar
> install…Maybe totally unrelated
>
>  As we speak I am testing this script on my sandbox which I am fairly
> sure will work since I don't have LZO compression on my sandbox but I want
> to verify
>
>  What is CTAS semantics ? I don't know so please tell me… But even if I
> create intermediate tables, I will eventually need to join them…
>
>  Thanks
> sanjay
>
>   From: Stephen Sprague <[EMAIL PROTECTED]>
> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Date: Friday, May 17, 2013 11:18 AM
>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Subject: Re: need help with an error - script used to work and now it
> does not :-(
>
>   in the meantime why don't you breakup your single query into a series
> of queries (using CTAS semantics to create intermediate tables  ).
>
> The idea is narrow the problem down to a minimal size that _isolates the
> problem_  .  what you have there is overly complex to expect someone to
> troubleshoot for you.  try to minimize the failure case. take out your
> UDF's. Does it work then or fail?   strip it down to the bare necessities!
>
>
> On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <
> [EMAIL PROTECTED]> wrote:
>
>>  I am using Hive 0.9.0+155  that is bundled in Cloudera Manager version
>> 4.1.2
>> Still getting the errors  listed below :-(
>> Any clues will be be cool !!!
>> Thanks
>>
>>  sanjay
>>
>>
>>   From: Sanjay Subramanian <[EMAIL PROTECTED]>
>> Date: Thursday, May 16, 2013 9:42 PM
>>
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Subject: Re: need help with an error - script used to work and now it
>> does not :-(
>>
>>   :-( Still facing problems in large datasets
>> Were u able to solve this Edward ?
>> Thanks
>> sanjay
>>
>>   From: Sanjay Subramanian <[EMAIL PROTECTED]>
>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Date: Thursday, May 16, 2013 8:25 PM
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Subject: Re: need help with an error - script used to work and now it
>> does not :-(
>>
>>   Thanks Edward…I just checked all instances of guava jars…except those
>> in red all seem same version
>>
>>  /usr/lib/hadoop/client/guava-11.0.2.jar
>> /usr/lib/hadoop/client-0.20/guava-11.0.2.jar
>> /usr/lib/hadoop/lib/guava-11.0.2.jar
>> /usr/lib/hadoop-httpfs/webapps/webhdfs/WEB-INF/lib/guava-11.0.2.jar
>> /usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar
>> /usr/lib/oozie/libtools/guava-11.0.2.jar
>> /usr/lib/hive/lib/guava-11.0.2.jar
>> /usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar
>> /usr/lib/hbase/lib/guava-11.0.2.jar
>> /usr/lib/flume-ng/lib/guava-11.0.2.jar
>> /usr/share/cmf/lib/cdh3/guava-r09-jarjar.jar
>> /usr/share/cmf/lib/guava-12.0.1.jar
>>
>>  But I made a small change in my query (I just removed the text marked
>> in blue) that seemed to solve it at least for the test data set that I
>> had….Now I need to run it in production for a days worth of data
>>
>>  Will keep u guys posted
>>
>>
>> -------------------------
+
Sanjay Subramanian 2013-05-17, 18:45
+
Stephen Sprague 2013-05-17, 19:01
+
Ramki Palle 2013-05-18, 16:54