Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> need help with an error - script used to work and now it does not :-(


+
Sanjay Subramanian 2013-05-17, 02:34
+
Edward Capriolo 2013-05-17, 02:51
+
Sanjay Subramanian 2013-05-17, 03:25
+
Sanjay Subramanian 2013-05-17, 04:42
+
Sanjay Subramanian 2013-05-17, 17:56
+
Stephen Sprague 2013-05-17, 18:18
+
Sanjay Subramanian 2013-05-17, 18:25
Copy link to this message
-
Re: need help with an error - script used to work and now it does not :-(
ok. so it sounds like you are doing A/B testing then.   so if it works in
your sandbox but doesn't in prod then you can slowing transform your
sandbox - one component at time - to look like your prod system until it
breaks.  The last component you add then is an area of interest.

CTAS is short for "Create Table <blah> AS"
On Fri, May 17, 2013 at 11:25 AM, Sanjay Subramanian <
[EMAIL PROTECTED]> wrote:

>  Hi
> I actually did all of the following
> - tested all UDFs…they return values correctly
> - tested left side of LEFT OUTER JOIN
> - tested right side of LEFT OUTER JOIN
>
>  But when I add that ON statement
>  *     sh.date_seller=h.header_date*
>
>  I start getting this error…and this script has had no change for 3
> weeks….used to run fine in production and we did 15 days of aggregations
> using this script.
> Two days back we installed LZO compression on the production
> servers….Circumstancial…but the script is failing after that LZO jar
> install…Maybe totally unrelated
>
>  As we speak I am testing this script on my sandbox which I am fairly
> sure will work since I don't have LZO compression on my sandbox but I want
> to verify
>
>  What is CTAS semantics ? I don't know so please tell me… But even if I
> create intermediate tables, I will eventually need to join them…
>
>  Thanks
> sanjay
>
>   From: Stephen Sprague <[EMAIL PROTECTED]>
> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Date: Friday, May 17, 2013 11:18 AM
>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Subject: Re: need help with an error - script used to work and now it
> does not :-(
>
>   in the meantime why don't you breakup your single query into a series
> of queries (using CTAS semantics to create intermediate tables  ).
>
> The idea is narrow the problem down to a minimal size that _isolates the
> problem_  .  what you have there is overly complex to expect someone to
> troubleshoot for you.  try to minimize the failure case. take out your
> UDF's. Does it work then or fail?   strip it down to the bare necessities!
>
>
> On Fri, May 17, 2013 at 10:56 AM, Sanjay Subramanian <
> [EMAIL PROTECTED]> wrote:
>
>>  I am using Hive 0.9.0+155  that is bundled in Cloudera Manager version
>> 4.1.2
>> Still getting the errors  listed below :-(
>> Any clues will be be cool !!!
>> Thanks
>>
>>  sanjay
>>
>>
>>   From: Sanjay Subramanian <[EMAIL PROTECTED]>
>> Date: Thursday, May 16, 2013 9:42 PM
>>
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Subject: Re: need help with an error - script used to work and now it
>> does not :-(
>>
>>   :-( Still facing problems in large datasets
>> Were u able to solve this Edward ?
>> Thanks
>> sanjay
>>
>>   From: Sanjay Subramanian <[EMAIL PROTECTED]>
>> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Date: Thursday, May 16, 2013 8:25 PM
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Subject: Re: need help with an error - script used to work and now it
>> does not :-(
>>
>>   Thanks Edward…I just checked all instances of guava jars…except those
>> in red all seem same version
>>
>>  /usr/lib/hadoop/client/guava-11.0.2.jar
>> /usr/lib/hadoop/client-0.20/guava-11.0.2.jar
>> /usr/lib/hadoop/lib/guava-11.0.2.jar
>> /usr/lib/hadoop-httpfs/webapps/webhdfs/WEB-INF/lib/guava-11.0.2.jar
>> /usr/lib/hadoop-hdfs/lib/guava-11.0.2.jar
>> /usr/lib/oozie/libtools/guava-11.0.2.jar
>> /usr/lib/hive/lib/guava-11.0.2.jar
>> /usr/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar
>> /usr/lib/hbase/lib/guava-11.0.2.jar
>> /usr/lib/flume-ng/lib/guava-11.0.2.jar
>> /usr/share/cmf/lib/cdh3/guava-r09-jarjar.jar
>> /usr/share/cmf/lib/guava-12.0.1.jar
>>
>>  But I made a small change in my query (I just removed the text marked
>> in blue) that seemed to solve it at least for the test data set that I
>> had….Now I need to run it in production for a days worth of data
>>
>>  Will keep u guys posted
>>
>>
>> -------------------------
+
Sanjay Subramanian 2013-05-17, 18:45
+
Stephen Sprague 2013-05-17, 19:01
+
Ramki Palle 2013-05-18, 16:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB