Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive job fails on hive client even though all map-red stages finish but succeeds on hive server


Copy link to this message
-
Re: Hive job fails on hive client even though all map-red stages finish but succeeds on hive server
Hi Anurag,

How much space is for /user and /tmp directory on client.

Did you check that part? , anything which might stop move task from
finishing.

-----------
Sent from Mobile , short and crisp.
On 11-Aug-2012 1:37 PM, "Anurag Tangri" <[EMAIL PROTECTED]> wrote:

> Hi,
> We are facing this issue where we run a hive job over huge data about ~6
> TB input.
>
> We run this from hive client and hive metastore server is on another
> machine.
>
>
> If we have smaller input, this job succeeds but for above input size, it
> fails with error :
>
> 2012-08-11 01:34:01,722 Stage-1 map = 100%,  reduce = 100%
>
> 2012-08-11 01:35:02,195 Stage-1 map = 100%,  reduce = 100%
>
> 2012-08-11 01:36:02,682 Stage-1 map = 100%,  reduce = 100%
>
> 2012-08-11 01:37:03,215 Stage-1 map = 100%,  reduce = 100%
>
> 2012-08-11 01:38:03,719 Stage-1 map = 100%,  reduce = 100%
>
> 2012-08-11 01:39:04,311 Stage-1 map = 100%,  reduce = 100%
>
> Ended Job = job_201207072204_34432
>
> Loading data to table default.atangri_test_1
>
> Failed with exception Unable to fetch table atangri_test_1
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MoveTask
>
>
> If we have smaller input (~2 TB), this job succeeds but for above input
> size, it fails with error : We have set
> hive.metastore.client.socket.timeout to big value like 86400  but still it
> fails after about 8-9 hours.
>
> Does anyone face the same issue or any pointers ?
>
> The job succeeds if it is directly run on hive server.
>
> Thanks,
> Anurag Tangri
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB