Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hive mapred problem


Copy link to this message
-
RE: hive mapred problem
Hi Nitin,

Thanks for your reply.

I want configuration file of hive with respect to distributed hadoop.

Again My jps command is also failed in datanode.

When I run command on master node:-->

[hadoop@NHCLT-PC44-2 hadoop]$ jps

5694 NameNode

5992 SecondaryNameNode

5843 DataNode

6224 TaskTracker

6085 JobTracker

6701 Jps

4785 RunJar

But when I run the same command on slave node:-->

[hadoop@NHCLT-PC44-4 hadoop]$ jps

-bash: jps: command not found

Can you please help me in this regard.

-----Original Message-----
From: Nitin Pawar [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, October 09, 2012 4:03 PM
To: [EMAIL PROTECTED]
Subject: Re: hive mapred problem

I am not sure what you mean by distibuted hive.

but a multinode hadoop cluster setup you can setup using following link

http://ankitasblogger.blogspot.in/2011/01/hadoop-cluster-setup.html

if you run into hic-ups feel free to reach me out on my email (dont

want hive userlist to be bugged for hadoop installations)

On Tue, Oct 9, 2012 at 3:54 PM, Ajit Kumar Shreevastava

<[EMAIL PROTECTED]> wrote:

> Hi Nitin

>

> Sorry Nitin, Actually I mean Fully distributed mode( hadoop on multimode).

> I want configuration file for both hadoop and hive.

>

>

>

> -----Original Message-----

> From: Nitin Pawar [mailto:[EMAIL PROTECTED]]

> Sent: Tuesday, October 09, 2012 2:46 PM

> To: [EMAIL PROTECTED]

> Subject: Re: hive mapred problem

>

> I did not get the distributed mode for hadoop and hive question. can

> you explain what exactly what you want to achieve ?

>

> Thanks,

> Nitin

>

> On Tue, Oct 9, 2012 at 2:42 PM, Ajit Kumar Shreevastava

> <[EMAIL PROTECTED]> wrote:

>> Hi Nitin,

>>

>>

>>

>> Thanks for your reply...

>>

>>

>>

>> Now my query is running but output is like :-->

>>

>>

>>

>> hive> select count(1) from pokes;

>>

>> Total MapReduce jobs = 1

>>

>> Launching Job 1 out of 1

>>

>> Number of reduce tasks determined at compile time: 1

>>

>> In order to change the average load for a reducer (in bytes):

>>

>>   set hive.exec.reducers.bytes.per.reducer=<number>

>>

>> In order to limit the maximum number of reducers:

>>

>>   set hive.exec.reducers.max=<number>

>>

>> In order to set a constant number of reducers:

>>

>>   set mapred.reduce.tasks=<number>

>>

>> Starting Job = job_201210091435_0001, Tracking URL
>> http://NHCLT-PC44-2:50030/jobdetails.jsp?jobid=job_201210091435_0001

>>

>> Kill Command = /home/hadoop/hadoop-1.0.3/bin/hadoop job  -kill

>> job_201210091435_0001

>>

>> Hadoop job information for Stage-1: number of mappers: 1; number of

>> reducers: 2

>>

>> 2012-10-09 14:37:14,587 Stage-1 map = 0%,  reduce = 0%

>>

>> 2012-10-09 14:37:20,609 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:21,613 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:22,620 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:23,625 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:24,630 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:25,634 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:26,638 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:27,642 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:28,650 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:29,654 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:30,658 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:31,662 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:32,667 Stage-1 map = 100%,  reduce = 50%, Cumulative CPU

>> 1.66 sec

>>

>> 2012-10-09 14:37:33,672 Stage-1 map = 100%,  reduce = 50%, Cumulative CPU



























































































































Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB