Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> hive mapred problem


+
Ajit Kumar Shreevastava 2012-10-08, 10:43
+
Nitin Pawar 2012-10-08, 10:49
+
Ajit Kumar Shreevastava 2012-10-08, 12:00
+
Nitin Pawar 2012-10-08, 12:21
+
Ajit Kumar Shreevastava 2012-10-09, 09:12
+
Nitin Pawar 2012-10-09, 09:15
+
Ajit Kumar Shreevastava 2012-10-09, 10:24
+
Nitin Pawar 2012-10-09, 10:32
Copy link to this message
-
RE: hive mapred problem
Hi Nitin,

Thanks for your reply.

I want configuration file of hive with respect to distributed hadoop.

Again My jps command is also failed in datanode.

When I run command on master node:-->

[hadoop@NHCLT-PC44-2 hadoop]$ jps

5694 NameNode

5992 SecondaryNameNode

5843 DataNode

6224 TaskTracker

6085 JobTracker

6701 Jps

4785 RunJar

But when I run the same command on slave node:-->

[hadoop@NHCLT-PC44-4 hadoop]$ jps

-bash: jps: command not found

Can you please help me in this regard.

-----Original Message-----
From: Nitin Pawar [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, October 09, 2012 4:03 PM
To: [EMAIL PROTECTED]
Subject: Re: hive mapred problem

I am not sure what you mean by distibuted hive.

but a multinode hadoop cluster setup you can setup using following link

http://ankitasblogger.blogspot.in/2011/01/hadoop-cluster-setup.html

if you run into hic-ups feel free to reach me out on my email (dont

want hive userlist to be bugged for hadoop installations)

On Tue, Oct 9, 2012 at 3:54 PM, Ajit Kumar Shreevastava

<[EMAIL PROTECTED]> wrote:

> Hi Nitin

>

> Sorry Nitin, Actually I mean Fully distributed mode( hadoop on multimode).

> I want configuration file for both hadoop and hive.

>

>

>

> -----Original Message-----

> From: Nitin Pawar [mailto:[EMAIL PROTECTED]]

> Sent: Tuesday, October 09, 2012 2:46 PM

> To: [EMAIL PROTECTED]

> Subject: Re: hive mapred problem

>

> I did not get the distributed mode for hadoop and hive question. can

> you explain what exactly what you want to achieve ?

>

> Thanks,

> Nitin

>

> On Tue, Oct 9, 2012 at 2:42 PM, Ajit Kumar Shreevastava

> <[EMAIL PROTECTED]> wrote:

>> Hi Nitin,

>>

>>

>>

>> Thanks for your reply...

>>

>>

>>

>> Now my query is running but output is like :-->

>>

>>

>>

>> hive> select count(1) from pokes;

>>

>> Total MapReduce jobs = 1

>>

>> Launching Job 1 out of 1

>>

>> Number of reduce tasks determined at compile time: 1

>>

>> In order to change the average load for a reducer (in bytes):

>>

>>   set hive.exec.reducers.bytes.per.reducer=<number>

>>

>> In order to limit the maximum number of reducers:

>>

>>   set hive.exec.reducers.max=<number>

>>

>> In order to set a constant number of reducers:

>>

>>   set mapred.reduce.tasks=<number>

>>

>> Starting Job = job_201210091435_0001, Tracking URL
>> http://NHCLT-PC44-2:50030/jobdetails.jsp?jobid=job_201210091435_0001

>>

>> Kill Command = /home/hadoop/hadoop-1.0.3/bin/hadoop job  -kill

>> job_201210091435_0001

>>

>> Hadoop job information for Stage-1: number of mappers: 1; number of

>> reducers: 2

>>

>> 2012-10-09 14:37:14,587 Stage-1 map = 0%,  reduce = 0%

>>

>> 2012-10-09 14:37:20,609 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:21,613 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:22,620 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:23,625 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:24,630 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:25,634 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:26,638 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:27,642 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:28,650 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:29,654 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:30,658 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:31,662 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU

>> 0.47 sec

>>

>> 2012-10-09 14:37:32,667 Stage-1 map = 100%,  reduce = 50%, Cumulative CPU

>> 1.66 sec

>>

>> 2012-10-09 14:37:33,672 Stage-1 map = 100%,  reduce = 50%, Cumulative CPU



























































































































Nitin Pawar
+
Bejoy KS 2012-10-09, 12:27