Hi All,

even If if perform count(*) operation on table it shows error,

select count(*) from dummysite;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201207231123_0011, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201207231123_0011 with errors
Error during job, obtaining debugging information...
Examining task ID: task_201207231123_0011_m_000002 (and more) from job job_201207231123_0011
Exception in thread "Thread-93" java.lang.RuntimeException: Error while reading from task log url
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
    at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Server returned HTTP response code: 407 for URL: http://10.203.33.81:50060/tasklog?taskid=attempt_201207231123_0011_r_000000_0&start=-8193
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at java.net.URL.openStream(URL.java:1010)
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
    ... 3 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1  Reduce: 1   HDFS Read: 24 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
Please suggest why this error is comming :-(

Regards
Yogesh Kumar

Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

www.wipro.com

Hello Yogesh,

        Are any proxy settings involved??Error code 407 indicates that
the client must first authenticate itself with the proxy in order to
proceed further. Just make sure everything is in place.

Regards,
    Mohammad Tariq
On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> even If if perform count(*) operation on table it shows error,
>
> select count(*) from dummysite;
>
>
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
>   set mapred.reduce.tasks=<number>
> Starting Job = job_201207231123_0011, Tracking URL > http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
> Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 1
> 2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
> 2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
> 2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
> 2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
> Ended Job = job_201207231123_0011 with errors
> Error during job, obtaining debugging information...
> Examining task ID: task_201207231123_0011_m_000002 (and more) from job
> job_201207231123_0011
> Exception in thread "Thread-93" java.lang.RuntimeException: Error while
> reading from task log url
>     at
> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
>     at
> org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
>     at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
>     at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Server returned HTTP response code: 407 for
> URL:
> http://10.203.33.81:50060/tasklog?taskid=attempt_201207231123_0011_r_000000_0&start=-8193
>     at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
>     at java.net.URL.openStream(URL.java:1010)
>     at
> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
>     ... 3 more
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
> MapReduce Jobs Launched:
> Job 0: Map: 1  Reduce: 1   HDFS Read: 24 HDFS Write: 0 FAIL
> Total MapReduce CPU Time Spent: 0 msec
>
>
> Please suggest why this error is comming :-(
>
> Regards
> Yogesh Kumar
>
> Please do not print this email unless it is absolutely necessary.
>
> The information contained in this electronic message and any attachments to
> this message are intended for the exclusive use of the addressee(s) and may
> contain proprietary, confidential or privileged information. If you are not
> the intended recipient, you should not disseminate, distribute or copy this
> e-mail. Please notify the sender immediately and destroy all copies of this
> message and any attachments.
>
> WARNING: Computer viruses can be transmitted via email. The recipient should
> check this email and any attachments for the presence of viruses. The
> company accepts no liability for any damage caused by any virus transmitted
> by this email.
>
> www.wipro.com

Thanks Mohammad :-),

I didn't get regarding which proxy settings Please explain it,

If i do operations like.
select * from dummysite,
or
select * from dummysite where id=10;

such command shows proper result and doesn't through any error.

Please suggest.

Regards
Yogesh Kumar
________________________________________
From: Mohammad Tariq [[EMAIL PROTECTED]]
Sent: Tuesday, July 24, 2012 2:00 PM
To: [EMAIL PROTECTED]
Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:

Hello Yogesh,

        Are any proxy settings involved??Error code 407 indicates that
the client must first authenticate itself with the proxy in order to
proceed further. Just make sure everything is in place.

Regards,
    Mohammad Tariq
On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> even If if perform count(*) operation on table it shows error,
>
> select count(*) from dummysite;
>
>
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
>   set mapred.reduce.tasks=<number>
> Starting Job = job_201207231123_0011, Tracking URL > http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
> Hadoop job information for Stage-1: number of mappers: 1; number of
> reducers: 1
> 2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
> 2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
> 2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
> 2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
> Ended Job = job_201207231123_0011 with errors
> Error during job, obtaining debugging information...
> Examining task ID: task_201207231123_0011_m_000002 (and more) from job
> job_201207231123_0011
> Exception in thread "Thread-93" java.lang.RuntimeException: Error while
> reading from task log url
>     at
> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
>     at
> org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
>     at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
>     at java.lang.Thread.run(Thread.java:680)
> Caused by: java.io.IOException: Server returned HTTP response code: 407 for
> URL:
> http://10.203.33.81:50060/tasklog?taskid=attempt_201207231123_0011_r_000000_0&start=-8193
>     at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
>     at java.net.URL.openStream(URL.java:1010)
>     at
> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
>     ... 3 more
> FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
> MapReduce Jobs Launched:
> Job 0: Map: 1  Reduce: 1   HDFS Read: 24 HDFS Write: 0 FAIL
> Total MapReduce CPU Time Spent: 0 msec
>
>
> Please suggest why this error is comming :-(
>
> Regards
> Yogesh Kumar
>
> Please do not print this email unless it is absolutely necessary.
>
> The information contained in this electronic message and any attachments to
> this message are intended for the exclusive use of the addressee(s) and may
> contain proprietary, confidential or privileged information. If you are not
> the intended recipient, you should not disseminate, distribute or copy this
> e-mail. Please notify the sender immediately and destroy all copies of this
> message and any attachments.
>
> WARNING: Computer viruses can be transmitted via email. The recipient should
> check this email and any attachments for the presence of viruses. The
> company accepts no liability for any damage caused by any virus transmitted

Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

www.wipro.com

yogesh,
is your network setup with proxy authentication? normally the error
which you are getting is, when a node tries to connect to another node
but then after some time, the connection needs to be reauthorized.

when you do a select * from table, it does not launch a MR, it just
does a dfs -cat of the hdfs file

you will need to check the cluster health of your hadoop cluster. Also
make sure that on all the nodes the default linux security is off
(iptables )

On Tue, Jul 24, 2012 at 2:09 PM,  <[EMAIL PROTECTED]> wrote:
> Thanks Mohammad :-),
>
> I didn't get regarding which proxy settings Please explain it,
>
> If i do operations like.
> select * from dummysite,
> or
> select * from dummysite where id=10;
>
> such command shows proper result and doesn't through any error.
>
> Please suggest.
>
> Regards
> Yogesh Kumar
> ________________________________________
> From: Mohammad Tariq [[EMAIL PROTECTED]]
> Sent: Tuesday, July 24, 2012 2:00 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>
> Hello Yogesh,
>
>         Are any proxy settings involved??Error code 407 indicates that
> the client must first authenticate itself with the proxy in order to
> proceed further. Just make sure everything is in place.
>
> Regards,
>     Mohammad Tariq
>
>
> On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
>> Hi All,
>>
>> even If if perform count(*) operation on table it shows error,
>>
>> select count(*) from dummysite;
>>
>>
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks determined at compile time: 1
>> In order to change the average load for a reducer (in bytes):
>>   set hive.exec.reducers.bytes.per.reducer=<number>
>> In order to limit the maximum number of reducers:
>>   set hive.exec.reducers.max=<number>
>> In order to set a constant number of reducers:
>>   set mapred.reduce.tasks=<number>
>> Starting Job = job_201207231123_0011, Tracking URL >> http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
>> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
>> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
>> Hadoop job information for Stage-1: number of mappers: 1; number of
>> reducers: 1
>> 2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
>> 2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
>> 2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
>> 2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
>> Ended Job = job_201207231123_0011 with errors
>> Error during job, obtaining debugging information...
>> Examining task ID: task_201207231123_0011_m_000002 (and more) from job
>> job_201207231123_0011
>> Exception in thread "Thread-93" java.lang.RuntimeException: Error while
>> reading from task log url
>>     at
>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
>>     at
>> org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
>>     at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
>>     at java.lang.Thread.run(Thread.java:680)
>> Caused by: java.io.IOException: Server returned HTTP response code: 407 for
>> URL:
>> http://10.203.33.81:50060/tasklog?taskid=attempt_201207231123_0011_r_000000_0&start=-8193
>>     at
>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
>>     at java.net.URL.openStream(URL.java:1010)
>>     at
>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
>>     ... 3 more
>> FAILED: Execution Error, return code 2 from
>> org.apache.hadoop.hive.ql.exec.MapRedTask
>> MapReduce Jobs Launched:
>> Job 0: Map: 1  Reduce: 1   HDFS Read: 24 HDFS Write: 0 FAIL
>> Total MapReduce CPU Time Spent: 0 msec
>>
>>
>> Please suggest why this error is comming :-(
>>
>> Regards
>> Yogesh Kumar
>>
>> Please do not print this email unless it is absolutely necessary.

Nitin Pawar

Thanks Nitin :-),

I ensure the cluster health,

hadoop dfsadmin -report,

mediaadmins-iMac-2:~ mediaadmin$ hadoop dfsadmin -report
Configured Capacity: 499763888128 (465.44 GB)
Present Capacity: 51694825472 (48.14 GB)
DFS Remaining: 49239232512 (45.86 GB)
DFS Used: 2455592960 (2.29 GB)
DFS Used%: 4.75%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 1 (1 total, 0 dead)

Name: 127.0.0.1:50010
Decommission Status : Normal
Configured Capacity: 499763888128 (465.44 GB)
DFS Used: 2455592960 (2.29 GB)
Non DFS Used: 448069062656 (417.3 GB)
DFS Remaining: 49239232512(45.86 GB)
DFS Used%: 0.49%
DFS Remaining%: 9.85%
Last contact: Tue Jul 24 14:27:05 GMT+05:30 2012
Also If I go for map-reduce jobs like.

Select Name, Grade from Result where RollNo = 325;
it runs map-reduce and gives proper result.

Here I am using Mac O.S, Please suggest me is that error is still because of proxy authentication, if it is so then how to come out of it.

Thanks & regards
Yogesh Kumar

________________________________________
From: Nitin Pawar [[EMAIL PROTECTED]]
Sent: Tuesday, July 24, 2012 2:13 PM
To: [EMAIL PROTECTED]
Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:

yogesh,
is your network setup with proxy authentication? normally the error
which you are getting is, when a node tries to connect to another node
but then after some time, the connection needs to be reauthorized.

when you do a select * from table, it does not launch a MR, it just
does a dfs -cat of the hdfs file

you will need to check the cluster health of your hadoop cluster. Also
make sure that on all the nodes the default linux security is off
(iptables )

On Tue, Jul 24, 2012 at 2:09 PM,  <[EMAIL PROTECTED]> wrote:
> Thanks Mohammad :-),
>
> I didn't get regarding which proxy settings Please explain it,
>
> If i do operations like.
> select * from dummysite,
> or
> select * from dummysite where id=10;
>
> such command shows proper result and doesn't through any error.
>
> Please suggest.
>
> Regards
> Yogesh Kumar
> ________________________________________
> From: Mohammad Tariq [[EMAIL PROTECTED]]
> Sent: Tuesday, July 24, 2012 2:00 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>
> Hello Yogesh,
>
>         Are any proxy settings involved??Error code 407 indicates that
> the client must first authenticate itself with the proxy in order to
> proceed further. Just make sure everything is in place.
>
> Regards,
>     Mohammad Tariq
>
>
> On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
>> Hi All,
>>
>> even If if perform count(*) operation on table it shows error,
>>
>> select count(*) from dummysite;
>>
>>
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks determined at compile time: 1
>> In order to change the average load for a reducer (in bytes):
>>   set hive.exec.reducers.bytes.per.reducer=<number>
>> In order to limit the maximum number of reducers:
>>   set hive.exec.reducers.max=<number>
>> In order to set a constant number of reducers:
>>   set mapred.reduce.tasks=<number>
>> Starting Job = job_201207231123_0011, Tracking URL >> http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
>> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
>> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
>> Hadoop job information for Stage-1: number of mappers: 1; number of
>> reducers: 1
>> 2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
>> 2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
>> 2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
>> 2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
>> Ended Job = job_201207231123_0011 with errors
>> Error during job, obtaining debugging information...

Nitin Pawar

Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

www.wipro.com

You are correct.

I have run two commands,

1) select site_id from site where id = 2;

it shows result and details are.

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201207241536_0003, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201207241536_0003
Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201207241536_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2012-07-24 15:42:09,251 Stage-1 map = 0%,  reduce = 0%
2012-07-24 15:42:12,332 Stage-1 map = 100%,  reduce = 0%
2012-07-24 15:42:15,391 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201207241536_0003
MapReduce Jobs Launched:
Job 0: Map: 1   HDFS Read: 24 HDFS Write: 3 SUCESS
Total MapReduce CPU Time Spent: 0 msec
OK
12
Time taken: 17.696 seconds

When I have no reduce operator it works fine, but in second case.
2) select count(*) from site;

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201207241536_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201207241536_0002
Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201207241536_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2012-07-24 15:37:53,818 Stage-1 map = 0%,  reduce = 0%
2012-07-24 15:38:00,176 Stage-1 map = 100%,  reduce = 0%
2012-07-24 15:39:01,068 Stage-1 map = 100%,  reduce = 0%
2012-07-24 15:39:03,307 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201207241536_0002 with errors
Error during job, obtaining debugging information...
Examining task ID: task_201207241536_0002_m_000002 (and more) from job job_201207241536_0002
Exception in thread "Thread-40" java.lang.RuntimeException: Error while reading from task log url
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
    at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Server returned HTTP response code: 407 for URL: http://10.203.33.81:50060/tasklog?taskid=attempt_201207241536_0002_r_000000_0&start=-8193
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at java.net.URL.openStream(URL.java:1010)
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
    ... 3 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1  Reduce: 1   HDFS Read: 24 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

When I have Reduce operator it gets fail.
 
Please suggest how do I make all nodes free from security issues and n/w proxy.
This is offc's machine (Mac OS X 10.6.8).

Thanks & Regards
Yogesh Kumar
8147912904
________________________________________
From: Nitin Pawar [[EMAIL PROTECTED]]
Sent: Tuesday, July 24, 2012 2:13 PM
To: [EMAIL PROTECTED]
Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:

yogesh,
is your network setup with proxy authentication? normally the error
which you are getting is, when a node tries to connect to another node
but then after some time, the connection needs to be reauthorized.

when you do a select * from table, it does not launch a MR, it just
does a dfs -cat of the hdfs file

you will need to check the cluster health of your hadoop cluster. Also
make sure that on all the nodes the default linux security is off
(iptables )

On Tue, Jul 24, 2012 at 2:09 PM,  <[EMAIL PROTECTED]> wrote:

Nitin Pawar

Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

www.wipro.com

Thanks Nitin :-), Thank you very much.

Yes my n/w setup with proxy authentication, if I do remove proxy foe HTTP, HTTPS then it works but the Internet goes down.

I set the proxy bypass for machine and it worked. :-)

Thanks a Lot
Yogesh Kumar
________________________________________
From: Nitin Pawar [[EMAIL PROTECTED]]
Sent: Tuesday, July 24, 2012 2:13 PM
To: [EMAIL PROTECTED]
Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:

yogesh,
is your network setup with proxy authentication? normally the error
which you are getting is, when a node tries to connect to another node
but then after some time, the connection needs to be reauthorized.

when you do a select * from table, it does not launch a MR, it just
does a dfs -cat of the hdfs file

you will need to check the cluster health of your hadoop cluster. Also
make sure that on all the nodes the default linux security is off
(iptables )

On Tue, Jul 24, 2012 at 2:09 PM,  <[EMAIL PROTECTED]> wrote:
> Thanks Mohammad :-),
>
> I didn't get regarding which proxy settings Please explain it,
>
> If i do operations like.
> select * from dummysite,
> or
> select * from dummysite where id=10;
>
> such command shows proper result and doesn't through any error.
>
> Please suggest.
>
> Regards
> Yogesh Kumar
> ________________________________________
> From: Mohammad Tariq [[EMAIL PROTECTED]]
> Sent: Tuesday, July 24, 2012 2:00 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>
> Hello Yogesh,
>
>         Are any proxy settings involved??Error code 407 indicates that
> the client must first authenticate itself with the proxy in order to
> proceed further. Just make sure everything is in place.
>
> Regards,
>     Mohammad Tariq
>
>
> On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
>> Hi All,
>>
>> even If if perform count(*) operation on table it shows error,
>>
>> select count(*) from dummysite;
>>
>>
>> Total MapReduce jobs = 1
>> Launching Job 1 out of 1
>> Number of reduce tasks determined at compile time: 1
>> In order to change the average load for a reducer (in bytes):
>>   set hive.exec.reducers.bytes.per.reducer=<number>
>> In order to limit the maximum number of reducers:
>>   set hive.exec.reducers.max=<number>
>> In order to set a constant number of reducers:
>>   set mapred.reduce.tasks=<number>
>> Starting Job = job_201207231123_0011, Tracking URL >> http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
>> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
>> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
>> Hadoop job information for Stage-1: number of mappers: 1; number of
>> reducers: 1
>> 2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
>> 2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
>> 2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
>> 2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
>> Ended Job = job_201207231123_0011 with errors
>> Error during job, obtaining debugging information...
>> Examining task ID: task_201207231123_0011_m_000002 (and more) from job
>> job_201207231123_0011
>> Exception in thread "Thread-93" java.lang.RuntimeException: Error while
>> reading from task log url
>>     at
>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
>>     at
>> org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
>>     at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
>>     at java.lang.Thread.run(Thread.java:680)
>> Caused by: java.io.IOException: Server returned HTTP response code: 407 for
>> URL:
>> http://10.203.33.81:50060/tasklog?taskid=attempt_201207231123_0011_r_000000_0&start=-8193
>>     at
>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
>>     at java.net.URL.openStream(URL.java:1010)

Nitin Pawar

Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

www.wipro.com

if you have a single node cluster, then just configure everything to
localhost. But if you have two nodes then thats a problem cause in any
cluster nodes need to talk with each other without authentication on
network level

On Tue, Jul 24, 2012 at 4:19 PM,  <[EMAIL PROTECTED]> wrote:
> Thanks Nitin :-), Thank you very much.
>
> Yes my n/w setup with proxy authentication, if I do remove proxy foe HTTP, HTTPS then it works but the Internet goes down.
>
> I set the proxy bypass for machine and it worked. :-)
>
> Thanks a Lot
> Yogesh Kumar
> ________________________________________
> From: Nitin Pawar [[EMAIL PROTECTED]]
> Sent: Tuesday, July 24, 2012 2:13 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>
> yogesh,
> is your network setup with proxy authentication? normally the error
> which you are getting is, when a node tries to connect to another node
> but then after some time, the connection needs to be reauthorized.
>
> when you do a select * from table, it does not launch a MR, it just
> does a dfs -cat of the hdfs file
>
> you will need to check the cluster health of your hadoop cluster. Also
> make sure that on all the nodes the default linux security is off
> (iptables )
>
> On Tue, Jul 24, 2012 at 2:09 PM,  <[EMAIL PROTECTED]> wrote:
>> Thanks Mohammad :-),
>>
>> I didn't get regarding which proxy settings Please explain it,
>>
>> If i do operations like.
>> select * from dummysite,
>> or
>> select * from dummysite where id=10;
>>
>> such command shows proper result and doesn't through any error.
>>
>> Please suggest.
>>
>> Regards
>> Yogesh Kumar
>> ________________________________________
>> From: Mohammad Tariq [[EMAIL PROTECTED]]
>> Sent: Tuesday, July 24, 2012 2:00 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>>
>> Hello Yogesh,
>>
>>         Are any proxy settings involved??Error code 407 indicates that
>> the client must first authenticate itself with the proxy in order to
>> proceed further. Just make sure everything is in place.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>> On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
>>> Hi All,
>>>
>>> even If if perform count(*) operation on table it shows error,
>>>
>>> select count(*) from dummysite;
>>>
>>>
>>> Total MapReduce jobs = 1
>>> Launching Job 1 out of 1
>>> Number of reduce tasks determined at compile time: 1
>>> In order to change the average load for a reducer (in bytes):
>>>   set hive.exec.reducers.bytes.per.reducer=<number>
>>> In order to limit the maximum number of reducers:
>>>   set hive.exec.reducers.max=<number>
>>> In order to set a constant number of reducers:
>>>   set mapred.reduce.tasks=<number>
>>> Starting Job = job_201207231123_0011, Tracking URL >>> http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
>>> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
>>> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
>>> Hadoop job information for Stage-1: number of mappers: 1; number of
>>> reducers: 1
>>> 2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
>>> 2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
>>> 2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
>>> 2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
>>> Ended Job = job_201207231123_0011 with errors
>>> Error during job, obtaining debugging information...
>>> Examining task ID: task_201207231123_0011_m_000002 (and more) from job
>>> job_201207231123_0011
>>> Exception in thread "Thread-93" java.lang.RuntimeException: Error while
>>> reading from task log url
>>>     at
>>> org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
>>>     at
>>> org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
>>>     at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)

Nitin Pawar

Hi yogesh,

     Sorry for being unresponsive. Have a tight schedule today. By
looking at your logs, It seems you are running your cluster in pseudo
distributed mode. If that is the case just make sure you have
commented the following line in your /etc/hosts file
127.0.1.1

Regards,
    Mohammad Tariq
On Tue, Jul 24, 2012 at 5:21 PM, Nitin Pawar <[EMAIL PROTECTED]> wrote:
> if you have a single node cluster, then just configure everything to
> localhost. But if you have two nodes then thats a problem cause in any
> cluster nodes need to talk with each other without authentication on
> network level
>
> On Tue, Jul 24, 2012 at 4:19 PM,  <[EMAIL PROTECTED]> wrote:
>> Thanks Nitin :-), Thank you very much.
>>
>> Yes my n/w setup with proxy authentication, if I do remove proxy foe HTTP, HTTPS then it works but the Internet goes down.
>>
>> I set the proxy bypass for machine and it worked. :-)
>>
>> Thanks a Lot
>> Yogesh Kumar
>> ________________________________________
>> From: Nitin Pawar [[EMAIL PROTECTED]]
>> Sent: Tuesday, July 24, 2012 2:13 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>>
>> yogesh,
>> is your network setup with proxy authentication? normally the error
>> which you are getting is, when a node tries to connect to another node
>> but then after some time, the connection needs to be reauthorized.
>>
>> when you do a select * from table, it does not launch a MR, it just
>> does a dfs -cat of the hdfs file
>>
>> you will need to check the cluster health of your hadoop cluster. Also
>> make sure that on all the nodes the default linux security is off
>> (iptables )
>>
>> On Tue, Jul 24, 2012 at 2:09 PM,  <[EMAIL PROTECTED]> wrote:
>>> Thanks Mohammad :-),
>>>
>>> I didn't get regarding which proxy settings Please explain it,
>>>
>>> If i do operations like.
>>> select * from dummysite,
>>> or
>>> select * from dummysite where id=10;
>>>
>>> such command shows proper result and doesn't through any error.
>>>
>>> Please suggest.
>>>
>>> Regards
>>> Yogesh Kumar
>>> ________________________________________
>>> From: Mohammad Tariq [[EMAIL PROTECTED]]
>>> Sent: Tuesday, July 24, 2012 2:00 PM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>>>
>>> Hello Yogesh,
>>>
>>>         Are any proxy settings involved??Error code 407 indicates that
>>> the client must first authenticate itself with the proxy in order to
>>> proceed further. Just make sure everything is in place.
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>> On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
>>>> Hi All,
>>>>
>>>> even If if perform count(*) operation on table it shows error,
>>>>
>>>> select count(*) from dummysite;
>>>>
>>>>
>>>> Total MapReduce jobs = 1
>>>> Launching Job 1 out of 1
>>>> Number of reduce tasks determined at compile time: 1
>>>> In order to change the average load for a reducer (in bytes):
>>>>   set hive.exec.reducers.bytes.per.reducer=<number>
>>>> In order to limit the maximum number of reducers:
>>>>   set hive.exec.reducers.max=<number>
>>>> In order to set a constant number of reducers:
>>>>   set mapred.reduce.tasks=<number>
>>>> Starting Job = job_201207231123_0011, Tracking URL >>>> http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
>>>> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
>>>> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
>>>> Hadoop job information for Stage-1: number of mappers: 1; number of
>>>> reducers: 1
>>>> 2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
>>>> 2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
>>>> 2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
>>>> 2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
>>>> Ended Job = job_201207231123_0011 with errors
>>>> Error during job, obtaining debugging information...

Hello Dear Mohammad :-),

Its O.K :-), Nitin & Bejoyed helped me a lot :-).
Yes I am using pseudo distributed mode.

Thanks & Regards
Yogesh Kumar
:-)
________________________________________
From: Mohammad Tariq [[EMAIL PROTECTED]]
Sent: Tuesday, July 24, 2012 5:29 PM
To: [EMAIL PROTECTED]
Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:

Hi yogesh,

     Sorry for being unresponsive. Have a tight schedule today. By
looking at your logs, It seems you are running your cluster in pseudo
distributed mode. If that is the case just make sure you have
commented the following line in your /etc/hosts file
127.0.1.1

Regards,
    Mohammad Tariq
On Tue, Jul 24, 2012 at 5:21 PM, Nitin Pawar <[EMAIL PROTECTED]> wrote:
> if you have a single node cluster, then just configure everything to
> localhost. But if you have two nodes then thats a problem cause in any
> cluster nodes need to talk with each other without authentication on
> network level
>
> On Tue, Jul 24, 2012 at 4:19 PM,  <[EMAIL PROTECTED]> wrote:
>> Thanks Nitin :-), Thank you very much.
>>
>> Yes my n/w setup with proxy authentication, if I do remove proxy foe HTTP, HTTPS then it works but the Internet goes down.
>>
>> I set the proxy bypass for machine and it worked. :-)
>>
>> Thanks a Lot
>> Yogesh Kumar
>> ________________________________________
>> From: Nitin Pawar [[EMAIL PROTECTED]]
>> Sent: Tuesday, July 24, 2012 2:13 PM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>>
>> yogesh,
>> is your network setup with proxy authentication? normally the error
>> which you are getting is, when a node tries to connect to another node
>> but then after some time, the connection needs to be reauthorized.
>>
>> when you do a select * from table, it does not launch a MR, it just
>> does a dfs -cat of the hdfs file
>>
>> you will need to check the cluster health of your hadoop cluster. Also
>> make sure that on all the nodes the default linux security is off
>> (iptables )
>>
>> On Tue, Jul 24, 2012 at 2:09 PM,  <[EMAIL PROTECTED]> wrote:
>>> Thanks Mohammad :-),
>>>
>>> I didn't get regarding which proxy settings Please explain it,
>>>
>>> If i do operations like.
>>> select * from dummysite,
>>> or
>>> select * from dummysite where id=10;
>>>
>>> such command shows proper result and doesn't through any error.
>>>
>>> Please suggest.
>>>
>>> Regards
>>> Yogesh Kumar
>>> ________________________________________
>>> From: Mohammad Tariq [[EMAIL PROTECTED]]
>>> Sent: Tuesday, July 24, 2012 2:00 PM
>>> To: [EMAIL PROTECTED]
>>> Subject: Re: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched:
>>>
>>> Hello Yogesh,
>>>
>>>         Are any proxy settings involved??Error code 407 indicates that
>>> the client must first authenticate itself with the proxy in order to
>>> proceed further. Just make sure everything is in place.
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>> On Tue, Jul 24, 2012 at 1:55 PM,  <[EMAIL PROTECTED]> wrote:
>>>> Hi All,
>>>>
>>>> even If if perform count(*) operation on table it shows error,
>>>>
>>>> select count(*) from dummysite;
>>>>
>>>>
>>>> Total MapReduce jobs = 1
>>>> Launching Job 1 out of 1
>>>> Number of reduce tasks determined at compile time: 1
>>>> In order to change the average load for a reducer (in bytes):
>>>>   set hive.exec.reducers.bytes.per.reducer=<number>
>>>> In order to limit the maximum number of reducers:
>>>>   set hive.exec.reducers.max=<number>
>>>> In order to set a constant number of reducers:
>>>>   set mapred.reduce.tasks=<number>
>>>> Starting Job = job_201207231123_0011, Tracking URL >>>> http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
>>>> Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job
>>>> -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011

Please do not print this email unless it is absolutely necessary.

The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments.

WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email.

www.wipro.com