Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user -


Copy link to this message
-
Re:
Harsh J 2013-06-01, 16:43
Shahab - I see he has mentioned generally that security is enabled
(but not that it happens iff security is enabled), and the issue here
doesn't have anything to do with security really.

Azurry - Lets discuss the code issues on the JIRA (instead of here) or
on the mapreduce-dev lists.

On Sat, Jun 1, 2013 at 10:05 PM, Shahab Yunus <[EMAIL PROTECTED]> wrote:
> HI Harsh,
>
> Quick question though: why do you think it only happens if the OP 'uses
> security' as he mentioned?
>
> Regards,
> Shahab
>
>
> On Sat, Jun 1, 2013 at 11:49 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> Does smell like a bug as that number you get is simply Long.MAX_VALUE,
>> or 8 exbibytes.
>>
>> Looking at the sources, this turns out to be a rather funny Java issue
>> (there's a divide by zero happening and [1] suggests Long.MAX_VALUE
>> return in such a case). I've logged a bug report for this at
>> https://issues.apache.org/jira/browse/MAPREDUCE-5288 with a
>> reproducible case.
>>
>> Does this happen consistently for you?
>>
>> [1]
>> http://docs.oracle.com/javase/6/docs/api/java/lang/Math.html#round(double)
>>
>> On Sat, Jun 1, 2013 at 7:27 PM, Lanati, Matteo <[EMAIL PROTECTED]>
>> wrote:
>> > Hi all,
>> >
>> > I stumbled upon this problem as well while trying to run the default
>> > wordcount shipped with Hadoop 1.2.0. My testbed is made up of 2 virtual
>> > machines: Debian 7, Oracle Java 7, 2 GB RAM, 25 GB hard disk. One node is
>> > used as JT+NN, the other as TT+DN. Security is enabled. The input file is
>> > about 600 kB and the error is
>> >
>> > 2013-06-01 12:22:51,999 WARN org.apache.hadoop.mapred.JobInProgress: No
>> > room for map task. Node 10.156.120.49 has 22854692864 bytes free; but we
>> > expect map to take 9223372036854775807
>> >
>> > The logfile is attached, together with the configuration files. The
>> > version I'm using is
>> >
>> > Hadoop 1.2.0
>> > Subversion
>> > https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r
>> > 1479473
>> > Compiled by hortonfo on Mon May  6 06:59:37 UTC 2013
>> > From source with checksum 2e0dac51ede113c1f2ca8e7d82fb3405
>> > This command was run using
>> > /home/lu95jib/hadoop-exmpl/hadoop-1.2.0/hadoop-core-1.2.0.jar
>> >
>> > If I run the default configuration (i.e. no securty), then the job
>> > succeeds.
>> >
>> > Is there something missing in how I set up my nodes? How is it possible
>> > that the envisaged value for the needed space is so big?
>> >
>> > Thanks in advance.
>> >
>> > Matteo
>> >
>> >
>> >
>> >>Which version of Hadoop are you using. A quick search shows me a bug
>> >>https://issues.apache.org/jira/browse/HADOOP-5241 that seems to show
>> >>similar symptoms. However, that was fixed a long while ago.
>> >>
>> >>
>> >>On Sat, Mar 23, 2013 at 4:40 PM, Redwane belmaati cherkaoui <
>> >>[EMAIL PROTECTED]> wrote:
>> >>
>> >>> This the content of the jobtracker log file :
>> >>> 2013-03-23 12:06:48,912 INFO org.apache.hadoop.mapred.JobInProgress:
>> >>> Input
>> >>> size for job job_201303231139_0001 = 6950001. Number of splits = 7
>> >>> 2013-03-23 12:06:48,925 INFO org.apache.hadoop.mapred.JobInProgress:
>> >>> tip:task_201303231139_0001_m_000000 has split on
>> >>> node:/default-rack/hadoop0.novalocal
>> >>> 2013-03-23 12:06:48,927 INFO org.apache.hadoop.mapred.JobInProgress:
>> >>> tip:task_201303231139_0001_m_000001 has split on
>> >>> node:/default-rack/hadoop0.novalocal
>> >>> 2013-03-23 12:06:48,930 INFO org.apache.hadoop.mapred.JobInProgress:
>> >>> tip:task_201303231139_0001_m_000002 has split on
>> >>> node:/default-rack/hadoop0.novalocal
>> >>> 2013-03-23 12:06:48,931 INFO org.apache.hadoop.mapred.JobInProgress:
>> >>> tip:task_201303231139_0001_m_000003 has split on
>> >>> node:/default-rack/hadoop0.novalocal
>> >>> 2013-03-23 12:06:48,933 INFO org.apache.hadoop.mapred.JobInProgress:
>> >>> tip:task_201303231139_0001_m_000004 has split on
>> >>> node:/default-rack/hadoop0.novalocal
>> >>> 2013-03-23 12:06:48,934 INFO org.apache.hadoop.mapred.JobInProgress:

Harsh J