Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> mysterious NumberFormatException


Copy link to this message
-
Re: mysterious NumberFormatException
MAPREDUCE-3583 has been logged.

Cheers

On Tue, Dec 20, 2011 at 5:11 AM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:

> Hi Ted,
>
>
> Use String is a good idea.  Or we may use Long but we have to be more
> careful to parse 64-bit integers (use negative long when >= 2^63.)  Either
> way is fine.
>
>
> Could you file a JIRA?  Thanks for catching the bug.
>
>
> Nicholas
>
>
>
> ________________________________
>  From: Ted Yu <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]; Tsz Wo Sze <[EMAIL PROTECTED]>
> Cc: giridharan kesavan <[EMAIL PROTECTED]>
> Sent: Monday, December 19, 2011 11:20 PM
> Subject: Re: mysterious NumberFormatException
>
> Thanks for the analysis, Nicolas.
>
> Is it reasonable to change allProcessInfo to Map<String, ProcessInfo> so
> that we don't encounter this problem by avoiding parsing large integer ?
>
> On Mon, Dec 19, 2011 at 9:59 PM, Tsz Wo Sze <[EMAIL PROTECTED]> wrote:
>
> > Hi,
> >
> > It looks like that the ppid is a 64-bit positive integer but Java long is
> > signed and so only works with 63-bit positive integers.  In your case,
> >
> >   2^64 > 18446743988060683582 > 2^63.
> >
> > Therefore, there is a NFE.  I think it is a bug in
> ProcfsBasedProcessTree.
> >
> >
> > Regards,
> >
> > Nicholas Sze
> >
> >
> >
> > ________________________________
> >  From: Ted Yu <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Cc: giridharan kesavan <[EMAIL PROTECTED]>
> > Sent: Monday, December 19, 2011 8:24 PM
> > Subject: mysterious NumberFormatException
> >
> > Hi,
> > HBase PreCommit builds frequently gave us mysterious
> NumberFormatException
> >
> > From
> >
> >
> https://builds.apache.org/job/PreCommit-HBASE-Build/553//testReport/org.apache.hadoop.hbase.mapreduce/TestHFileOutputFormat/testMRIncrementalLoad/
> > :
> >
> > 2011-12-20
>  01:44:01,180 WARN  [main] mapred.JobClient(784): No job jar
> > file set.  User classes may not be found. See JobConf(Class) or
> > JobConf#setJar(String).
> > java.lang.NumberFormatException: For input string: "18446743988060683582"
> >     at
> >
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> >     at java.lang.Long.parseLong(Long.java:422)
> >     at java.lang.Long.parseLong(Long.java:468)
> >     at
> >
> org.apache.hadoop.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:413)
> >     at
> >
> org.apache.hadoop.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:148)
> >     at
> >
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin.getProcResourceValues(LinuxResourceCalculatorPlugin.java:401)
> >     at
>  org.apache.hadoop.mapred.Task.initialize(Task.java:536)
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
> >     at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >     at java.security.AccessController.doPrivileged(Native Method)
> >     at javax.security.auth.Subject.doAs(Subject.java:396)
> >     at
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
> >     at org.apache.hadoop.mapred.Child.main(Child.java:249)
> >
> > From hadoop 0.20.205 source code, looks like ppid was
> 18446743988060683582,
> > causing NFE:
> >         // Set (name) (ppid) (pgrpId) (session) (utime) (stime) (vsize)
> > (rss)
> >          pinfo.updateProcessInfo(m.group(2),
> Integer.parseInt(m.group(3)),
> >
> > You can find information on the OS at the
>  beginning of
> > https://builds.apache.org/job/PreCommit-HBASE-Build/553/console:
> >
> > asf011.sp2.ygridcore.net
> > Linux asf011.sp2.ygridcore.net 2.6.32-33-server #71-Ubuntu SMP Wed Jul
> > 20 17:42:25 UTC 2011 x86_64 GNU/Linux
> > core file size          (blocks, -c) 0
> > data seg size           (kbytes, -d) unlimited
> > scheduling priority             (-e) 20
> > file size               (blocks, -f) unlimited
> > pending signals                 (-i) 16382
> > max locked memory       (kbytes, -l) 64
> > max memory size         (kbytes, -m) unlimited
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB