Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Decision Tree implementation not working in cluster !


Copy link to this message
-
Decision Tree implementation not working in cluster !
Desicion tree is working perfectly in Eclipse Juno.

But wen i tried to run that in my cluster it is showing error
=====================================================================*In main() *
*In main() +++++++ run *
*13/12/05 16:10:40 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.*
*13/12/05 16:10:41 INFO mapred.FileInputFormat: Total input paths to
process : 1*
*13/12/05 16:10:41 INFO mapred.JobClient: Running job:
job_201310231648_0427*
*13/12/05 16:10:42 INFO mapred.JobClient:  map 0% reduce 0%*
*13/12/05 16:10:49 INFO mapred.JobClient:  map 100% reduce 0%*
*13/12/05 16:10:53 INFO mapred.JobClient:  map 100% reduce 100%*
*13/12/05 16:10:55 INFO mapred.JobClient: Job complete:
job_201310231648_0427*
*13/12/05 16:10:55 INFO mapred.JobClient: Counters: 33*
*13/12/05 16:10:55 INFO mapred.JobClient:   File System Counters*
*13/12/05 16:10:55 INFO mapred.JobClient:     FILE: Number of bytes
read=987*
*13/12/05 16:10:55 INFO mapred.JobClient:     FILE: Number of bytes
written=589647*
*13/12/05 16:10:55 INFO mapred.JobClient:     FILE: Number of read
operations=0*
*13/12/05 16:10:55 INFO mapred.JobClient:     FILE: Number of large read
operations=0*
*13/12/05 16:10:55 INFO mapred.JobClient:     FILE: Number of write
operations=0*
*13/12/05 16:10:55 INFO mapred.JobClient:     HDFS: Number of bytes
read=846*
*13/12/05 16:10:55 INFO mapred.JobClient:     HDFS: Number of bytes
written=264*
*13/12/05 16:10:55 INFO mapred.JobClient:     HDFS: Number of read
operations=5*
*13/12/05 16:10:55 INFO mapred.JobClient:     HDFS: Number of large read
operations=0*
*13/12/05 16:10:55 INFO mapred.JobClient:     HDFS: Number of write
operations=2*
*13/12/05 16:10:55 INFO mapred.JobClient:   Job Counters *
*13/12/05 16:10:55 INFO mapred.JobClient:     Launched map tasks=2*
*13/12/05 16:10:55 INFO mapred.JobClient:     Launched reduce tasks=1*
*13/12/05 16:10:55 INFO mapred.JobClient:     Data-local map tasks=2*
*13/12/05 16:10:55 INFO mapred.JobClient:     Total time spent by all maps
in occupied slots (ms)=11179*
*13/12/05 16:10:55 INFO mapred.JobClient:     Total time spent by all
reduces in occupied slots (ms)=3964*
*13/12/05 16:10:55 INFO mapred.JobClient:     Total time spent by all maps
waiting after reserving slots (ms)=0*
*13/12/05 16:10:55 INFO mapred.JobClient:     Total time spent by all
reduces waiting after reserving slots (ms)=0*
*13/12/05 16:10:55 INFO mapred.JobClient:   Map-Reduce Framework*
*13/12/05 16:10:55 INFO mapred.JobClient:     Map input records=14*
*13/12/05 16:10:55 INFO mapred.JobClient:     Map output records=56*
*13/12/05 16:10:55 INFO mapred.JobClient:     Map output bytes=869*
*13/12/05 16:10:55 INFO mapred.JobClient:     Input split bytes=220*
*13/12/05 16:10:55 INFO mapred.JobClient:     Combine input records=0*
*13/12/05 16:10:55 INFO mapred.JobClient:     Combine output records=0*
*13/12/05 16:10:55 INFO mapred.JobClient:     Reduce input groups=20*
*13/12/05 16:10:55 INFO mapred.JobClient:     Reduce shuffle bytes=993*
*13/12/05 16:10:55 INFO mapred.JobClient:     Reduce input records=56*
*13/12/05 16:10:55 INFO mapred.JobClient:     Reduce output records=20*
*13/12/05 16:10:55 INFO mapred.JobClient:     Spilled Records=112*
*13/12/05 16:10:55 INFO mapred.JobClient:     CPU time spent (ms)=2450*
*13/12/05 16:10:55 INFO mapred.JobClient:     Physical memory (bytes)
snapshot=636420096*
*13/12/05 16:10:55 INFO mapred.JobClient:     Virtual memory (bytes)
snapshot=2955980800*
*13/12/05 16:10:55 INFO mapred.JobClient:     Total committed heap usage
(bytes)=524484608*
*13/12/05 16:10:55 INFO mapred.JobClient:   File Input Format Counters *
*13/12/05 16:10:55 INFO mapred.JobClient:     Bytes Read=417*
*Current  NODE INDEX . ::0*
*java.io.FileNotFoundException: C45/intermediate0.txt (No such file or
directory)*
* at java.io.FileInputStream.open(Native Method)*
* at java.io.FileInputStream.<init>(FileInputStream.java:138)*
* at java.io.FileInputStream.<init>(FileInputStream.java:97)*
* at tech.GainRatio.getcount(GainRatio.java:110)*
* at tech.C45.main(C45.java:105)*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)*
* at java.lang.reflect.Method.invoke(Method.java:601)*
* at org.apache.hadoop.util.RunJar.main(RunJar.java:208)*
*Exception in thread "main" java.lang.NumberFormatException: null*
* at java.lang.Integer.parseInt(Integer.java:454)*
* at java.lang.Integer.parseInt(Integer.java:527)*
* at tech.GainRatio.currNodeEntophy(GainRatio.java:28)*
* at tech.C45.main(C45.java:106)*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)*
* at java.lang.reflect.Method.invoke(Method.java:601)*
* at org.apache.hadoop.util.RunJar.main(RunJar.java:208)*
=====================================================================But the folder "C45 " is created ,
but the intermediate files are not created within it why is it so?
Code availabe :https://github.com/studhadoop/DT
Any suggesions

*Thanks & Regards*

Unmesha Sreeveni U.B

*Junior Developer*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB