Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Problem with MR job


Copy link to this message
-
Problem with MR job

Hi all,

We are trying to run a mahout job in a hadoop cluster, but we keep
getting the same status. The job passes the initial mahout stages and
when it comes to be executed as a MR job, it seems to be stuck at 0%
progress. Through the UI we see that it is submitted but not running.
After a while it gets killed. In the logs the error shown is this one:

2011-09-21 07:47:50,507 INFO org.apache.hadoop.mapred.JobTracker:
problem cleaning system directory:
hdfs://master/var/lib/hadoop-0.20/cache/hdfs/mapred/system
org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot create
directory /var/lib/hadoop-0.20/cache/hdfs/mapred/system. Name nod$
The reported blocks 0 needs additional 12 blocks to reach the threshold
0.9990 of total blocks 13. Safe mode will be turned off automatically.
         at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirsInternal(FSNamesystem.java:1966)
         at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:1940)
         at
org.apache.hadoop.hdfs.server.namenode.NameNode.mkdirs(NameNode.java:770)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
         at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
         at java.lang.reflect.Method.invoke(Method.java:597)
Some staging files seem to have been created however.

I was thinking of sending this to the mahout mailing list but it seems a
more core hadoop issue.

We are using the following command to launch the mahout example:
./mahout org.apache.mahout.clustering.syntheticcontrol.kmeans.Job
--input hdfs://master/user/hdfs/testdata/synthetic_control.data --output
hdfs://master/user/hdfs/testdata/output --t1 0.5 --t2 1 --maxIter 50

Any clues?
George

--

---------------------------

George Kousiouris
Electrical and Computer Engineer
Division of Communications,
Electronics and Information Engineering
School of Electrical and Computer Engineering
Tel: +30 210 772 2546
Mobile: +30 6939354121
Fax: +30 210 772 2569
Email: [EMAIL PROTECTED]
Site: http://users.ntua.gr/gkousiou/

National Technical University of Athens
9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB