Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Debuging hadoop core


+
Pramy Bhats 2010-07-13, 22:07
+
Pramy Bhats 2010-07-13, 22:31
+
Ted Yu 2010-07-14, 03:09
+
Pramy Bhats 2010-07-14, 05:38
Copy link to this message
-
Re: Debuging hadoop core
It points to hdfs.
Please replace with your namenode IP or hostname.

On Tue, Jul 13, 2010 at 10:38 PM, Pramy Bhats <[EMAIL PROTECTED]>wrote:

> Hi Ted,
>
> Sorry, I didn't get your solution. Why would I put this value
> <value>hdfs://
> sjc9-flash-grid04.ciq.com:9000</value> ?
>
> Could you please explain me the reasoning behind this ?
>
> thanks,
> --PB
>
> On Wed, Jul 14, 2010 at 5:09 AM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Find hadoop-site.xml which Eclipse claimed was in your classpath.
> > In the same directory, look for core-site.xml and add the following:
> > <property>
> >  <name>fs.default.name</name>
> >  <value>hdfs://sjc9-flash-grid04.ciq.com:9000</value>
> >
> > On Tue, Jul 13, 2010 at 3:07 PM, Pramy Bhats <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi,
> > >
> > > I am trying to debug the new built hadoop-core-dev.jar in Eclipse. To
> > > simplify the debug process, firstly I setup the Hadoop in single-node
> > mode
> > > on my localhost.
> > >
> > >
> > > a)  configure debug in eclipse,
> > >
> > >    under tab main:
> > >      project : hadoop-all
> > >      main-class: org.apache.hadoop.util.RunJar
> > >
> > >   under tab arguments:
> > >
> > >   program arguments: <absolute path for wordcount jar
> file>/wordcount.jar
> > >  org.wordcount.WordCount   <input-text-file-already-in-hdfs> (text)
> > >  <desired-output-file> (output)
> > >   VM arguments: -Xmx256M
> > >
> > >
> > >  under tab classpath:
> > >
> > >  user entries :  add external jar  ( hadoop-0.20.3-core-dev.jar ) ==>
> so
> > > that I can debug my new built hadoop core jar.
> > >
> > >
> > > under tab source:
> > >
> > >  I add the source file folder for the wordcount example ( in order
> lookup
> > > for the debug process).
> > >
> > >
> > > I apply these configuration and start debug process.
> > >
> > >
> > > b)  the debugging works fine, and i can perform all operations for
> debug.
> > > However, i get following problem
> > >
> > > 2010-07-14 00:02:15,816 WARN  conf.Configuration
> > > (Configuration.java:<clinit>(176)) - DEPRECATED: hadoop-site.xml found
> in
> > > the classpath. Usage of hadoop-site.xml is deprecated. Instead use
> > > core-site.xml, mapred-site.xml and hdfs-site.xml to override properties
> > of
> > > core-default.xml, mapred-default.xml and hdfs-default.xml respectively
> > > 2010-07-14 00:02:16,535 INFO  jvm.JvmMetrics (JvmMetrics.java:init(71))
> -
> > > Initializing JVM Metrics with processName=JobTracker, sessionId> > > Exception in thread "main"
> > > org.apache.hadoop.mapreduce.lib.input.InvalidInputException: *Input
> path
> > > does not exist: file:/home/hadoop/code/hadoop-0.20.2/text*
> > > at
> > >
> > >
> >
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:224)
> > > at
> > >
> > >
> >
> org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:241)
> > > at
> org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:885)
> > > at
> > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779)
> > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
> > > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447)
> > > at org.selfadjust.wordcount.WordCount.run(WordCount.java:32)
> > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> > > at org.selfadjust.wordcount.WordCount.main(WordCount.java:43)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > > at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > >
> > >
> > > However, the file named "text" is the file already stored in the hdfs.
> > >
> > >
> > > Could you please help me with debugging process here, any pointers to