Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> setting hadoop for pseudo distributed mode.


Copy link to this message
-
setting hadoop for pseudo distributed mode.
Hi,
  So I am still in process of learning hadoop.
I tried to run wordcount.java (by writing my own mapper reducer.. creating
jar and then running it in a pseudo distributed mode).

At that time I got an error, something like
 ERROR security.UserGroupInformation: PriviledgedActionException as:mhduser
cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://localhost:54310/user/hduser/wiki
So I googled around.. and found that I should put the following two lines
on my driver code:

    conf.addResource(new Path("/usr/local/hadoop/conf/core-site.xml"));
    conf.addResource(new Path("/usr/local/hadoop/conf/hdfs-site.xml file"));

(path of where my core-site and hdfs-site are) and after that it ran just
fine.

Q1) How did putting those two lines solved the issue??
Q2)
  I am now using third party libraries which are taking input from hdfs and
writing output to hdfs...
But in an intermediatory step, it creates a raw output.. I am again getting
the error:
 ERROR security.UserGroupInformation: PriviledgedActionException as:mhduser
cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://localhost:54310/user/hduser/wiki-inter-output
How do I resolve this?

Any suggestions.
THanks
Jamal.