Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Program trying to read from local instead of hdfs


Copy link to this message
-
Program trying to read from local instead of hdfs
Hi,
  I am not sure what I am doing wrong.
I copy my input files from local to hdfs at local

/user/hduser/data/input1.txt
/user/hduser/data/input2.txt

In my driver code: I have
 MultipleInputs.addInputPath(conf, new Path(args[0]),
TextInputFormat.class, UserFileMapper.class);
              MultipleInputs.addInputPath(conf, new Path(args[1]),
TextInputFormat.class, DeliveryFileMapper.class);

And then when i try to run the code I get an error:
Exception in thread "main" java.io.FileNotFoundException: File
/user/hduser/data/input1.txt does not exist.

And lets say on my local I have the path as
/Users/local/project/input1.txt
/Users/local/project/input2.txt
And then when I try to run the file it throws an error
13/01/17 15:26:06 INFO mapred.JobClient: Cleaning up the staging area
hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/hduser/.staging/job_201301021121_0151
13/01/17 15:26:06 ERROR security.UserGroupInformation:
PriviledgedActionException as:mhduser
cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://localhost:54310/Users/local/project/input1.txt

Well offcourse, its not in there.. its on my local.
So when I give the hdfs localtion.. it reads thru local and when i give my
local location.. it is pointing to hdfs..
What am I doing wrong?
For reference, I am trying to run this code:
http://kickstarthadoop.blogspot.com/2011/09/joins-with-plain-map-reduce.html
THanks
+
Mohammad Tariq 2013-01-18, 00:00
+
jamal sasha 2013-01-18, 00:06
+
Jay Vyas 2013-01-18, 00:16
+
Mohammad Tariq 2013-01-18, 00:14
+
jamal sasha 2013-01-18, 00:18
+
Mohammad Tariq 2013-01-18, 01:04
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB