Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Program trying to read from local instead of hdfs


Copy link to this message
-
Re: Program trying to read from local instead of hdfs
And what's happening if you give the complete path, say
hdfs://your_namenode:9000/user/hduser/data/input1.txt ??

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Fri, Jan 18, 2013 at 5:36 AM, jamal sasha <[EMAIL PROTECTED]> wrote:

> No.
>  Its not working. :( same error.
>
>
>
> On Thu, Jan 17, 2013 at 4:00 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>
>> Hello Jamal,
>>
>>     Add the following 2 lines in your code and see if it works for you :
>>
>> Configuration.addResource(new Path("PATH_TO_YOUR_core-site.xml"));
>> Configuration.addResource(new Path("PATH_TO_YOUR_hdfs-site.xml"));
>>
>> Warm Regards,
>> Tariq
>> https://mtariq.jux.com/
>> cloudfront.blogspot.com
>>
>>
>> On Fri, Jan 18, 2013 at 5:26 AM, jamal sasha <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>   I am not sure what I am doing wrong.
>>> I copy my input files from local to hdfs at local
>>>
>>> /user/hduser/data/input1.txt
>>> /user/hduser/data/input2.txt
>>>
>>> In my driver code: I have
>>>  MultipleInputs.addInputPath(conf, new Path(args[0]),
>>> TextInputFormat.class, UserFileMapper.class);
>>>               MultipleInputs.addInputPath(conf, new Path(args[1]),
>>> TextInputFormat.class, DeliveryFileMapper.class);
>>>
>>> And then when i try to run the code I get an error:
>>> Exception in thread "main" java.io.FileNotFoundException: File
>>> /user/hduser/data/input1.txt does not exist.
>>>
>>> And lets say on my local I have the path as
>>> /Users/local/project/input1.txt
>>> /Users/local/project/input2.txt
>>> And then when I try to run the file it throws an error
>>> 13/01/17 15:26:06 INFO mapred.JobClient: Cleaning up the staging area
>>> hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/hduser/.staging/job_201301021121_0151
>>> 13/01/17 15:26:06 ERROR security.UserGroupInformation:
>>> PriviledgedActionException as:mhduser
>>> cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not
>>> exist: hdfs://localhost:54310/Users/local/project/input1.txt
>>>
>>> Well offcourse, its not in there.. its on my local.
>>> So when I give the hdfs localtion.. it reads thru local and when i give
>>> my local location.. it is pointing to hdfs..
>>> What am I doing wrong?
>>> For reference, I am trying to run this code:
>>> http://kickstarthadoop.blogspot.com/2011/09/joins-with-plain-map-reduce.html
>>> THanks
>>>
>>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB