Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Program trying to read from local instead of hdfs


Copy link to this message
-
Re: Program trying to read from local instead of hdfs
No.
 Its not working. :( same error.

On Thu, Jan 17, 2013 at 4:00 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello Jamal,
>
>     Add the following 2 lines in your code and see if it works for you :
>
> Configuration.addResource(new Path("PATH_TO_YOUR_core-site.xml"));
> Configuration.addResource(new Path("PATH_TO_YOUR_hdfs-site.xml"));
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Fri, Jan 18, 2013 at 5:26 AM, jamal sasha <[EMAIL PROTECTED]>wrote:
>
>> Hi,
>>   I am not sure what I am doing wrong.
>> I copy my input files from local to hdfs at local
>>
>> /user/hduser/data/input1.txt
>> /user/hduser/data/input2.txt
>>
>> In my driver code: I have
>>  MultipleInputs.addInputPath(conf, new Path(args[0]),
>> TextInputFormat.class, UserFileMapper.class);
>>               MultipleInputs.addInputPath(conf, new Path(args[1]),
>> TextInputFormat.class, DeliveryFileMapper.class);
>>
>> And then when i try to run the code I get an error:
>> Exception in thread "main" java.io.FileNotFoundException: File
>> /user/hduser/data/input1.txt does not exist.
>>
>> And lets say on my local I have the path as
>> /Users/local/project/input1.txt
>> /Users/local/project/input2.txt
>> And then when I try to run the file it throws an error
>> 13/01/17 15:26:06 INFO mapred.JobClient: Cleaning up the staging area
>> hdfs://localhost:54310/app/hadoop/tmp/mapred/staging/hduser/.staging/job_201301021121_0151
>> 13/01/17 15:26:06 ERROR security.UserGroupInformation:
>> PriviledgedActionException as:mhduser
>> cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not
>> exist: hdfs://localhost:54310/Users/local/project/input1.txt
>>
>> Well offcourse, its not in there.. its on my local.
>> So when I give the hdfs localtion.. it reads thru local and when i give
>> my local location.. it is pointing to hdfs..
>> What am I doing wrong?
>> For reference, I am trying to run this code:
>> http://kickstarthadoop.blogspot.com/2011/09/joins-with-plain-map-reduce.html
>> THanks
>>
>>
>