Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Error:Hdfs Client for hadoop using native java api
Sandeep Reddy P
2012-07-18, 15:33
John Hancock
2012-07-19, 09:41
Sandeep Reddy P
2012-07-19, 13:16
shashwat shriparv
2012-07-19, 16:02
shashwat shriparv
2012-07-19, 16:05
Sandeep Reddy P
2012-07-19, 17:18
Minh Duc Nguyen
2012-07-22, 16:26
shashwat shriparv
2012-07-22, 20:31
Sandeep Reddy P
2012-07-22, 21:48
Shailesh Samudrala
2012-07-30, 20:24
Copy link to this message
-
Re: Error:Hdfs Client for hadoop using native java api
shashwat shriparv 2012-07-19, 16:06
Or try something like this

Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(conf);

HFile.Writer hwriter = new HFileWriterV2(conf, new CacheConfig(conf), fs, new
Path(fs.getWorkingDirectory() + "/foo"));

On Thu, Jul 19, 2012 at 9:35 PM, shashwat shriparv <
[EMAIL PROTECTED]> wrote:

> By the way you have these two options
>
>
>    1.
>
>    Put the configuration files in the classpath, so that the code picks
>    it.
>    2.
>
>    Use Configuration.set()<http://hadoop.apache.org/common/docs/r0.21.0/api/org/apache/hadoop/conf/Configuration.html#set%28java.lang.String,%20java.lang.String%29>to set the required parameters in the code.
>
> because the configuration may be directing to local file whice code is nt
> abl to find
>
>
> On Thu, Jul 19, 2012 at 9:32 PM, shashwat shriparv <
> [EMAIL PROTECTED]> wrote:
>
>> Can you provide the code(some part of it if you don't want to throw the
>> full code here) and let us know which part of your code is throwing this
>> error.
>>
>> Regards
>>
>> ∞
>> Shashwat Shriparv
>>
>>
>>
>>
>> On Thu, Jul 19, 2012 at 6:46 PM, Sandeep Reddy P <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi John,
>>> We have applications in windows. So our dev's need to connect to HDFS
>>> from
>>> eclipse installed in windows. I'm trying to put data from <local-file> to
>>> <hdfs-file> using java code from windows.
>>>
>>> On Thu, Jul 19, 2012 at 5:41 AM, John Hancock <[EMAIL PROTECTED]
>>> >wrote:
>>>
>>> > Sandeep,
>>> >
>>> > I don't understand your situation completely, but why not just use
>>> > bin/hadoop dfs -copyFromLocal <local-file-name> <hdfs-file-name> ?
>>> >
>>> > -John
>>> >
>>> > On Wed, Jul 18, 2012 at 11:33 AM, Sandeep Reddy P <
>>> > [EMAIL PROTECTED]> wrote:
>>> >
>>> > > Hi,
>>> > > I'm trying to load data into hdfs from local linux file system using
>>> java
>>> > > code from a windows machine.But i'm getting the error
>>> > >
>>> > > java.lang.IllegalArgumentException: Wrong FS:
>>> > > hdfs://hadoop1.devqa.local:8020/user/hdfs/java, expected: file:///
>>> > >     at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:410)
>>> > >     at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:56)
>>> > >     at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:404)
>>> > >     at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:251)
>>> > >     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:797)
>>> > >     at org.apache.hadoop.fs.FileUtil.checkDest(FileUtil.java:349)
>>> > >     at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:205)
>>> > >     at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:157)
>>> > >     at
>>> > >
>>> > >
>>> >
>>> org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:55)
>>> > >     at
>>> > >
>>> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1292)
>>> > >     at Hdfs.main(Hdfs.java:18)
>>> > > File not found
>>> > >
>>> > > Can any one please help me with the issue.
>>> > >
>>> > > --
>>> > > Thanks,
>>> > > sandeep
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> Thanks,
>>> sandeep
>>>
>>
>>
>>
>> --
>>
>>
>> ∞
>> Shashwat Shriparv
>>
>>
>>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
>
>
--
∞
Shashwat Shriparv