Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Automatically upload files into HDFS


Copy link to this message
-
Re: Automatically upload files into HDFS
shekhar sharma 2012-11-20, 19:06
By the way how are you building and running your project.Are u running from
any IDE?
The best practises you can follow:

(1) Create your project as maven project and give the dependency of
hadoop-X.Y.Z . So your project will automatically will have all the
necessary jars
and i am sure you will not face these kind of errors
(2) And In your HADOOP_CLASSPATH provide the path for $HADOOP_LIB.

Regards,
SOm

On Tue, Nov 20, 2012 at 9:52 PM, kashif khan <[EMAIL PROTECTED]> wrote:

> Dear Tariq
>
> Many thanks, finally I have created the directory and upload the file.
>
> Once again many thanks
>
> Best regards
>
>
> On Tue, Nov 20, 2012 at 3:04 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>
>> Dear Many thanks
>>
>>
>> I have downloaded the jar file and added to project. Now getting another
>> error as:
>>
>> og4j:WARN No appenders could be found for logger
>> (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
>> log4j:WARN Please initialize the log4j system properly.
>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
>> more info.
>> Exception in thread "main" java.io.IOException: No FileSystem for scheme:
>> hdfs
>>     at
>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>>     at
>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>>     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
>>     at
>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
>>
>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
>>     at CopyFile.main(CopyFile.java:14)
>>
>> Have any idea about this?
>>
>> Thanks again
>>
>>
>>
>>
>>
>> On Tue, Nov 20, 2012 at 2:53 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>
>>> You can download the jar here :
>>> http://search.maven.org/remotecontent?filepath=com/google/guava/guava/13.0.1/guava-13.0.1.jar
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>>
>>> On Tue, Nov 20, 2012 at 8:06 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>
>>>> Could please let me know the name of jar file and location
>>>>
>>>> Many thanks
>>>>
>>>> Best regards
>>>>
>>>>
>>>> On Tue, Nov 20, 2012 at 2:33 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Download the required jar and include it in your project.
>>>>>
>>>>> Regards,
>>>>>     Mohammad Tariq
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 20, 2012 at 7:57 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Dear Tariq Thanks
>>>>>>
>>>>>> I have added the jar files from Cdh and download the cdh4 eclipse
>>>>>> plugin and copied into eclipse plugin folder. The previous error I think
>>>>>> sorted out but now I am getting another strange error.
>>>>>>
>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>>> com/google/common/collect/Maps
>>>>>>     at
>>>>>> org.apache.hadoop.metrics2.lib.MetricsRegistry.<init>(MetricsRegistry.java:42)
>>>>>>     at
>>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:87)
>>>>>>     at
>>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:133)
>>>>>>     at
>>>>>> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38)
>>>>>>     at
>>>>>> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36)
>>>>>>     at
>>>>>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:97)
>>>>>>     at
>>>>>> org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:190)
>>>>>>     at
>>>>>> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2373)
>>>>>>     at
>>>>>> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2365)
>>>>>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2233)
>>>>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)