Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Automatically upload files into HDFS


Copy link to this message
-
Re: Automatically upload files into HDFS
Hello Khasif,
Sorry for late reply...Are you done? or u still struggling?

mail me: [EMAIL PROTECTED]
Regards,
Som Shekhar Sharma

On Wed, Nov 21, 2012 at 6:06 PM, kashif khan <[EMAIL PROTECTED]> wrote:

> Dear Shankar Sharma.
>
> I am using Eclipse as IDE. I dont have any idea, how to create the project
> as maven project. I have downloaded Mave2 but given me some strange error.
> So if you can help me then I will try the maven. Actually, I am trying to
> automatically upload the files into HDFS and then will apply some
> algorithms to analyze the data. The algorithms will implement in mapreduce
> . So if you think maven will good for me then please let me know how I can
> create the project as maven project.
>
>
> Many thanks
>
> Best regards,
>
> KK
>
>
>
>  On Tue, Nov 20, 2012 at 7:06 PM, shekhar sharma <[EMAIL PROTECTED]>wrote:
>
>> By the way how are you building and running your project.Are u running
>> from any IDE?
>> The best practises you can follow:
>>
>> (1) Create your project as maven project and give the dependency of
>> hadoop-X.Y.Z . So your project will automatically will have all the
>> necessary jars
>> and i am sure you will not face these kind of errors
>> (2) And In your HADOOP_CLASSPATH provide the path for $HADOOP_LIB.
>>
>> Regards,
>> SOm
>>
>>
>>
>>
>>
>> On Tue, Nov 20, 2012 at 9:52 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>
>>> Dear Tariq
>>>
>>> Many thanks, finally I have created the directory and upload the file.
>>>
>>> Once again many thanks
>>>
>>> Best regards
>>>
>>>
>>> On Tue, Nov 20, 2012 at 3:04 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>
>>>> Dear Many thanks
>>>>
>>>>
>>>> I have downloaded the jar file and added to project. Now getting
>>>> another error as:
>>>>
>>>> og4j:WARN No appenders could be found for logger
>>>> (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
>>>> log4j:WARN Please initialize the log4j system properly.
>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor more info.
>>>> Exception in thread "main" java.io.IOException: No FileSystem for
>>>> scheme: hdfs
>>>>     at
>>>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>>>>     at
>>>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>>>>     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
>>>>     at
>>>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
>>>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
>>>>
>>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
>>>>     at CopyFile.main(CopyFile.java:14)
>>>>
>>>> Have any idea about this?
>>>>
>>>> Thanks again
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Nov 20, 2012 at 2:53 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> You can download the jar here :
>>>>> http://search.maven.org/remotecontent?filepath=com/google/guava/guava/13.0.1/guava-13.0.1.jar
>>>>>
>>>>> Regards,
>>>>>     Mohammad Tariq
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 20, 2012 at 8:06 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Could please let me know the name of jar file and location
>>>>>>
>>>>>> Many thanks
>>>>>>
>>>>>> Best regards
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 20, 2012 at 2:33 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>> Download the required jar and include it in your project.
>>>>>>>
>>>>>>> Regards,
>>>>>>>     Mohammad Tariq
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 20, 2012 at 7:57 PM, kashif khan <[EMAIL PROTECTED]
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Dear Tariq Thanks
>>>>>>>>
>>>>>>>> I have added the jar files from Cdh and download the cdh4 eclipse
>>>>>>>> plugin and copied into eclipse plugin folder. The previous error I think
>>>>>>>> sorted out but now I am getting another strange error.
>>>>>>>>
>>>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>>>>> com/google/common/collect/Maps