Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Automatically upload files into HDFS


+
kashif khan 2012-11-19, 10:44
+
Mohammad Tariq 2012-11-19, 10:50
+
kashif khan 2012-11-19, 12:30
+
Mohammad Tariq 2012-11-19, 12:34
+
Mohammad Tariq 2012-11-19, 12:35
+
Alexander Alten-Lorenz 2012-11-19, 12:26
+
kashif khan 2012-11-19, 12:35
+
Mohammad Tariq 2012-11-19, 12:41
+
kashif khan 2012-11-19, 12:53
+
Mohammad Tariq 2012-11-19, 13:18
+
kashif khan 2012-11-19, 14:01
+
Mohammad Tariq 2012-11-19, 14:04
+
kashif khan 2012-11-19, 14:29
+
kashif khan 2012-11-19, 14:43
+
Mohammad Tariq 2012-11-19, 14:53
+
kashif khan 2012-11-19, 15:01
+
Mohammad Tariq 2012-11-19, 15:10
+
kashif khan 2012-11-19, 15:34
+
Mohammad Tariq 2012-11-19, 15:41
+
kashif khan 2012-11-20, 10:40
+
Mohammad Tariq 2012-11-20, 14:19
+
kashif khan 2012-11-20, 14:27
+
Mohammad Tariq 2012-11-20, 14:33
+
kashif khan 2012-11-20, 14:36
+
Mohammad Tariq 2012-11-20, 14:53
+
kashif khan 2012-11-20, 15:04
+
kashif khan 2012-11-20, 16:22
+
shekhar sharma 2012-11-20, 19:06
Copy link to this message
-
Re: Automatically upload files into HDFS
Dear Shankar Sharma.

I am using Eclipse as IDE. I dont have any idea, how to create the project
as maven project. I have downloaded Mave2 but given me some strange error.
So if you can help me then I will try the maven. Actually, I am trying to
automatically upload the files into HDFS and then will apply some
algorithms to analyze the data. The algorithms will implement in mapreduce
. So if you think maven will good for me then please let me know how I can
create the project as maven project.

Many thanks

Best regards,

KK

On Tue, Nov 20, 2012 at 7:06 PM, shekhar sharma <[EMAIL PROTECTED]>wrote:

> By the way how are you building and running your project.Are u running
> from any IDE?
> The best practises you can follow:
>
> (1) Create your project as maven project and give the dependency of
> hadoop-X.Y.Z . So your project will automatically will have all the
> necessary jars
> and i am sure you will not face these kind of errors
> (2) And In your HADOOP_CLASSPATH provide the path for $HADOOP_LIB.
>
> Regards,
> SOm
>
>
>
>
>
> On Tue, Nov 20, 2012 at 9:52 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>
>> Dear Tariq
>>
>> Many thanks, finally I have created the directory and upload the file.
>>
>> Once again many thanks
>>
>> Best regards
>>
>>
>> On Tue, Nov 20, 2012 at 3:04 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>
>>> Dear Many thanks
>>>
>>>
>>> I have downloaded the jar file and added to project. Now getting another
>>> error as:
>>>
>>> og4j:WARN No appenders could be found for logger
>>> (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
>>> log4j:WARN Please initialize the log4j system properly.
>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor more info.
>>> Exception in thread "main" java.io.IOException: No FileSystem for
>>> scheme: hdfs
>>>     at
>>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>>>     at
>>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>>>     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
>>>     at
>>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
>>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
>>>
>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
>>>     at CopyFile.main(CopyFile.java:14)
>>>
>>> Have any idea about this?
>>>
>>> Thanks again
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Nov 20, 2012 at 2:53 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>
>>>> You can download the jar here :
>>>> http://search.maven.org/remotecontent?filepath=com/google/guava/guava/13.0.1/guava-13.0.1.jar
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Tue, Nov 20, 2012 at 8:06 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Could please let me know the name of jar file and location
>>>>>
>>>>> Many thanks
>>>>>
>>>>> Best regards
>>>>>
>>>>>
>>>>> On Tue, Nov 20, 2012 at 2:33 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Download the required jar and include it in your project.
>>>>>>
>>>>>> Regards,
>>>>>>     Mohammad Tariq
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 20, 2012 at 7:57 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>> Dear Tariq Thanks
>>>>>>>
>>>>>>> I have added the jar files from Cdh and download the cdh4 eclipse
>>>>>>> plugin and copied into eclipse plugin folder. The previous error I think
>>>>>>> sorted out but now I am getting another strange error.
>>>>>>>
>>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>>>> com/google/common/collect/Maps
>>>>>>>     at
>>>>>>> org.apache.hadoop.metrics2.lib.MetricsRegistry.<init>(MetricsRegistry.java:42)
>>>>>>>     at
>>>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:87)
>>>>>>>     at
>>>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:133)
>>>>>>>     at
>>>>
+
shekhar sharma 2012-11-26, 16:42
+
kashif khan 2012-11-27, 21:25