Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Automatically upload files into HDFS


+
kashif khan 2012-11-19, 10:44
+
Mohammad Tariq 2012-11-19, 10:50
+
kashif khan 2012-11-19, 12:30
+
Mohammad Tariq 2012-11-19, 12:34
+
Mohammad Tariq 2012-11-19, 12:35
+
Alexander Alten-Lorenz 2012-11-19, 12:26
+
kashif khan 2012-11-19, 12:35
+
Mohammad Tariq 2012-11-19, 12:41
+
kashif khan 2012-11-19, 12:53
+
Mohammad Tariq 2012-11-19, 13:18
+
kashif khan 2012-11-19, 14:01
+
Mohammad Tariq 2012-11-19, 14:04
+
kashif khan 2012-11-19, 14:29
+
kashif khan 2012-11-19, 14:43
+
Mohammad Tariq 2012-11-19, 14:53
+
kashif khan 2012-11-19, 15:01
+
Mohammad Tariq 2012-11-19, 15:10
+
kashif khan 2012-11-19, 15:34
+
Mohammad Tariq 2012-11-19, 15:41
+
kashif khan 2012-11-20, 10:40
+
Mohammad Tariq 2012-11-20, 14:19
+
kashif khan 2012-11-20, 14:27
+
Mohammad Tariq 2012-11-20, 14:33
+
kashif khan 2012-11-20, 14:36
+
Mohammad Tariq 2012-11-20, 14:53
+
kashif khan 2012-11-20, 15:04
+
kashif khan 2012-11-20, 16:22
+
shekhar sharma 2012-11-20, 19:06
Copy link to this message
-
Re: Automatically upload files into HDFS
Dear Shankar Sharma.

I am using Eclipse as IDE. I dont have any idea, how to create the project
as maven project. I have downloaded Mave2 but given me some strange error.
So if you can help me then I will try the maven. Actually, I am trying to
automatically upload the files into HDFS and then will apply some
algorithms to analyze the data. The algorithms will implement in mapreduce
. So if you think maven will good for me then please let me know how I can
create the project as maven project.

Many thanks

Best regards,

KK

On Tue, Nov 20, 2012 at 7:06 PM, shekhar sharma <[EMAIL PROTECTED]>wrote:

> By the way how are you building and running your project.Are u running
> from any IDE?
> The best practises you can follow:
>
> (1) Create your project as maven project and give the dependency of
> hadoop-X.Y.Z . So your project will automatically will have all the
> necessary jars
> and i am sure you will not face these kind of errors
> (2) And In your HADOOP_CLASSPATH provide the path for $HADOOP_LIB.
>
> Regards,
> SOm
>
>
>
>
>
> On Tue, Nov 20, 2012 at 9:52 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>
>> Dear Tariq
>>
>> Many thanks, finally I have created the directory and upload the file.
>>
>> Once again many thanks
>>
>> Best regards
>>
>>
>> On Tue, Nov 20, 2012 at 3:04 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>
>>> Dear Many thanks
>>>
>>>
>>> I have downloaded the jar file and added to project. Now getting another
>>> error as:
>>>
>>> og4j:WARN No appenders could be found for logger
>>> (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
>>> log4j:WARN Please initialize the log4j system properly.
>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor more info.
>>> Exception in thread "main" java.io.IOException: No FileSystem for
>>> scheme: hdfs
>>>     at
>>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>>>     at
>>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>>>     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
>>>     at
>>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
>>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
>>>
>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
>>>     at CopyFile.main(CopyFile.java:14)
>>>
>>> Have any idea about this?
>>>
>>> Thanks again
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Nov 20, 2012 at 2:53 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>
>>>> You can download the jar here :
>>>> http://search.maven.org/remotecontent?filepath=com/google/guava/guava/13.0.1/guava-13.0.1.jar
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Tue, Nov 20, 2012 at 8:06 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Could please let me know the name of jar file and location
>>>>>
>>>>> Many thanks
>>>>>
>>>>> Best regards
>>>>>
>>>>>
>>>>> On Tue, Nov 20, 2012 at 2:33 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Download the required jar and include it in your project.
>>>>>>
>>>>>> Regards,
>>>>>>     Mohammad Tariq
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 20, 2012 at 7:57 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>> Dear Tariq Thanks
>>>>>>>
>>>>>>> I have added the jar files from Cdh and download the cdh4 eclipse
>>>>>>> plugin and copied into eclipse plugin folder. The previous error I think
>>>>>>> sorted out but now I am getting another strange error.
>>>>>>>
>>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>>>> com/google/common/collect/Maps
>>>>>>>     at
>>>>>>> org.apache.hadoop.metrics2.lib.MetricsRegistry.<init>(MetricsRegistry.java:42)
>>>>>>>     at
>>>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:87)
>>>>>>>     at
>>>>>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:133)
>>>>>>>     at
>>>>
+
shekhar sharma 2012-11-26, 16:42
+
kashif khan 2012-11-27, 21:25
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB