Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Automatically upload files into HDFS


Copy link to this message
-
Re: Automatically upload files into HDFS
Hello Khasif,
Sorry for late reply...Are you done? or u still struggling?

mail me: [EMAIL PROTECTED]
Regards,
Som Shekhar Sharma

On Wed, Nov 21, 2012 at 6:06 PM, kashif khan <[EMAIL PROTECTED]> wrote:

> Dear Shankar Sharma.
>
> I am using Eclipse as IDE. I dont have any idea, how to create the project
> as maven project. I have downloaded Mave2 but given me some strange error.
> So if you can help me then I will try the maven. Actually, I am trying to
> automatically upload the files into HDFS and then will apply some
> algorithms to analyze the data. The algorithms will implement in mapreduce
> . So if you think maven will good for me then please let me know how I can
> create the project as maven project.
>
>
> Many thanks
>
> Best regards,
>
> KK
>
>
>
>  On Tue, Nov 20, 2012 at 7:06 PM, shekhar sharma <[EMAIL PROTECTED]>wrote:
>
>> By the way how are you building and running your project.Are u running
>> from any IDE?
>> The best practises you can follow:
>>
>> (1) Create your project as maven project and give the dependency of
>> hadoop-X.Y.Z . So your project will automatically will have all the
>> necessary jars
>> and i am sure you will not face these kind of errors
>> (2) And In your HADOOP_CLASSPATH provide the path for $HADOOP_LIB.
>>
>> Regards,
>> SOm
>>
>>
>>
>>
>>
>> On Tue, Nov 20, 2012 at 9:52 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>
>>> Dear Tariq
>>>
>>> Many thanks, finally I have created the directory and upload the file.
>>>
>>> Once again many thanks
>>>
>>> Best regards
>>>
>>>
>>> On Tue, Nov 20, 2012 at 3:04 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>
>>>> Dear Many thanks
>>>>
>>>>
>>>> I have downloaded the jar file and added to project. Now getting
>>>> another error as:
>>>>
>>>> og4j:WARN No appenders could be found for logger
>>>> (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
>>>> log4j:WARN Please initialize the log4j system properly.
>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor more info.
>>>> Exception in thread "main" java.io.IOException: No FileSystem for
>>>> scheme: hdfs
>>>>     at
>>>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>>>>     at
>>>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>>>>     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
>>>>     at
>>>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
>>>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
>>>>
>>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
>>>>     at CopyFile.main(CopyFile.java:14)
>>>>
>>>> Have any idea about this?
>>>>
>>>> Thanks again
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Nov 20, 2012 at 2:53 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> You can download the jar here :
>>>>> http://search.maven.org/remotecontent?filepath=com/google/guava/guava/13.0.1/guava-13.0.1.jar
>>>>>
>>>>> Regards,
>>>>>     Mohammad Tariq
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 20, 2012 at 8:06 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Could please let me know the name of jar file and location
>>>>>>
>>>>>> Many thanks
>>>>>>
>>>>>> Best regards
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 20, 2012 at 2:33 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>> Download the required jar and include it in your project.
>>>>>>>
>>>>>>> Regards,
>>>>>>>     Mohammad Tariq
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 20, 2012 at 7:57 PM, kashif khan <[EMAIL PROTECTED]
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Dear Tariq Thanks
>>>>>>>>
>>>>>>>> I have added the jar files from Cdh and download the cdh4 eclipse
>>>>>>>> plugin and copied into eclipse plugin folder. The previous error I think
>>>>>>>> sorted out but now I am getting another strange error.
>>>>>>>>
>>>>>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>>>>> com/google/common/collect/Maps
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB