Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Automatically upload files into HDFS


Copy link to this message
-
Re: Automatically upload files into HDFS
Dear shekhar,

I am still struggling. I have written some java code, it does work but not
100%  as some time sending file with 0KB data. If you have any solution,
please let me know. I shall be very grateful to you.

Many thanks

Best regards

On Mon, Nov 26, 2012 at 4:42 PM, shekhar sharma <[EMAIL PROTECTED]>wrote:

> Hello Khasif,
> Sorry for late reply...Are you done? or u still struggling?
>
> mail me: [EMAIL PROTECTED]
> Regards,
> Som Shekhar Sharma
>
>
>
> On Wed, Nov 21, 2012 at 6:06 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>
>> Dear Shankar Sharma.
>>
>> I am using Eclipse as IDE. I dont have any idea, how to create the
>> project as maven project. I have downloaded Mave2 but given me some strange
>> error. So if you can help me then I will try the maven. Actually, I am
>> trying to automatically upload the files into HDFS and then will apply some
>> algorithms to analyze the data. The algorithms will implement in mapreduce
>> . So if you think maven will good for me then please let me know how I can
>> create the project as maven project.
>>
>>
>> Many thanks
>>
>> Best regards,
>>
>> KK
>>
>>
>>
>>  On Tue, Nov 20, 2012 at 7:06 PM, shekhar sharma <[EMAIL PROTECTED]>wrote:
>>
>>> By the way how are you building and running your project.Are u running
>>> from any IDE?
>>> The best practises you can follow:
>>>
>>> (1) Create your project as maven project and give the dependency of
>>> hadoop-X.Y.Z . So your project will automatically will have all the
>>> necessary jars
>>> and i am sure you will not face these kind of errors
>>> (2) And In your HADOOP_CLASSPATH provide the path for $HADOOP_LIB.
>>>
>>> Regards,
>>> SOm
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Nov 20, 2012 at 9:52 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>
>>>> Dear Tariq
>>>>
>>>> Many thanks, finally I have created the directory and upload the file.
>>>>
>>>> Once again many thanks
>>>>
>>>> Best regards
>>>>
>>>>
>>>> On Tue, Nov 20, 2012 at 3:04 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Dear Many thanks
>>>>>
>>>>>
>>>>> I have downloaded the jar file and added to project. Now getting
>>>>> another error as:
>>>>>
>>>>> og4j:WARN No appenders could be found for logger
>>>>> (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
>>>>> log4j:WARN Please initialize the log4j system properly.
>>>>> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfigfor more info.
>>>>> Exception in thread "main" java.io.IOException: No FileSystem for
>>>>> scheme: hdfs
>>>>>     at
>>>>> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2206)
>>>>>     at
>>>>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2213)
>>>>>     at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:80)
>>>>>     at
>>>>> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2252)
>>>>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2234)
>>>>>
>>>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>>>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
>>>>>     at CopyFile.main(CopyFile.java:14)
>>>>>
>>>>> Have any idea about this?
>>>>>
>>>>> Thanks again
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Nov 20, 2012 at 2:53 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> You can download the jar here :
>>>>>> http://search.maven.org/remotecontent?filepath=com/google/guava/guava/13.0.1/guava-13.0.1.jar
>>>>>>
>>>>>> Regards,
>>>>>>     Mohammad Tariq
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 20, 2012 at 8:06 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>>>>
>>>>>>> Could please let me know the name of jar file and location
>>>>>>>
>>>>>>> Many thanks
>>>>>>>
>>>>>>> Best regards
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 20, 2012 at 2:33 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>>>>>>
>>>>>>>> Download the required jar and include it in your project.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>     Mohammad Tariq
>>>>>>>>
>>>>>>>>
>>>>>>>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB