Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Automatically upload files into HDFS


Copy link to this message
-
Re: Automatically upload files into HDFS
kashif khan 2012-11-20, 14:36
Could please let me know the name of jar file and location

Many thanks

Best regards

On Tue, Nov 20, 2012 at 2:33 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Download the required jar and include it in your project.
>
> Regards,
>     Mohammad Tariq
>
>
>
> On Tue, Nov 20, 2012 at 7:57 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>
>> Dear Tariq Thanks
>>
>> I have added the jar files from Cdh and download the cdh4 eclipse plugin
>> and copied into eclipse plugin folder. The previous error I think sorted
>> out but now I am getting another strange error.
>>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> com/google/common/collect/Maps
>>     at
>> org.apache.hadoop.metrics2.lib.MetricsRegistry.<init>(MetricsRegistry.java:42)
>>     at
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:87)
>>     at
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:133)
>>     at
>> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38)
>>     at
>> org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36)
>>     at
>> org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:97)
>>     at
>> org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:190)
>>     at
>> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2373)
>>     at
>> org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2365)
>>     at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2233)
>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
>>     at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
>>     at CopyFile.main(CopyFile.java:14)
>> Caused by: java.lang.ClassNotFoundException:
>> com.google.common.collect.Maps
>>     at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
>>     at java.security.AccessController.doPrivileged(Native Method)
>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
>>     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
>>     ... 13 more
>>
>> Have any idea about this error.
>>
>> Many thanks
>>
>>
>>
>>
>>
>> On Tue, Nov 20, 2012 at 2:19 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>>
>>> Hello Kashif,
>>>
>>>      You are correct. This because of some version mismatch. I am not
>>> using CDH personally but AFAIK, CDH4 uses Hadoop-2.x.
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>>
>>> On Tue, Nov 20, 2012 at 4:10 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>>>
>>>> HI M Tariq
>>>>
>>>>
>>>> I am trying the following the program to create directory and copy file
>>>> to hdfs. But I am getting the following errors
>>>>
>>>>
>>>>
>>>> Program:
>>>>
>>>> import org.apache.hadoop.conf.Configuration;
>>>> import org.apache.hadoop.fs.FileSystem;
>>>> import org.apache.hadoop.fs.Path;
>>>> import java.io.IOException;
>>>>
>>>> public class CopyFile {
>>>>
>>>>
>>>>         public static void main(String[] args) throws IOException{
>>>>         Configuration conf = new Configuration();
>>>>          conf.set("fs.default.name", "hadoop1.example.com:8020");
>>>>         FileSystem dfs = FileSystem.get(conf);
>>>>         String dirName = "Test1";
>>>>         Path src = new Path(dfs.getWorkingDirectory() + "/" + dirName);
>>>>         dfs.mkdirs(src);
>>>>         Path scr1 = new Path("/usr/Eclipse/Output.csv");
>>>>         Path dst = new Path(dfs.getWorkingDirectory() + "/Test1/");
>>>>         dfs.copyFromLocalFile(src, dst);
>>>>
>>>>         }
>>>>         }
>>>>
>>>>
>>>>     Exception in thread "main" org.apache.hadoop.ipc.RemoteException:
>>>> Server IPC version 7 cannot communicate with client version 4
>>>>     at org.apache.hadoop.ipc.Client.call(Client.java:1070)
>>>>     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)