Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Automatically upload files into HDFS


Copy link to this message
-
Re: Automatically upload files into HDFS
Dear Tariq Thanks

I have added the jar files from Cdh and download the cdh4 eclipse plugin
and copied into eclipse plugin folder. The previous error I think sorted
out but now I am getting another strange error.

Exception in thread "main" java.lang.NoClassDefFoundError:
com/google/common/collect/Maps
    at
org.apache.hadoop.metrics2.lib.MetricsRegistry.<init>(MetricsRegistry.java:42)
    at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:87)
    at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.<init>(MetricsSystemImpl.java:133)
    at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<init>(DefaultMetricsSystem.java:38)
    at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.<clinit>(DefaultMetricsSystem.java:36)
    at
org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:97)
    at
org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:190)
    at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2373)
    at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:2365)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2233)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:300)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:156)
    at CopyFile.main(CopyFile.java:14)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Maps
    at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
    ... 13 more

Have any idea about this error.

Many thanks
On Tue, Nov 20, 2012 at 2:19 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello Kashif,
>
>      You are correct. This because of some version mismatch. I am not
> using CDH personally but AFAIK, CDH4 uses Hadoop-2.x.
>
> Regards,
>     Mohammad Tariq
>
>
>
> On Tue, Nov 20, 2012 at 4:10 PM, kashif khan <[EMAIL PROTECTED]>wrote:
>
>> HI M Tariq
>>
>>
>> I am trying the following the program to create directory and copy file
>> to hdfs. But I am getting the following errors
>>
>>
>>
>> Program:
>>
>> import org.apache.hadoop.conf.Configuration;
>> import org.apache.hadoop.fs.FileSystem;
>> import org.apache.hadoop.fs.Path;
>> import java.io.IOException;
>>
>> public class CopyFile {
>>
>>
>>         public static void main(String[] args) throws IOException{
>>         Configuration conf = new Configuration();
>>          conf.set("fs.default.name", "hadoop1.example.com:8020");
>>         FileSystem dfs = FileSystem.get(conf);
>>         String dirName = "Test1";
>>         Path src = new Path(dfs.getWorkingDirectory() + "/" + dirName);
>>         dfs.mkdirs(src);
>>         Path scr1 = new Path("/usr/Eclipse/Output.csv");
>>         Path dst = new Path(dfs.getWorkingDirectory() + "/Test1/");
>>         dfs.copyFromLocalFile(src, dst);
>>
>>         }
>>         }
>>
>>
>>     Exception in thread "main" org.apache.hadoop.ipc.RemoteException:
>> Server IPC version 7 cannot communicate with client version 4
>>     at org.apache.hadoop.ipc.Client.call(Client.java:1070)
>>     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>>     at $Proxy1.getProtocolVersion(Unknown Source)
>>     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
>>     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
>>     at
>> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
>>     at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
>>     at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
>>     at
>> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
>>     at
>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)