Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: How to tell my Hadoop cluster to read data from an external server


Copy link to this message
-
RE: How to tell my Hadoop cluster to read data from an external server
can you addInputPath(hdfs://……),dont change fs.default.name, It cannot
solve your problem.
On Mar 26, 2013 7:03 PM, "Agarwal, Nikhil" <[EMAIL PROTECTED]>
wrote:

>  Hi,****
>
> Thanks for your reply. I do not know about cascading. Should I google it
> as “cascading in hadoop”? Also, what I was thinking is to implement a file
> system which overrides the functions provided by fs.FileSystem interface in
> Hadoop. I tried to write some portions of the filesystem (for my external
> server) so that it recompiles successfully but when I submit a MR job I get
> the following error:****
>
> ** **
>
> 13/03/26 06:09:10 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 0 time(s).
> 13/03/26 06:09:11 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 1 time(s).
> 13/03/26 06:09:12 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 2 time(s).
> 13/03/26 06:09:13 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 3 time(s).
> 13/03/26 06:09:14 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 4 time(s).
> 13/03/26 06:09:15 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 5 time(s).
> 13/03/26 06:09:16 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 6 time(s).
> 13/03/26 06:09:17 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 7 time(s).
> 13/03/26 06:09:18 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 8 time(s).
> 13/03/26 06:09:19 INFO ipc.Client: Retrying connect to server: localhost/
> 127.0.0.1:54312. Already tried 9 time(s).
> 13/03/26 06:10:20 ERROR security.UserGroupInformation:
> PriviledgedActionException as:nikhil cause:java.net.ConnectException: Call
> to localhost/127.0.0.1:54312 failed on connection exception:
> java.net.ConnectException: Connection refused
> java.net.ConnectException: Call to localhost/127.0.0.1:54312 failed on
> connection exception: java.net.ConnectException: Connection refused
>     at org.apache.hadoop.ipc.Client.wrapException(Client.java:1099)
>     at org.apache.hadoop.ipc.Client.call(Client.java:1075)
>     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
>     at org.apache.hadoop.mapred.$Proxy2.getProtocolVersion(Unknown Source)
>     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
>     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
>     at
> org.apache.hadoop.mapred.JobClient.createRPCProxy(JobClient.java:480)
>     at org.apache.hadoop.mapred.JobClient.init(JobClient.java:474)
>     at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:457)
>     at org.apache.hadoop.mapreduce.Job$1.run(Job.java:513)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:416)
>     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>     at org.apache.hadoop.mapreduce.Job.connect(Job.java:511)
>     at org.apache.hadoop.mapreduce.Job.submit(Job.java:499)
>     at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
>     at org.apache.hadoop.examples.WordCount.main(WordCount.java:67)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>     at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:616)
>     at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
>     at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
>     at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)