Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - writing to hdfs via java api
Jay Vyas
2011-10-28, 05:52
Harsh J
2011-10-28, 06:03
Arpit Gupta
2011-10-28, 06:05
Tom Melendez
2011-10-29, 00:24
Jay Vyas
2011-10-29, 02:57
Tom Melendez
2011-10-29, 03:41
Alex Gauthier
2011-10-29, 03:43
Alex Gauthier
2011-10-29, 03:43
JAX
2011-10-29, 04:16
Alex Gauthier
2011-10-29, 04:17
JAX
2011-10-29, 04:19
Copy link to this message
-
Re: writing to hdfs via java api
Jay Vyas 2011-10-28, 23:04
Hi guys : Made more progress debugging my hadoop connection, but still
haven't got it working......  It looks like my VM (cloudera hadoop) won't
let me in.  I find that there is no issue connecting to the name node - that
is , using hftp and 50070......

via standard HFTP as in here :

//This method works fine - connecting directly to hadoop's namenode and
querying the filesystem
public static void main1(String[] args) throws Exception
    {
        String uri = "hftp://155.37.101.76:50070/";

        System.out.println( "uri: " + uri );
        Configuration conf = new Configuration();

        FileSystem fs = FileSystem.get( URI.create( uri ), conf );
        fs.printStatistics();
    }
But unfortunately, I can't get into hdfs ..... Any thoughts on this ?  I am
modifying the uri to access port 8020
which is what is in my core-site.xml .

   // This fails, resulting (trys to connect over and over again, eventually
gives up printing "already tried to connect 20 times"....)
    public static void main(String[] args)
    {
        try {
            String uri = "hdfs://155.37.101.76:8020/";

            System.out.println( "uri: " + uri );
            Configuration conf = new Configuration();

            FileSystem fs = FileSystem.get( URI.create( uri ), conf );
            fs.printStatistics();
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

The error message is :

11/10/28 19:03:38 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 0 time(s).
11/10/28 19:03:39 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 1 time(s).
11/10/28 19:03:40 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 2 time(s).
11/10/28 19:03:41 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 3 time(s).

Any thoughts on this would be *really* be appreciated  ... Thanks guys.