Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> writing to hdfs via java api


Copy link to this message
-
Re: writing to hdfs via java api
Hi guys : Made more progress debugging my hadoop connection, but still
haven't got it working......  It looks like my VM (cloudera hadoop) won't
let me in.  I find that there is no issue connecting to the name node - that
is , using hftp and 50070......

via standard HFTP as in here :

//This method works fine - connecting directly to hadoop's namenode and
querying the filesystem
public static void main1(String[] args) throws Exception
    {
        String uri = "hftp://155.37.101.76:50070/";

        System.out.println( "uri: " + uri );
        Configuration conf = new Configuration();

        FileSystem fs = FileSystem.get( URI.create( uri ), conf );
        fs.printStatistics();
    }
But unfortunately, I can't get into hdfs ..... Any thoughts on this ?  I am
modifying the uri to access port 8020
which is what is in my core-site.xml .

   // This fails, resulting (trys to connect over and over again, eventually
gives up printing "already tried to connect 20 times"....)
    public static void main(String[] args)
    {
        try {
            String uri = "hdfs://155.37.101.76:8020/";

            System.out.println( "uri: " + uri );
            Configuration conf = new Configuration();

            FileSystem fs = FileSystem.get( URI.create( uri ), conf );
            fs.printStatistics();
        } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

The error message is :

11/10/28 19:03:38 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 0 time(s).
11/10/28 19:03:39 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 1 time(s).
11/10/28 19:03:40 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 2 time(s).
11/10/28 19:03:41 INFO ipc.Client: Retrying connect to server: /
155.37.101.76:8020. Already tried 3 time(s).

Any thoughts on this would be *really* be appreciated  ... Thanks guys.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB