Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> write seqfiles to hdfs from a java app without "hadoop jar" command


Copy link to this message
-
write seqfiles to hdfs from a java app without "hadoop jar" command
Hey there, I need to write seqfiles to an hdfs cluster using a normal java
app but can't use "hadoop jar".
I have access to the installation of a hadoop node.
I've reached the goal launching my app with java -jar and manually changing
de configuration in the code:

Configuration hadoopConf = new Configuration();
hadoopConf.addResource(new Path("/opt/hadoop/install/conf/core-site.xml"));
hadoopConf.addResource(new Path("/opt/hadoop/install/conf/hdfs-site.xml"));
hadoopConf.reloadConfiguration();
sfw = SequenceFile.createWriter(FileSystem.get(hadoopConf), hadoopConf,
...);

** /opt/hadoop/install/conf/core-site.xml and
/opt/hadoop/install/conf/hdfs-site.xml are the conf files of the cluster.

This works but I would like to know if there's a more elegant way to do
that. Specifying hadoop env vars to the java classpath or something like
that.

Thanks in advance.

--
View this message in context: http://lucene.472066.n3.nabble.com/write-seqfiles-to-hdfs-from-a-java-app-without-hadoop-jar-command-tp2677234p2677234.html
Sent from the Hadoop lucene-users mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB