Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> efficient export w/o HDFS/copying


+
Ted Tuttle 2012-03-26, 23:31
+
Stack 2012-03-27, 04:07
Copy link to this message
-
Re: efficient export w/o HDFS/copying
Wouldn't that mean having the NAS attached to all of the nodes in the cluster?
Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 26, 2012, at 11:07 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Mon, Mar 26, 2012 at 4:31 PM, Ted Tuttle <[EMAIL PROTECTED]> wrote:
>> Is there a method of exporting that skips the HDFS step?  We would
>> ideally like to export from HBase directly to an external filesystem
>> (e.g. our big slow NAS) skipping the HDFS step.
>>
>
> Do an OutputFormat that just writes files to your NAS and hook it up
> to the export tool in place of SequenceFileOutputFormat.  Set your new
> NASOutputFormat instead of SequenceFileOutputFormat here:
> http://hbase.apache.org/xref/org/apache/hadoop/hbase/mapreduce/Export.html#99
> (You'll probably have to override Exporter to do your customization
> copying bulk of createSubmittableJob into subclass)
>
> St.Ack
>
+
Stack 2012-03-28, 17:01
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB