Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> efficient export w/o HDFS/copying

Ted Tuttle 2012-03-26, 23:31
Stack 2012-03-27, 04:07
Copy link to this message
Re: efficient export w/o HDFS/copying
Wouldn't that mean having the NAS attached to all of the nodes in the cluster?
Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 26, 2012, at 11:07 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Mon, Mar 26, 2012 at 4:31 PM, Ted Tuttle <[EMAIL PROTECTED]> wrote:
>> Is there a method of exporting that skips the HDFS step?  We would
>> ideally like to export from HBase directly to an external filesystem
>> (e.g. our big slow NAS) skipping the HDFS step.
> Do an OutputFormat that just writes files to your NAS and hook it up
> to the export tool in place of SequenceFileOutputFormat.  Set your new
> NASOutputFormat instead of SequenceFileOutputFormat here:
> http://hbase.apache.org/xref/org/apache/hadoop/hbase/mapreduce/Export.html#99
> (You'll probably have to override Exporter to do your customization
> copying bulk of createSubmittableJob into subclass)
> St.Ack
Stack 2012-03-28, 17:01