Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - efficient export w/o HDFS/copying


Copy link to this message
-
Re: efficient export w/o HDFS/copying
Michel Segel 2012-03-28, 07:59
Wouldn't that mean having the NAS attached to all of the nodes in the cluster?
Sent from a remote device. Please excuse any typos...

Mike Segel

On Mar 26, 2012, at 11:07 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Mon, Mar 26, 2012 at 4:31 PM, Ted Tuttle <[EMAIL PROTECTED]> wrote:
>> Is there a method of exporting that skips the HDFS step?  We would
>> ideally like to export from HBase directly to an external filesystem
>> (e.g. our big slow NAS) skipping the HDFS step.
>>
>
> Do an OutputFormat that just writes files to your NAS and hook it up
> to the export tool in place of SequenceFileOutputFormat.  Set your new
> NASOutputFormat instead of SequenceFileOutputFormat here:
> http://hbase.apache.org/xref/org/apache/hadoop/hbase/mapreduce/Export.html#99
> (You'll probably have to override Exporter to do your customization
> copying bulk of createSubmittableJob into subclass)
>
> St.Ack
>