Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Hbase Bulk Load Java Sample Code


Copy link to this message
-
Re: Hbase Bulk Load Java Sample Code
Doug Meil 2012-08-27, 18:01

Hi there, in addition there is a fair amount of documentation about bulk
loads and importtsv in the Hbase RefGuide.

http://hbase.apache.org/book.html#importtsv

On 8/27/12 9:34 AM, "Ioakim Perros" <[EMAIL PROTECTED]> wrote:

>On 08/27/2012 04:18 PM, o brbrs wrote:
>> Hi,
>>
>> I'm new at hase and i want to make bulk load from hdfs to hbase with
>>java.
>> Is there any sample code which includes importtsv and completebulkload
>> libraries on java?
>>
>> Thanks.
>>
>Hi,
>
>Here is a sample configuration of a bulk loading job consisting only of
>map tasks:
>
>         Configuration config = HBaseConfiguration.create();
>
>         config.set(TableOutputFormat.OUTPUT_TABLE, tableNAME);
>         Path inputPath = new Path(inputStringPath);
>
>         Job job = new Job(config, "Sample job" );
>
>         job.setMapOutputKeyClass(mapperKey);
>         job.setMapOutputValueClass(mapperValue);
>
>         FileInputFormat.setInputPaths(job, inputPath);
>         job.setInputFormatClass(inputFormat);
>         FileOutputFormat.setOutputPath(job,new Path (HFileoutputPath));
>//directory at HDFS where HFiles will be placed
>         //before bulk loading
>
>         job.setOutputFormatClass(HFileOutputFormat.class);
>
>         job.setJarByClass(caller);
>         job.setMapperClass(mapper);
>
>         HTable hTable = new HTable(config, tableNAME); //tableNAME is a
>String representing a table which has to
>         //already exist in HBase
>         HFileOutputFormat.configureIncrementalLoad(job, hTable);
>//check respective API for the complete functionality of this
>//function
>
>         job.waitForCompletion(true);
>
>         /* after the job's completion, we have to write the HFiles
>          * into HBase's specified table */
>         LoadIncrementalHFiles lihf = new LoadIncrementalHFiles(config);
>         lihf.doBulkLoad(new Path(HFileoutputPath), hTable);
>
>Create a map task which produces key,value pairs just as you expect them
>to exist in your HBase's table (e.g.: key: ImmutableBytesWritable, Put)
>and you re done.
>
>Regards,
>Ioakim