Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Hbase Bulk Load Java Sample Code


Copy link to this message
-
Re: Hbase Bulk Load Java Sample Code

Hi there, in addition there is a fair amount of documentation about bulk
loads and importtsv in the Hbase RefGuide.

http://hbase.apache.org/book.html#importtsv

On 8/27/12 9:34 AM, "Ioakim Perros" <[EMAIL PROTECTED]> wrote:

>On 08/27/2012 04:18 PM, o brbrs wrote:
>> Hi,
>>
>> I'm new at hase and i want to make bulk load from hdfs to hbase with
>>java.
>> Is there any sample code which includes importtsv and completebulkload
>> libraries on java?
>>
>> Thanks.
>>
>Hi,
>
>Here is a sample configuration of a bulk loading job consisting only of
>map tasks:
>
>         Configuration config = HBaseConfiguration.create();
>
>         config.set(TableOutputFormat.OUTPUT_TABLE, tableNAME);
>         Path inputPath = new Path(inputStringPath);
>
>         Job job = new Job(config, "Sample job" );
>
>         job.setMapOutputKeyClass(mapperKey);
>         job.setMapOutputValueClass(mapperValue);
>
>         FileInputFormat.setInputPaths(job, inputPath);
>         job.setInputFormatClass(inputFormat);
>         FileOutputFormat.setOutputPath(job,new Path (HFileoutputPath));
>//directory at HDFS where HFiles will be placed
>         //before bulk loading
>
>         job.setOutputFormatClass(HFileOutputFormat.class);
>
>         job.setJarByClass(caller);
>         job.setMapperClass(mapper);
>
>         HTable hTable = new HTable(config, tableNAME); //tableNAME is a
>String representing a table which has to
>         //already exist in HBase
>         HFileOutputFormat.configureIncrementalLoad(job, hTable);
>//check respective API for the complete functionality of this
>//function
>
>         job.waitForCompletion(true);
>
>         /* after the job's completion, we have to write the HFiles
>          * into HBase's specified table */
>         LoadIncrementalHFiles lihf = new LoadIncrementalHFiles(config);
>         lihf.doBulkLoad(new Path(HFileoutputPath), hTable);
>
>Create a map task which produces key,value pairs just as you expect them
>to exist in your HBase's table (e.g.: key: ImmutableBytesWritable, Put)
>and you re done.
>
>Regards,
>Ioakim
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB