-Re: Database insertion by HAdoop
Hemanth Yamijala 2013-02-18, 14:58
What database is this ? Was hbase mentioned ?
On Monday, February 18, 2013, Mohammad Tariq wrote:
> Hello Masoud,
> You can use the Bulk Load feature. You might find it more
> efficient than normal client APIs or using the TableOutputFormat.
> The bulk load feature uses a MapReduce job to output table data
> in HBase's internal data format, and then directly loads the
> generated StoreFiles into a running cluster. Using bulk load will use
> less CPU and network resources than simply using the HBase API.
> For a detailed info you can go here :
> Warm Regards,
> > wrote:
>> Dear All,
>> We are going to do our experiment of a scientific papers, ]
>> We must insert data in our database for later consideration, it almost
>> 300 tables each one has 2/000/000 records.
>> as you know It takes lots of time to do it with a single machine,
>> we are going to use our Hadoop cluster (32 machines) and divide 300
>> insertion tasks between them,
>> I need some hint to progress faster,
>> 1- as i know we dont need to Reduser, just Mapper in enough.
>> 2- so wee need just implement Mapper class with needed code.
>> Please let me know if there is any point,
>> Best Regards