Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - HTable.put hangs on bulk loading


+
ajay.gov 2011-04-07, 02:10
+
Jean-Daniel Cryans 2011-04-07, 17:29
+
ajay.gov 2011-04-07, 19:20
+
Stan Barton 2011-04-26, 10:25
+
ajay.gov 2011-04-26, 17:17
+
Jeff Whiting 2011-04-26, 18:36
+
Stack 2011-04-26, 18:41
+
Stan Barton 2011-04-27, 09:30
+
Stack 2011-04-27, 16:40
+
Stan Barton 2011-04-28, 13:54
+
Stack 2011-05-07, 21:51
+
Stan Barton 2011-05-13, 14:44
+
Ted Yu 2011-05-13, 15:04
+
Stack 2011-05-13, 19:34
+
Stan Barton 2011-05-16, 11:55
+
Stack 2011-05-16, 18:30
+
Stan Barton 2011-05-18, 12:59
Copy link to this message
-
Re: HTable.put hangs on bulk loading
Michel Segel 2011-04-26, 15:43
How many regions on a region server?
Table splits? GC tuning parameters?
HBase heap size?

All of these qustions could be important...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 26, 2011, at 5:25 AM, Stan Barton <[EMAIL PROTECTED]> wrote:

>
> I am running into similar problem with HBase 0.90.2. My setting is 6 RSs one
> master server, 3 ZK servers and about 20 clients (on different servers)
> feeding the database with inserts. After few hours and around 2.5 millions
> rows inserted the process simply hangs with no what so ever error indication
> (neither by ZK, master, regionservers nor the clients). All the inserting
> clients stop virtually at the same moment, but HBase it self is not down and
> can be queried.
>
> In fact, the clients not even crash on time out exception for its
> connection. I have ran into such problem always when I attempted to run such
> importing progress with the new HBase versions (even 0.90.1). Can anybody
> address this problem? Does other have similar problems?
>
> I can provide further info about the configuration if needed.
>
> Stan Barton
>
>
>
>
> ajay.gov wrote:
>>
>> Sorry, my server config was not attached. Its here:
>> http://pastebin.com/U41QZGiq
>>
>> thanks
>> -ajay
>>
>>
>>
>> ajay.gov wrote:
>>>
>>> I am doing a load test for which I need to load a table with many rows.
>>> I have a small java program that has a for loop and calls HTable.put.  I
>>> am inserting a map of 2 items into a table that has one column family.
>>> The limit of the for loop is currently 20000. However after 15876 rows
>>> the call to Put hangs. I am using autoFlush on the HTable. Any ideas why
>>> this may happen?
>>>
>>> The table configuration:
>>> DESCRIPTION                                          ENABLED                    
>>> {NAME => 'TABLE2', FAMILIES => [{NAME => 'TABLE2_CF true                      
>>> 1', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0'                            
>>> , COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2                            
>>> 147483647', BLOCKSIZE => '65536', IN_MEMORY => 'fal                            
>>> se', BLOCKCACHE => 'true'}]}
>>>
>>> The HBase config on the client is the one in the hbase-default.xml. Some
>>> values:
>>> hbase.client.write.buffer=2097152
>>> hbase.client.pause=1000
>>> hbase.client.retries.number=10
>>>
>>> If i use another client I am able to put items to the table. I am also
>>> able to scan items from the table using the hbase shell.
>>>
>>> I have attached the server configuratio
>>> I don't see anything in the region server or master logs. I have them
>>> here.
>>>
>>> The master server log:
>>> 2011-04-06 19:02:40,149 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.rootScanner scanning meta region {server:
>>> 184.106.69.238:60020, regionname: -ROOT-,,0.70236052, startKey: <>}
>>> 2011-04-06 19:02:40,152 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>>> 184.106.69.238:60020, regionname: -ROOT-,,0.70236052, startKey: <>}
>>> complete
>>> 2011-04-06 19:02:40,157 INFO
>>> org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 0 dead,
>>> average load 42.0
>>> 2011-04-06 19:03:15,252 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.metaScanner scanning meta region {server:
>>> 184.106.69.238:60020, regionname: .META.,,1.1028785192, startKey: <>}
>>> 2011-04-06 19:03:15,265 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.metaScanner scan of 40 row(s) of meta region {server:
>>> 184.106.69.238:60020, regionname: .META.,,1.1028785192, startKey: <>}
>>> complete
>>> 2011-04-06 19:03:15,266 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> All 1 .META. region(s) scanned
>>>
>>>
>>> The region server logs:
>>> 2011-04-06 19:02:21,294 DEBUG
>>> org.apache.hadoop.hbase.regionserver.HRegion: Creating region
>>> TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.
+
Ajay Govindarajan 2011-04-08, 00:35
+
Ajay Govindarajan 2011-04-08, 19:21
+
Jean-Daniel Cryans 2011-04-08, 21:20
+
Ajay Govindarajan 2011-04-08, 23:27