Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase Is So Slow To Save Data?


Copy link to this message
-
Re: HBase Is So Slow To Save Data?
Dear all,

By the way, my HBase is in the pseudo-distributed mode. Thanks!

Best regards,
Bing

On Wed, Aug 29, 2012 at 10:04 PM, Bing Li <[EMAIL PROTECTED]> wrote:

> Dear all,
>
> According to my experiences, it is very slow for HBase to save data? Am I
> right?
>
> For example, today I need to save data in a HashMap to HBase. It took
> about more than three hours. However when saving the same HashMap in a file
> in the text format with the redirected System.out, it took only 4.5 seconds!
>
> Why is HBase so slow? It is indexing?
>
> My code to save data in HBase is as follows. I think the code must be
> correct.
>
>         ......
>         public synchronized void
> AddVirtualOutgoingHHNeighbors(ConcurrentHashMap<String,
> ConcurrentHashMap<String, Set<String>>> hhOutNeighborMap, int timingScale)
>         {
>                 List<Put> puts = new ArrayList<Put>();
>
>                 String hhNeighborRowKey;
>                 Put hubKeyPut;
>                 Put groupKeyPut;
>                 Put topGroupKeyPut;
>                 Put timingScalePut;
>                 Put nodeKeyPut;
>                 Put hubNeighborTypePut;
>
>                 for (Map.Entry<String, ConcurrentHashMap<String,
> Set<String>>> sourceHubGroupNeighborEntry : hhOutNeighborMap.entrySet())
>                 {
>                         for (Map.Entry<String, Set<String>>
> groupNeighborEntry : sourceHubGroupNeighborEntry.getValue().entrySet())
>                         {
>                                 for (String neighborKey :
> groupNeighborEntry.getValue())
>                                 {
>                                         hhNeighborRowKey > NeighborStructure.HUB_HUB_NEIGHBOR_ROW +
> Tools.GetAHash(sourceHubGroupNeighborEntry.getKey() +
> groupNeighborEntry.getKey() + timingScale + neighborKey);
>
>                                         hubKeyPut = new
> Put(Bytes.toBytes(hhNeighborRowKey));
>
> hubKeyPut.add(Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY),
> Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_HUB_KEY_COLUMN),
> Bytes.toBytes(sourceHubGroupNeighborEntry.getKey()));
>                                         puts.add(hubKeyPut);
>
>                                         groupKeyPut = new
> Put(Bytes.toBytes(hhNeighborRowKey));
>
> groupKeyPut.add(Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY),
> Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_GROUP_KEY_COLUMN),
> Bytes.toBytes(groupNeighborEntry.getKey()));
>                                         puts.add(groupKeyPut);
>
>                                         topGroupKeyPut = new
> Put(Bytes.toBytes(hhNeighborRowKey));
>
> topGroupKeyPut.add(Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY),
> Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_TOP_GROUP_KEY_COLUMN),
> Bytes.toBytes(GroupRegistry.WWW().GetParentGroupKey(groupNeighborEntry.getKey())));
>                                         puts.add(topGroupKeyPut);
>
>                                         timingScalePut = new
> Put(Bytes.toBytes(hhNeighborRowKey));
>
> timingScalePut.add(Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY),
> Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_TIMING_SCALE_COLUMN),
> Bytes.toBytes(timingScale));
>                                         puts.add(timingScalePut);
>
>                                         nodeKeyPut = new
> Put(Bytes.toBytes(hhNeighborRowKey));
>
> nodeKeyPut.add(Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY),
> Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_NODE_KEY_COLUMN),
> Bytes.toBytes(neighborKey));
>                                         puts.add(nodeKeyPut);
>
>                                         hubNeighborTypePut = new
> Put(Bytes.toBytes(hhNeighborRowKey));
>
> hubNeighborTypePut.add(Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_FAMILY),
> Bytes.toBytes(NeighborStructure.HUB_HUB_NEIGHBOR_TYPE_COLUMN),
> Bytes.toBytes(SocialRole.VIRTUAL_NEIGHBOR));
>                                         puts.add(hubNeighborTypePut);
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB