Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Performance Are Affected? - Table and Family


Copy link to this message
-
Performance Are Affected? - Table and Family
Bing Li 2013-08-12, 17:00
Dear all,

I have one additional question about table and family.

A table which has less families is faster than the one which has more
families if the amount of data they have is the same? Correct or not?

Is it a higher performance design to put fewer families into a table?

Thanks so much!

Best regards,
Bing
On Tue, Aug 13, 2013 at 12:31 AM, Stas Maksimov <[EMAIL PROTECTED]> wrote:
> Hi there,
>
> On your second point, I don't think column family can ever be an optional
> parameter, so I'm not sure this understanding is correct.
>
> Regards,
> Stas.
>
>
> On 12 August 2013 17:22, Bing Li <[EMAIL PROTECTED]> wrote:
>>
>> Hi, all,
>>
>> My understandings about HBase table and its family are as follows.
>>
>> 1) Each table can consist of multiple families;
>>
>> 2) When retrieving with SingleColumnValueFilter, if the family is
>> specified, other families contained in the same table are not
>> affected.
>>
>> Are these claims right? But I got a problem which conflicts with the
>> above understandings.
>>
>> In the following code, even though no any data in the family of
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY, the
>> for-loop runs many times if other families has the column of
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN.
>>
>> Is that normal in HBase? If so, I think it is not a good design. No
>> column overlaps must exist among the families of the same table?
>> Otherwise, retrieving the table must cause waste of scanning loops?
>>
>> Thanks so much!
>>
>> Best wishes,
>> Bing
>>
>>         SingleColumnValueFilter dcKeyFilter = new
>>
>> SingleColumnValueFilter(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_BASICS_FAMILY,
>> ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_DC_KEY_COLUMN,
>> CompareFilter.CompareOp.EQUAL, new SubstringComparator(dcKey));
>>         Scan scan = new Scan();
>>         scan.setFilter(dcKeyFilter);
>>         scan.setCaching(Parameters.CACHING_SIZE);
>>         scan.setBatch(Parameters.BATCHING_SIZE);
>>
>>         String qualifier;
>>         String hostNodeKey = SocialRole.NO_NODE_KEY;
>>         String groupKey = SocialGroup.NO_GROUP_KEY;
>>         int timingScale = TimingScale.NO_TIMING_SCALE;
>>         String key;
>>         try
>>         {
>>             ResultScanner scanner = this.neighborTable.getScanner(scan);
>>             for (Result result : scanner)
>>             {
>>                 for (KeyValue kv : result.raw())
>>                 {
>>                     qualifier = Bytes.toString(kv.getQualifier());
>>                     if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_NODE_KEY_STRING_COLUMN))
>>                     {
>>                         hostNodeKey = Bytes.toString(kv.getValue());
>>                     }
>>                     else if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_HOST_GROUP_KEY_STRING_COLUMN))
>>                     {
>>                         groupKey = Bytes.toString(kv.getValue());
>>                     }
>>                     else if
>>
>> (qualifier.equals(ContrivedNeighborStructure.NODE_NEIGHBOR_IN_GROUP_TIMING_SCALE_STRING_COLUMN))
>>                     {
>>                         timingScale = Bytes.toInt(kv.getValue());
>>                     }
>>                 }
>>                 if (!hostNodeKey.equals(SocialRole.NO_NODE_KEY) &&
>> !groupKey.equals(SocialGroup.NO_GROUP_KEY) && timingScale !>> TimingScale.NO_TIMING_SCALE)
>>                 {
>>                     key = Tools.GetKeyOfNode(hostNodeKey, groupKey,
>> timingScale);
>>                     if (!neighborMap.containsKey(key))
>>                     {
>>                         neighborMap.put(key, new
>> NodeNeighborInGroup(hostNodeKey, groupKey, timingScale));
>>                     }
>>                 }
>>                 hostNodeKey = SocialRole.NO_NODE_KEY;
>>                 groupKey = SocialGroup.NO_GROUP_KEY;
>>                 timingScale = TimingScale.NO_TIMING_SCALE;