Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Allowed upper limit to HColumnDescriptor.setMaxVersion(..)?


Copy link to this message
-
Re: Allowed upper limit to HColumnDescriptor.setMaxVersion(..)?
Are there any negative performance aspects to setting the max versions
to a large value if those extra stored versions are not used?  If I
set the max to 10k but really only store 100, there is not extra
diskspace/memory being consumed by the potential of having more
versions is there?  Also what about the inverse of writing, gets?  If
my "Gets" all call get.setMaxVersion() does setting that value to
being extremely despite there not being versions cause performance
problems?

Thanks for the help,
Micah

On Tue, Oct 4, 2011 at 10:52 PM, lars hofhansl <[EMAIL PROTECTED]> wrote:
> MaxVersions and MinVersions are different features.
> MaxVersion identifies the max number of versions you want to keep (just to state the obvious).
> MinVersion is used together with TTL (and soon with deletes - see HBASE-4536), to indicate the minimum number of version you want keep around even when they should be expired or were deleted.
>
>
> There is no way to disable MaxVersions, just set it to a very large number.
>
> MinVersions is by default disabled (setting is 0), which means rows past their TTL and deleted rows will be removed during compaction.
>
>
> I am thinking about how to state that more clearly in the documentation.
>
>
> -- Lars
>
>
>
> ________________________________
> From: Doug Meil <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Tuesday, October 4, 2011 4:32 PM
> Subject: Re: Allowed upper limit to HColumnDescriptor.setMaxVersion(..)?
>
>
> The default for versioning is 3, unfortunately the sub-section also cites
> (incorrectly) that the min is 0.  That sub-section is trying to indicate
> the minimum legal value.  I am working on clearing that entry up with
> another developer.
>
>
>
>
>
> On 10/4/11 6:04 PM, "Micah Whitacre" <[EMAIL PROTECTED]> wrote:
>
>>Are you surmising that from the description of setting a minimum version?
>>
>>On Tue, Oct 4, 2011 at 2:31 PM, Doug Meil <[EMAIL PROTECTED]>
>>wrote:
>>>
>>> http://hbase.apache.org/book.html#schema.versions
>>>
>>>
>>> I believe if you set that to 0 it should disable the versioning.
>>>
>>>
>>>
>>> On 10/4/11 2:21 PM, "Micah Whitacre" <[EMAIL PROTECTED]> wrote:
>>>
>>>>I guess what I'm asking is there a way to set "infinite" or no max
>>>>bounds on versions (e.g. setMaxVersion(-1) possibly)?  Or do I have to
>>>>call setMaxVersion(Integer.MAX_VALUE) or setMaxVersion(<some large
>>>>guess>)?  If a large guess is the way to go, what sort of overhead
>>>>costs might we need to consider when finding the right balance point
>>>>between room to grow and the maintenance support cost of needing to
>>>>expand later?
>>>>
>>>>We plan on building MapReduce jobs to clean up versions based on some
>>>>conditions so the value shouldn't get that large but the conditions
>>>>for cleaning up those versions might be decided by other consumers of
>>>>the service.  So having room to grow is ideal.
>>>>
>>>>On Tue, Oct 4, 2011 at 11:36 AM, Doug Meil
>>>><[EMAIL PROTECTED]> wrote:
>>>>>
>>>>> Hi there-
>>>>>
>>>>> re:  "i don't care store them all"
>>>>>
>>>>>
>>>>> What do you mean?
>>>>>
>>>>>
>>>>>
>>>>> On 10/4/11 12:20 PM, "Micah Whitacre" <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>>In reading the documentation all I've seen suggestions on how to set
>>>>>>the value and the default value.  However I haven't seen any
>>>>>>indication how to set the value to "i don't care store them all" or if
>>>>>>there is a maximum bounds aside from Integer.MAX_VALUE.  Does anyone
>>>>>>know?
>>>>>>
>>>>>>Thanks,
>>>>>>Micah
>>>>>>
>>>>>>[1] -
>>>>>>http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/HColumnDescrip
>>>>>>to
>>>>>>r.
>>>>>>html#setMaxVersions(int)
>>>>>
>>>>>
>>>
>>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB