so .. we all know that benchmarks can be misleading but ...

There seems to be some excitement surrounding the recent Hypertable vs Hbase benchmark, particularly because hbase (0.90) failed to complete some tests, and because the tests seem to be based on those outlined in the original BigTable paper.

Hypertable has stated (on Twitter) that they're going to re-run the benchmark on Hbase 0.92, and that they hope for participation from Hbase. I wondered whether it would be in the Accumulo community's interest to participate as well?

I believe that because Accumulo uses native management of memory for it's ingest operations that it won't have the same problem Hbase 0.90 did with the tests involving lots of small (10k and 100k) inserts. We know that Accumulo stands apart somewhat because of it's unique features, but performance could be an important differentiator too, or at least should not be a weakness ...

Thoughts?

Aaron

I think benchmarks are interesting and useful, but they're not the whole
story. If we can collaborate with the Hypertable testing, that'd be best as
it should avoid getting into a benchmark shoot out where each community
feels compelled to one-up the other.

-Joey

On Mon, Feb 13, 2012 at 4:07 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:

> so .. we all know that benchmarks can be misleading but ...
>
> There seems to be some excitement surrounding the recent Hypertable vs
> Hbase benchmark, particularly because hbase (0.90) failed to complete some
> tests, and because the tests seem to be based on those outlined in the
> original BigTable paper.
>
> Hypertable has stated (on Twitter) that they're going to re-run the
> benchmark on Hbase 0.92, and that they hope for participation from Hbase. I
> wondered whether it would be in the Accumulo community's interest to
> participate as well?
>
> I believe that because Accumulo uses native management of memory for it's
> ingest operations that it won't have the same problem Hbase 0.90 did with
> the tests involving lots of small (10k and 100k) inserts. We know that
> Accumulo stands apart somewhat because of it's unique features, but
> performance could be an important differentiator too, or at least should
> not be a weakness ...
>
> Thoughts?
>
> Aaron
--
Joseph Echeverria
Cloudera, Inc.
443.305.9434

I think its a good idea, but who is going to take the time to do it?

On Mon, Feb 13, 2012 at 4:42 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
> I think benchmarks are interesting and useful, but they're not the whole
> story. If we can collaborate with the Hypertable testing, that'd be best as
> it should avoid getting into a benchmark shoot out where each community
> feels compelled to one-up the other.
>
> -Joey
>
>
> On Mon, Feb 13, 2012 at 4:07 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>>
>> so .. we all know that benchmarks can be misleading but ...
>>
>> There seems to be some excitement surrounding the recent Hypertable vs
>> Hbase benchmark, particularly because hbase (0.90) failed to complete some
>> tests, and because the tests seem to be based on those outlined in the
>> original BigTable paper.
>>
>> Hypertable has stated (on Twitter) that they're going to re-run the
>> benchmark on Hbase 0.92, and that they hope for participation from Hbase. I
>> wondered whether it would be in the Accumulo community's interest to
>> participate as well?
>>
>> I believe that because Accumulo uses native management of memory for it's
>> ingest operations that it won't have the same problem Hbase 0.90 did with
>> the tests involving lots of small (10k and 100k) inserts. We know that
>> Accumulo stands apart somewhat because of it's unique features, but
>> performance could be an important differentiator too, or at least should not
>> be a weakness ...
>>
>> Thoughts?
>>
>> Aaron
>
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
>

I thought I heard Aaron volunteering ;)

-Joey

On Feb 13, 2012, at 16:58, Keith Turner <[EMAIL PROTECTED]> wrote:

> I think its a good idea, but who is going to take the time to do it?
>
> On Mon, Feb 13, 2012 at 4:42 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>> I think benchmarks are interesting and useful, but they're not the whole
>> story. If we can collaborate with the Hypertable testing, that'd be best as
>> it should avoid getting into a benchmark shoot out where each community
>> feels compelled to one-up the other.
>>
>> -Joey
>>
>>
>> On Mon, Feb 13, 2012 at 4:07 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>>>
>>> so .. we all know that benchmarks can be misleading but ...
>>>
>>> There seems to be some excitement surrounding the recent Hypertable vs
>>> Hbase benchmark, particularly because hbase (0.90) failed to complete some
>>> tests, and because the tests seem to be based on those outlined in the
>>> original BigTable paper.
>>>
>>> Hypertable has stated (on Twitter) that they're going to re-run the
>>> benchmark on Hbase 0.92, and that they hope for participation from Hbase. I
>>> wondered whether it would be in the Accumulo community's interest to
>>> participate as well?
>>>
>>> I believe that because Accumulo uses native management of memory for it's
>>> ingest operations that it won't have the same problem Hbase 0.90 did with
>>> the tests involving lots of small (10k and 100k) inserts. We know that
>>> Accumulo stands apart somewhat because of it's unique features, but
>>> performance could be an important differentiator too, or at least should not
>>> be a weakness ...
>>>
>>> Thoughts?
>>>
>>> Aaron
>>
>>
>>
>>
>> --
>> Joseph Echeverria
>> Cloudera, Inc.
>> 443.305.9434
>>

I would be interested in working w/ someone on this.  Right now I am
concentrating on getting 1.4 release ready, and that is keeping me
pretty busy.  After we release 1.4 I may have some time.

On Mon, Feb 13, 2012 at 5:47 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
> I thought I heard Aaron volunteering ;)
>
> -Joey
>
> On Feb 13, 2012, at 16:58, Keith Turner <[EMAIL PROTECTED]> wrote:
>
>> I think its a good idea, but who is going to take the time to do it?
>>
>> On Mon, Feb 13, 2012 at 4:42 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>>> I think benchmarks are interesting and useful, but they're not the whole
>>> story. If we can collaborate with the Hypertable testing, that'd be best as
>>> it should avoid getting into a benchmark shoot out where each community
>>> feels compelled to one-up the other.
>>>
>>> -Joey
>>>
>>>
>>> On Mon, Feb 13, 2012 at 4:07 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>>>>
>>>> so .. we all know that benchmarks can be misleading but ...
>>>>
>>>> There seems to be some excitement surrounding the recent Hypertable vs
>>>> Hbase benchmark, particularly because hbase (0.90) failed to complete some
>>>> tests, and because the tests seem to be based on those outlined in the
>>>> original BigTable paper.
>>>>
>>>> Hypertable has stated (on Twitter) that they're going to re-run the
>>>> benchmark on Hbase 0.92, and that they hope for participation from Hbase. I
>>>> wondered whether it would be in the Accumulo community's interest to
>>>> participate as well?
>>>>
>>>> I believe that because Accumulo uses native management of memory for it's
>>>> ingest operations that it won't have the same problem Hbase 0.90 did with
>>>> the tests involving lots of small (10k and 100k) inserts. We know that
>>>> Accumulo stands apart somewhat because of it's unique features, but
>>>> performance could be an important differentiator too, or at least should not
>>>> be a weakness ...
>>>>
>>>> Thoughts?
>>>>
>>>> Aaron
>>>
>>>
>>>
>>>
>>> --
>>> Joseph Echeverria
>>> Cloudera, Inc.
>>> 443.305.9434
>>>

Keith,

Why don't I contact the hypertable guys to see when their next benchmark is planned?

Sent from my iPhone

On Feb 13, 2012, at 6:23 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> I would be interested in working w/ someone on this.  Right now I am
> concentrating on getting 1.4 release ready, and that is keeping me
> pretty busy.  After we release 1.4 I may have some time.
>
> On Mon, Feb 13, 2012 at 5:47 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>> I thought I heard Aaron volunteering ;)
>>
>> -Joey
>>
>> On Feb 13, 2012, at 16:58, Keith Turner <[EMAIL PROTECTED]> wrote:
>>
>>> I think its a good idea, but who is going to take the time to do it?
>>>
>>> On Mon, Feb 13, 2012 at 4:42 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>>>> I think benchmarks are interesting and useful, but they're not the whole
>>>> story. If we can collaborate with the Hypertable testing, that'd be best as
>>>> it should avoid getting into a benchmark shoot out where each community
>>>> feels compelled to one-up the other.
>>>>
>>>> -Joey
>>>>
>>>>
>>>> On Mon, Feb 13, 2012 at 4:07 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>> so .. we all know that benchmarks can be misleading but ...
>>>>>
>>>>> There seems to be some excitement surrounding the recent Hypertable vs
>>>>> Hbase benchmark, particularly because hbase (0.90) failed to complete some
>>>>> tests, and because the tests seem to be based on those outlined in the
>>>>> original BigTable paper.
>>>>>
>>>>> Hypertable has stated (on Twitter) that they're going to re-run the
>>>>> benchmark on Hbase 0.92, and that they hope for participation from Hbase. I
>>>>> wondered whether it would be in the Accumulo community's interest to
>>>>> participate as well?
>>>>>
>>>>> I believe that because Accumulo uses native management of memory for it's
>>>>> ingest operations that it won't have the same problem Hbase 0.90 did with
>>>>> the tests involving lots of small (10k and 100k) inserts. We know that
>>>>> Accumulo stands apart somewhat because of it's unique features, but
>>>>> performance could be an important differentiator too, or at least should not
>>>>> be a weakness ...
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Aaron
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Joseph Echeverria
>>>> Cloudera, Inc.
>>>> 443.305.9434
>>>>

That sounds great.

On Tue, Feb 14, 2012 at 10:46 AM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
> Keith,
>
> Why don't I contact the hypertable guys to see when their next benchmark is planned?
>
> Sent from my iPhone
>
> On Feb 13, 2012, at 6:23 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>
>> I would be interested in working w/ someone on this.  Right now I am
>> concentrating on getting 1.4 release ready, and that is keeping me
>> pretty busy.  After we release 1.4 I may have some time.
>>
>> On Mon, Feb 13, 2012 at 5:47 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>>> I thought I heard Aaron volunteering ;)
>>>
>>> -Joey
>>>
>>> On Feb 13, 2012, at 16:58, Keith Turner <[EMAIL PROTECTED]> wrote:
>>>
>>>> I think its a good idea, but who is going to take the time to do it?
>>>>
>>>> On Mon, Feb 13, 2012 at 4:42 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>>>>> I think benchmarks are interesting and useful, but they're not the whole
>>>>> story. If we can collaborate with the Hypertable testing, that'd be best as
>>>>> it should avoid getting into a benchmark shoot out where each community
>>>>> feels compelled to one-up the other.
>>>>>
>>>>> -Joey
>>>>>
>>>>>
>>>>> On Mon, Feb 13, 2012 at 4:07 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>> so .. we all know that benchmarks can be misleading but ...
>>>>>>
>>>>>> There seems to be some excitement surrounding the recent Hypertable vs
>>>>>> Hbase benchmark, particularly because hbase (0.90) failed to complete some
>>>>>> tests, and because the tests seem to be based on those outlined in the
>>>>>> original BigTable paper.
>>>>>>
>>>>>> Hypertable has stated (on Twitter) that they're going to re-run the
>>>>>> benchmark on Hbase 0.92, and that they hope for participation from Hbase. I
>>>>>> wondered whether it would be in the Accumulo community's interest to
>>>>>> participate as well?
>>>>>>
>>>>>> I believe that because Accumulo uses native management of memory for it's
>>>>>> ingest operations that it won't have the same problem Hbase 0.90 did with
>>>>>> the tests involving lots of small (10k and 100k) inserts. We know that
>>>>>> Accumulo stands apart somewhat because of it's unique features, but
>>>>>> performance could be an important differentiator too, or at least should not
>>>>>> be a weakness ...
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> Aaron
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Joseph Echeverria
>>>>> Cloudera, Inc.
>>>>> 443.305.9434
>>>>>