Does Accumulo have the idea of record index within a table?  As an
example, take a table with the following values:

Row  ColFam  ColQual
1        1            10
1        1            20
1        1            30
As I understand it Accumulo will naturally sort the table in the above
order (first by row, then by colFam, lastly by colQual).  If I insert
(1, 1, 25), is there any way for me to get the index of the newly
inserted value (3 in this case)?

Further more is there anyway to lookup an index by knowing the full
row:colFam:colQual key (say (1, 1, 30))?

Lastly, is there an easy way to get the total number of rows within a
table (and the same question for ColFam's within a row and ColQual's
with a row:colFam pair)?

I've made it through the documentation and scanned through the mailing
list, but I haven't seen any information on the above (which lends me to
believe I'm asking for behaviour that Accumulo doesn't natively provide).

Many thanks for any information,

- MT

Michael,

Accumulo doesn't expose any internal offsets of Key-Value pairs through
the API. While it might be able to extrapolate some of this knowledge
from the underlying structure of Accumulo, that isn't the intent of what
Accumulo is trying to provide.

The first idea that comes to mind is that you could write an
SortedKeyValueIterator which takes some array of indices you wish to
find in the ColQual ([3], in your case) and perform the heavy-lifting on
the server. Your SKVI can scan through the data, consuming records which
do not match your index ([3]). Since you know that Accumulo will not
split your row "1" across multiple server (a tablet is never split
across a row), you should be able to fairly quickly get a stream back of
all key for the relative indices you provided. You can think of this as
a sequential read over the disk which should be relatively efficient.

The only caveat is that you would need to create a range to scan over
your entire row+colfam, for example:

Range=(1 1: , 1 1\x00: )

Where the colqual is empty.

- Josh

On 12/3/2012 9:04 PM, [EMAIL PROTECTED] wrote:
> Does Accumulo have the idea of record index within a table?  As an
> example, take a table with the following values:
>
> Row  ColFam  ColQual
> 1        1            10
> 1        1            20
> 1        1            30
>
>
> As I understand it Accumulo will naturally sort the table in the above
> order (first by row, then by colFam, lastly by colQual).  If I insert
> (1, 1, 25), is there any way for me to get the index of the newly
> inserted value (3 in this case)?
>
> Further more is there anyway to lookup an index by knowing the full
> row:colFam:colQual key (say (1, 1, 30))?
>
> Lastly, is there an easy way to get the total number of rows within a
> table (and the same question for ColFam's within a row and ColQual's
> with a row:colFam pair)?
>
> I've made it through the documentation and scanned through the mailing
> list, but I haven't seen any information on the above (which lends me to
> believe I'm asking for behaviour that Accumulo doesn't natively provide).
>
> Many thanks for any information,
>
> - MT

I was thinking a little more on the subject, and convinced myself that I
was wrong.

Since many files on disk correspond to a tablet, the best you can get is
the index of a key-value pair in a given file for a tablet. To get a
sorted stream of key-value pairs for this tablet (to compute index
offset for a key in a tablet), a merged read is performed over all of
those files. Local key offset for a file is meaningless as it does not
imply the correct offset for a tablet.

On 12/3/12 9:30 PM, Josh Elser wrote:
>
> Accumulo doesn't expose any internal offsets of Key-Value pairs
> through the API. While it might be able to extrapolate some of this
> knowledge from the underlying structure of Accumulo, that isn't the
> intent of what Accumulo is trying to provide.
>

It's worse than just merging the tablet sources and iterating to find the
offset... because the underlying sources may contain deleted records, old
versions that are filtered by an iterator, duplicates, and it is further
complicated if you are using combiners in the iterator stack.

Your best bet is probably to perform this sort of indexing within an ingest
framework that understands a bigger picture of how you will use the data
you are ingesting.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Tue, Dec 4, 2012 at 12:45 PM, Josh Elser <[EMAIL PROTECTED]> wrote:

> I was thinking a little more on the subject, and convinced myself that I
> was wrong.
>
> Since many files on disk correspond to a tablet, the best you can get is
> the index of a key-value pair in a given file for a tablet. To get a sorted
> stream of key-value pairs for this tablet (to compute index offset for a
> key in a tablet), a merged read is performed over all of those files. Local
> key offset for a file is meaningless as it does not imply the correct
> offset for a tablet.
>
>
> On 12/3/12 9:30 PM, Josh Elser wrote:
>
>>
>> Accumulo doesn't expose any internal offsets of Key-Value pairs through
>> the API. While it might be able to extrapolate some of this knowledge from
>> the underlying structure of Accumulo, that isn't the intent of what
>> Accumulo is trying to provide.
>>
>>
>