Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - org.apache.accumulo.core.iterators.Combiner: key scope?


Copy link to this message
-
org.apache.accumulo.core.iterators.Combiner: key scope?
Jason Trost 2012-03-15, 11:33
I found myself needing a combiner that will sum the values of a row
where each key has the same row and column family (but col qual
differs).

I was looking through the Combiner class and I was wondering if there
would be any issues with making this line of the code configurable.
(line 70 in org.apache.accumulo.core.iterators.Combiner)

private boolean _hasNext() {
      return source.hasTop() && !source.getTopKey().isDeleted() &&
topKey.equals(source.getTopKey(),
PartialKey.ROW_COLFAM_COLQUAL_COLVIS);
    }

Specifically I was thinking it would be useful to be able to configure
the partial key field using one of the following values (from
org.apache.accumulo.core.data.PartialKey):
    ROW
    ROW_COLFAM
    ROW_COLFAM_COLQUAL
    ROW_COLFAM_COLQUAL_COLVIS

I see the main value here in using Combiners at scan time to perform
various rollups and counts.

I am guessing there may be some security implications of doing this?
Maybe the labels of aggregations based on any partial key not
including colvis would need to be combined.

Thoughts on this?

Thanks,

--Jason