Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Using Hadoop's MulitpleInputs with AccumuloInputFormat in a MR job


Copy link to this message
-
Re: Using Hadoop's MulitpleInputs with AccumuloInputFormat in a MR job
Adding to my previous response- when you say you are setting different
iterators on a scan are you referring to a single table with different
iterators? Are the sets of iterators tied to different ranges? The changes
we are making to the current InputFormat will still not allow different
iterators on a single table but the use case sounds interesting.
On Mon, Sep 16, 2013 at 3:55 PM, Corey Nolet <[EMAIL PROTECTED]> wrote:

> Aaron,
>
> We are currently re-working the AccumuloInputFormat for Accumulo 1.6 to
> provide inputs from multiple tables (each with their own set of configured
> iterators, ranges, columns). Check out ACCUMULO-391.
>
>
>
>
> On Mon, Sep 16, 2013 at 11:41 AM, Aaron <[EMAIL PROTECTED]> wrote:
>
>> I was curious if this is possible (i am thinking it isn't):  from the
>> Java API, Accumulo 1.5, Hadoop 1.2.1
>>
>> Want to set 2 different iterators on a scan, and send those results to 2
>> different Mappers.
>>
>> So, how'd i do this with files as inputs, is just to use MultipleInputs
>> class, with 2 different Path, and 2 different Mapper Classes, maybe the
>> same InputFormat (e.g Text or Sequence)
>>
>> Since I'm using AccumulInputFormat, I would think I'd be ok..maybe with a
>> null Path in the MulitpleInputs.addInputPath(), but it's the static
>> addIterator() on the AccumuloInputFormat that I think is where I lose.
>>
>> Can I have 2 different AccumuloInputFormats, with different iterators?  I
>> think the answer is no, and briefly looking at the source, believe that to
>> be correct..but, was curious if others have done have done something.
>>
>> Cheers,
>> Aaron
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB