Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Using Hadoop's MulitpleInputs with AccumuloInputFormat in a MR job


Copy link to this message
-
Re: Using Hadoop's MulitpleInputs with AccumuloInputFormat in a MR job
Adding to my previous response- when you say you are setting different
iterators on a scan are you referring to a single table with different
iterators? Are the sets of iterators tied to different ranges? The changes
we are making to the current InputFormat will still not allow different
iterators on a single table but the use case sounds interesting.
On Mon, Sep 16, 2013 at 3:55 PM, Corey Nolet <[EMAIL PROTECTED]> wrote:

> Aaron,
>
> We are currently re-working the AccumuloInputFormat for Accumulo 1.6 to
> provide inputs from multiple tables (each with their own set of configured
> iterators, ranges, columns). Check out ACCUMULO-391.
>
>
>
>
> On Mon, Sep 16, 2013 at 11:41 AM, Aaron <[EMAIL PROTECTED]> wrote:
>
>> I was curious if this is possible (i am thinking it isn't):  from the
>> Java API, Accumulo 1.5, Hadoop 1.2.1
>>
>> Want to set 2 different iterators on a scan, and send those results to 2
>> different Mappers.
>>
>> So, how'd i do this with files as inputs, is just to use MultipleInputs
>> class, with 2 different Path, and 2 different Mapper Classes, maybe the
>> same InputFormat (e.g Text or Sequence)
>>
>> Since I'm using AccumulInputFormat, I would think I'd be ok..maybe with a
>> null Path in the MulitpleInputs.addInputPath(), but it's the static
>> addIterator() on the AccumuloInputFormat that I think is where I lose.
>>
>> Can I have 2 different AccumuloInputFormats, with different iterators?  I
>> think the answer is no, and briefly looking at the source, believe that to
>> be correct..but, was curious if others have done have done something.
>>
>> Cheers,
>> Aaron
>>
>
>