Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - setiter returns error, but custom iterator runs via scanner client?


Copy link to this message
-
Re: setiter returns error, but custom iterator runs via scanner client?
David Arsenault 2013-02-28, 13:08
Josh,

In terms of the documentation, it would be great to augment the Table
Configuration section of the user manual. Iterators are central to
Accumulo's functionality so it would aid users to see an entire
iterator life cycle:
* Develop iterator by extending...
* Install iterator as jar to lib/ext
* Run as in a client (fragment is shown)
* Set via shell (setiter, config for options)

There are hints to using config for existing filters but not how to
configure custom iterator code. For example, installing custom iterator
code for non-scan contexts is unclear.

With a little help on the internals to get this right, I am happy to
contribute writing and example code fragments.

Thanks,
David

On Wednesday, February 27, 2013, Josh Elser wrote:

> Great!
>
> Do you mind if I forward this back to the dev@a.a.o list for others to
> benefit from?
>
> Also, is there a relevant place you looked in the documentation that you
> would've expected to find something like this? Where/when we lack
> documentation, we like to make a conscious effort towards improving. It's
> difficult at times when you understand the subject matter to know the best
> place to document something. Any feedback you might have would be greatly
> appreciated.
>
> Thanks!
>
> - Josh
>
> On 02/27/2013 09:41 PM, David Arsenault wrote:
>
>> Josh,
>>
>> Got it! Yes, this iterator does have some options that require
>> setting. I'll implement the OptionDescriber as you suggest. This fully
>> explains why it works in scanner code where the options are set.
>>
>> The syntax for `config` with a class where not all that clear to me.
>> The manual is a bit confusing w/r/t how you specify the iterator code
>> then set the options.
>>
>> Thanks for the help. I really appreciate it.
>>
>> David
>>
>> On Wed, Feb 27, 2013 at 8:33 PM, Josh Elser <[EMAIL PROTECTED]> wrote:
>>
>>> David,
>>>
>>> If you want to use the `setiter` command, have your
>>> SortedKeyValueIterator
>>> implementation also implement o.a.a.c.i.OptionDescriber.
>>>
>>> This will force you to implement a method which describes any
>>> configurable
>>> options on your SKVI, describeOptions(), in addition to configuring your
>>> SKVI with provided a Map<String,String> of options,
>>> validateOptions(Map<String,**String>).
>>>
>>> I believe `setiter` is telling you about `config` as you could manually
>>> set
>>> the your SKVI at a given scope (scan, minc, majc) without the automatic
>>> configuration that the OptionDescriber implementation provides you with
>>> the
>>> `setiter` command.
>>>
>>> - Josh
>>>
>>>
>>> On 02/27/2013 07:50 PM, David Arsenault wrote:
>>>
>>>> Hi there. Hoping to get some pointers on what might be the issue with
>>>> setiter here ...
>>>>
>>>> I've got a custom iterator built and working fine. The jar is in the
>>>> lib/ext directory and is happy when called from a scanner client.
>>>>
>>>> However, when I attempt to attach the iterator to a table via the
>>>> shell bad voodoo happens.
>>>>
>>>> somebody@accumulo1 INtable> setiter -t INtable -scan -p 5 -n OptsIter
>>>> -class org.foo.TableOptionsIterator
>>>>
>>>> 2013-02-27 19:30:00,974 [shell.Shell] ERROR:
>>>> org.apache.accumulo.core.util.**shell.ShellCommandException: Command
>>>> could not be initialized (Unable to load org.foo.TableOptionsIterator
>>>> as type org.apache.accumulo.core.**iterators.OptionDescriber; configure
>>>> with 'config' instead).
>>>>
>>>> My class implements SortedKeyValueIterator<Key,**Value>.
>>>>
>>>> I've searched extensively and covered the manual end-to-end with no
>>>> luck. Any hints would be much appreciated and save me more time and
>>>> pain.
>>>>
>>>> Thank you.
>>>> David
>>>>
>>>
>>>
>