Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # dev >> client config files

Copy link to this message
Re: client config files
Okay, so there is implementation overlap, but that overlap is pretty
minimal (admittedly, it could potentially grow). The only thing it is
currently used for, is to carry the value of the RPC timeout, and this
is not currently very friendly to end users (they'd have to
instantiate something that extends AccumuloConfiguration, just to
override that one property's value, and then call

I like configuration with scopes, so I would prefer to have
configuration properties scoped to RPC. I implemented the
Property-enum configuration model to reflect this preference, but I
made the mistake of carrying all the scopes in a single configuration
object. I don't see a problem separating out the RPC scope into a
separate configuration object that could be shared between the other,
broader scopes ("client", "master", "tserver", etc.). In other words,
I'd prefer smaller, more localized scopes for configuration, and I
think it's sensible to have an RPC/connection scope (not necessarily
thrift-specific). See BatchWriterConfig for a good example of a
locally-scoped configuration.

Christopher L Tubbs II
On Fri, Aug 2, 2013 at 3:24 PM, Michael Berman <[EMAIL PROTECTED]> wrote:
> I believe it is an implementation overlap.  Both ZKInstance and the
> master-tablet thrift connections get created in ThriftUtil.getClient().
>  Higher up in the stack, in both paths, we have access to an Instance from
> which to draw configuration (with getConfiguration()).  In one case, it's a
> ZKInstance with a degenerate AccumuloConfiguration, and in the other case
> it's an HDFSInstance with a site.xml-backed configuration, but the thrift
> stack makes no distinction.  It seems silly to me to introduce a
> distinction all the way down the stack just so we can have two different
> config sources (which have many of the same flags).  Unless we were going
> to implement it as a ThriftConnectionConfiguration interface with named
> methods that both AccumuloConfiguration and ClientConfiguration could
> implement...but that would be a big departure from the Property enum
> configuration model.
> On Fri, Aug 2, 2013 at 2:29 PM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>> Yeah, I agree. Consistency with Hadoop here is probably not that valuable.
>> -Joey
>> On Fri, Aug 2, 2013 at 2:28 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>> > On Thu, Aug 1, 2013 at 4:33 PM, Joey Echeverria <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> I generally prefer properties files to XML, but there may be a argument
>> >> for reusing Hadoop's SSL configuration system which is XML based.
>> >>
>> >
>> > I also prefer prefer properties files over XML.   The only reason I can
>> > think that we might want to use XML is for consistency with Hadoop and
>> > Accumulo server side config.  But it does not seem like a very compelling
>> > reason, its not like it prop files are hard to use once you realize you
>> > need to use them.
>> >
>> >
>> >>
>> >>
>> >> -Joey
>> >> —
>> >> Sent from Mailbox for iPhone
>> >>
>> >> On Thu, Aug 1, 2013 at 3:08 PM, Christopher <[EMAIL PROTECTED]>
>> wrote:
>> >>
>> >> > ^ Another reason I like commons-configuration here is for
>> >> > property-interpolation with HierarchicalConfiguration.
>> >> > --
>> >> > Christopher L Tubbs II
>> >> > http://gravatar.com/ctubbsii
>> >> > On Thu, Aug 1, 2013 at 3:07 PM, Christopher <[EMAIL PROTECTED]>
>> wrote:
>> >> >> I absolutely DO think they should be combined in a properties file
>> >> >> located in $HOME/.accumulo/config
>> >> >> I absolutely DO NOT think this client configuration should be
>> >> >> exclusive to the shell, and I absolutely DO NOT think it should be
>> >> >> XML.
>> >> >>
>> >> >> I would love to see all our clients/client code use
>> >> >> commons-configuration to hold properties from the properties file, so
>> >> >> that only a --config parameter is needed (with reasonable defaults,
>> so
>> >> >> even that is not absolutely necessary). I also think that every