Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - Camel Accumulo


Copy link to this message
-
Re: Camel Accumulo
Jesse Yates 2011-10-26, 14:37
Comments inline.
-------------------
Jesse Yates
240-888-2200
@jesse_yates

On Wed, Oct 26, 2011 at 6:39 AM, <[EMAIL PROTECTED]> wrote:

>
>
> Eric,
>
>
>
> I'm confused about the need for a username/password. Does Accumulo validate
> against that, or does it accept a set of priveledges for a user?
It checks that user and password against an internal db of users, giving the
user ability to read from a subset of the permissions on the system
('Authorizations' in Accumulo parlance).
> Also, in your discussion of zookeeper, what does the "zookeepers" attribute
> represent?
>

The address of the zookeeper servers keeping track of the accumulo instance.
This is the 'root' truth of where things are for clients.
>
>
>
> For my first stab at this, I'm using the ReadandWrite example to show me
> how to connect to Accumulo and perform reads/writes .  Is there a better one
> that I should be using?  This example class seems pretty heavy-weight...
>
>
It might be a little bit heavy, but I would rather have a complete example,
than a bunch of scattered sub-pieces. I think once you have the connection
setup, the actual read/write shouldn't be too bad.
>
>
> For the table name, I'd like to keep the camel component as generic as
> possible. So, to allow for this, I'm going to require folks to pass in the
> table name  on their accumulate url. Is this ok?
>
>
>
> Full Ack on the "accumulate" versus "accum" uri.  The only reason I
> attempted to shorten it is that looks like the trend for other
> camel-components.  That said, I can live with a longer name if it makes it
> easier to use.  Along these lines, I've thought that having a different uri
> for reads, writes, and mutates would be an easy way to handle interactions
> through the camel component. To help with this, I'm thinking that he
> following uri's would be use ful (developed in this order) :
>
> accumulate-read
>
> accumulate-write
>
> accumulate-mutate
>
> accumulate-batch-read
>
> accumulate-batch-write
>
>
>
> How does that sound?
>

shouldn't this be 'accumulo' or is this for implying accumulation of
information via Camel?
>
>
>
> Also, how do you feel about changing the build so that it creates bundles
> instead of .jar files?  All a bundle is, is a .jar file with a MANIFEST.MF
> file that has some extra attributes.  The maven-bundle-plugin can take care
> of that pretty easily, but it will change the packaging type from jar to
> bundle.  This would really only impact folks who are looking for a packaging
> type in thier .pom file dependencies, which is not common.
>
>
I would argue vehemently against changing the core build for an external
process (no offense mike). I think it would be okay if we have a profile in
the build that builds a bundle, but since that is a pretty uncommon distro,
people will easily get confused, leading to lower adoption, lots of
questions on user@, etc. Yes, its a low likelyhood, but I would rather make
it is as easy as possible to 'do the right thing'.

Camel (and other systems that need a bundle) are a special case, so already
people are going to be be a little more advanced and expect slightly
'special' versions.

>
>
> Mike Van
>
>
>
> ----- Original Message -----
>
>
> From: "Eric Newton" <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Wednesday, October 26, 2011 9:06:19 AM
> Subject: Re: Camel Accumulo
>
> Hi Mike,
>
> Scan auths are needed for reading, not for writing.
>
> Looking at constructors and factory methods looks like we'll need:
>
> ZooKeeperInstance:
>     zookeepers
>     instance name
>
> getConnector():
>     username
>     password
>
> getMultiTableBatchWriter():
>      memory
>      latency
>      write threads
>
> This would allow you to stream in tuples of (table, row, cf, cq,
> visibility,
> timestamp).  If you already have a connector for HBase, we would probably
> want something compatible with that, and that is probably table oriented
> (just guessing, being unfamiliar with Camel).  So, perhaps if you specify a