Folks,

Now that we're past feature freeze for 1.5, I'd like to set up a discussion
of the features that Accumulo developers are interested in for 1.6 and
beyond. The purpose of this discussion will be to raise awareness of
projects that you think are cool and important, and to rally the community.
As a proposed format, let's plan on a series of 2-5 minute presentations of
individual features -- what it is, why it's important, how should we build
it, and who needs to help. We'll record the presentations for posterity.

Since the community is now distributed throughout several states, let's try
our hand at an online discussion. Depending on participation, we'll pick a
forum that can accomodate everyone. If you're interested in attending,
please fill out this doodle with what time you would be available:
http://www.doodle.com/w9zb77ya3ykxr4zv

If you can, please try to fill out the survey by Tuesday afternoon. After
we get a good feel for times and participation, I'll schedule an online
forum and send out a meeting invitation. If you would like to present a
feature, send me an email directly at [EMAIL PROTECTED], and I'll make sure
you get time to present it.

I look forward to seeing what everyone has up their sleeves!

Cheers,
Adam

- The ability to lock a table (or perhaps locality group(s)) to memory
would be very intriguing. I know this is somewhat achievable via the
data block and index block cache, however I'm not sure if that's
entirely the same? At a minimum, we currently only have on/off caching
for a table.
- Continuous-ingest is pretty good baseline for determining how good of
a configuration exists. I think we could easily take it a step farther
to make >1node installation/initial-setup much less error prone, not to
mention create a more stable test harness for various "normal" Accumulo
workloads. This could also make internal testing much easier and
reproducible.
- Focus on the proxy. I think making the proxy a very user-friendly
endpoint will vastly increase the usability of Accumulo overall. It no
longer forces everyone into Java (or jython/jruby); however the proxy
could still use some love (I went to use it one night and then realized
I had to update my installed version of Thrift, boo).

Parroting some stuff other people mentioned..

- cleanup hook on SKVI
- Scanner/BatchScanner/ScannerBase cleanup
- auto tablet merge
On 01/26/2013 08:10 PM, Adam Fuchs wrote:
> Folks,
>
> Now that we're past feature freeze for 1.5, I'd like to set up a discussion
> of the features that Accumulo developers are interested in for 1.6 and
> beyond. The purpose of this discussion will be to raise awareness of
> projects that you think are cool and important, and to rally the community.
> As a proposed format, let's plan on a series of 2-5 minute presentations of
> individual features -- what it is, why it's important, how should we build
> it, and who needs to help. We'll record the presentations for posterity.
>
> Since the community is now distributed throughout several states, let's try
> our hand at an online discussion. Depending on participation, we'll pick a
> forum that can accomodate everyone. If you're interested in attending,
> please fill out this doodle with what time you would be available:
> http://www.doodle.com/w9zb77ya3ykxr4zv
>
> If you can, please try to fill out the survey by Tuesday afternoon. After
> we get a good feel for times and participation, I'll schedule an online
> forum and send out a meeting invitation. If you would like to present a
> feature, send me an email directly at [EMAIL PROTECTED], and I'll make sure
> you get time to present it.
>
> I look forward to seeing what everyone has up their sleeves!
>
> Cheers,
> Adam
>

This will be very useful.

Some new features I am thinking about for 1.6 are replication, CAS,
and Percolator.  I would like to put together a bit of an initial
design document for each of these.  I'm thinking that a follow up
online deep dive on the design of each large new 1.6 feature would
also be very useful.

In the coming week I plan to spend most of my time testing,
documenting, fixing bugs, and polishing 1.5.0.  I do not think this
would leave much time to work on 1.6.  So interleaving design
discussion about new 1.6 features while 1.5 is being polished would be
nice.  So once 1.5.0 ships, I would have peer reviewed designs to
start implementing.

Keith

On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
> Folks,
>
> Now that we're past feature freeze for 1.5, I'd like to set up a discussion
> of the features that Accumulo developers are interested in for 1.6 and
> beyond. The purpose of this discussion will be to raise awareness of
> projects that you think are cool and important, and to rally the community.
> As a proposed format, let's plan on a series of 2-5 minute presentations of
> individual features -- what it is, why it's important, how should we build
> it, and who needs to help. We'll record the presentations for posterity.
>
> Since the community is now distributed throughout several states, let's try
> our hand at an online discussion. Depending on participation, we'll pick a
> forum that can accomodate everyone. If you're interested in attending,
> please fill out this doodle with what time you would be available:
> http://www.doodle.com/w9zb77ya3ykxr4zv
>
> If you can, please try to fill out the survey by Tuesday afternoon. After
> we get a good feel for times and participation, I'll schedule an online
> forum and send out a meeting invitation. If you would like to present a
> feature, send me an email directly at [EMAIL PROTECTED], and I'll make sure
> you get time to present it.
>
> I look forward to seeing what everyone has up their sleeves!
>
> Cheers,
> Adam

What is CAS?

   - Switch to Curator.
   - Monitoring.  I'm hoping this will just be integration with an existing
   tool.
   - Namespaces.
   - A simple query language.  Perhaps there's something we can collaborate
   with some of the other NoSQL folks.
   - Update the wiki search example.
   - Automatic tablet merge.
   - Scalable NN

Some of these don't need design documents, but some discussion of
requirements would be nice.

-Eric

On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> This will be very useful.
>
> Some new features I am thinking about for 1.6 are replication, CAS,
> and Percolator.  I would like to put together a bit of an initial
> design document for each of these.  I'm thinking that a follow up
> online deep dive on the design of each large new 1.6 feature would
> also be very useful.
>
> In the coming week I plan to spend most of my time testing,
> documenting, fixing bugs, and polishing 1.5.0.  I do not think this
> would leave much time to work on 1.6.  So interleaving design
> discussion about new 1.6 features while 1.5 is being polished would be
> nice.  So once 1.5.0 ships, I would have peer reviewed designs to
> start implementing.
>
> Keith
>
> On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
> > Folks,
> >
> > Now that we're past feature freeze for 1.5, I'd like to set up a
> discussion
> > of the features that Accumulo developers are interested in for 1.6 and
> > beyond. The purpose of this discussion will be to raise awareness of
> > projects that you think are cool and important, and to rally the
> community.
> > As a proposed format, let's plan on a series of 2-5 minute presentations
> of
> > individual features -- what it is, why it's important, how should we
> build
> > it, and who needs to help. We'll record the presentations for posterity.
> >
> > Since the community is now distributed throughout several states, let's
> try
> > our hand at an online discussion. Depending on participation, we'll pick
> a
> > forum that can accomodate everyone. If you're interested in attending,
> > please fill out this doodle with what time you would be available:
> > http://www.doodle.com/w9zb77ya3ykxr4zv
> >
> > If you can, please try to fill out the survey by Tuesday afternoon. After
> > we get a good feel for times and participation, I'll schedule an online
> > forum and send out a meeting invitation. If you would like to present a
> > feature, send me an email directly at [EMAIL PROTECTED], and I'll make
> sure
> > you get time to present it.
> >
> > I look forward to seeing what everyone has up their sleeves!
> >
> > Cheers,
> > Adam
>

Just saw the ticket: CAS == ACCUMULO-1000

Compare-And-Set.

-Eric

On Sun, Jan 27, 2013 at 3:43 PM, Eric Newton <[EMAIL PROTECTED]> wrote:

> What is CAS?
>
>    - Switch to Curator.
>    - Monitoring.  I'm hoping this will just be integration with an
>    existing tool.
>    - Namespaces.
>    - A simple query language.  Perhaps there's something we can
>    collaborate with some of the other NoSQL folks.
>    - Update the wiki search example.
>    - Automatic tablet merge.
>    - Scalable NN
>
> Some of these don't need design documents, but some discussion of
> requirements would be nice.
>
> -Eric
>
>
>
> On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>
>> This will be very useful.
>>
>> Some new features I am thinking about for 1.6 are replication, CAS,
>> and Percolator.  I would like to put together a bit of an initial
>> design document for each of these.  I'm thinking that a follow up
>> online deep dive on the design of each large new 1.6 feature would
>> also be very useful.
>>
>> In the coming week I plan to spend most of my time testing,
>> documenting, fixing bugs, and polishing 1.5.0.  I do not think this
>> would leave much time to work on 1.6.  So interleaving design
>> discussion about new 1.6 features while 1.5 is being polished would be
>> nice.  So once 1.5.0 ships, I would have peer reviewed designs to
>> start implementing.
>>
>> Keith
>>
>> On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>> > Folks,
>> >
>> > Now that we're past feature freeze for 1.5, I'd like to set up a
>> discussion
>> > of the features that Accumulo developers are interested in for 1.6 and
>> > beyond. The purpose of this discussion will be to raise awareness of
>> > projects that you think are cool and important, and to rally the
>> community.
>> > As a proposed format, let's plan on a series of 2-5 minute
>> presentations of
>> > individual features -- what it is, why it's important, how should we
>> build
>> > it, and who needs to help. We'll record the presentations for posterity.
>> >
>> > Since the community is now distributed throughout several states, let's
>> try
>> > our hand at an online discussion. Depending on participation, we'll
>> pick a
>> > forum that can accomodate everyone. If you're interested in attending,
>> > please fill out this doodle with what time you would be available:
>> > http://www.doodle.com/w9zb77ya3ykxr4zv
>> >
>> > If you can, please try to fill out the survey by Tuesday afternoon.
>> After
>> > we get a good feel for times and participation, I'll schedule an online
>> > forum and send out a meeting invitation. If you would like to present a
>> > feature, send me an email directly at [EMAIL PROTECTED], and I'll make
>> sure
>> > you get time to present it.
>> >
>> > I look forward to seeing what everyone has up their sleeves!
>> >
>> > Cheers,
>> > Adam
>>
>
>

How about letting users see and restart processes from the Monitor page?
Splitting on tablet record count as well as tablet size?

I think new monitor features could use a whole separate discussion. Being
able to do everything from the browser implies an officially supported
webservice of sorts.

Seems like monitor upgrades should be on the list though.
On Mon, Jan 28, 2013 at 10:46 AM, David Medinets
<[EMAIL PROTECTED]>wrote:

> How about letting users see and restart processes from the Monitor page?
> Splitting on tablet record count as well as tablet size?
>

A big concern with the monitor has always been data spillage due to tablet
split points being only base64 encoded.

Perhaps wrapping the currents calls with some authentication would make the
current monitor more extendable in addition to the ease of other monitor
processes being created.

Regardless, yes to think about the monitor :)

On Monday, January 28, 2013, Jim Klucar <[EMAIL PROTECTED]> wrote:
> I think new monitor features could use a whole separate discussion. Being
> able to do everything from the browser implies an officially supported
> webservice of sorts.
>
> Seems like monitor upgrades should be on the list though.
>
>
> On Mon, Jan 28, 2013 at 10:46 AM, David Medinets
> <[EMAIL PROTECTED]>wrote:
>
>> How about letting users see and restart processes from the Monitor page?
>> Splitting on tablet record count as well as tablet size?
>>
>

Just to be clear, split point data displayed is an MD5 hash, not base64
data from the metadata table.

-Eric

On Mon, Jan 28, 2013 at 12:03 PM, Josh Elser <[EMAIL PROTECTED]> wrote:

> A big concern with the monitor has always been data spillage due to tablet
> split points being only base64 encoded.
>
> Perhaps wrapping the currents calls with some authentication would make the
> current monitor more extendable in addition to the ease of other monitor
> processes being created.
>
> Regardless, yes to think about the monitor :)
>
> On Monday, January 28, 2013, Jim Klucar <[EMAIL PROTECTED]> wrote:
> > I think new monitor features could use a whole separate discussion. Being
> > able to do everything from the browser implies an officially supported
> > webservice of sorts.
> >
> > Seems like monitor upgrades should be on the list though.
> >
> >
> > On Mon, Jan 28, 2013 at 10:46 AM, David Medinets
> > <[EMAIL PROTECTED]>wrote:
> >
> >> How about letting users see and restart processes from the Monitor page?
> >> Splitting on tablet record count as well as tablet size?
> >>
> >
>

Ack, thanks for the correction, Eric.

On 01/28/2013 02:26 PM, Eric Newton wrote:
> Just to be clear, split point data displayed is an MD5 hash, not base64
> data from the metadata table.
>
> -Eric
>
>
>
> On Mon, Jan 28, 2013 at 12:03 PM, Josh Elser <[EMAIL PROTECTED]> wrote:
>
>> A big concern with the monitor has always been data spillage due to tablet
>> split points being only base64 encoded.
>>
>> Perhaps wrapping the currents calls with some authentication would make the
>> current monitor more extendable in addition to the ease of other monitor
>> processes being created.
>>
>> Regardless, yes to think about the monitor :)
>>
>> On Monday, January 28, 2013, Jim Klucar <[EMAIL PROTECTED]> wrote:
>>> I think new monitor features could use a whole separate discussion. Being
>>> able to do everything from the browser implies an officially supported
>>> webservice of sorts.
>>>
>>> Seems like monitor upgrades should be on the list though.
>>>
>>>
>>> On Mon, Jan 28, 2013 at 10:46 AM, David Medinets
>>> <[EMAIL PROTECTED]>wrote:
>>>
>>>> How about letting users see and restart processes from the Monitor page?
>>>> Splitting on tablet record count as well as tablet size?
>>>>

I'd like to see:

- Data triggers on insertion
- REST interface for looking up ranges of keys
- A DSL or some other interpreted language for crafting iterators
  - there's the clojure iterator, but something like python (via jython) or
javascript (via rhino) would be more adoptable
- Adding a clean up hook to iterators
- Allowing iterators to launch connections to other services (caching,
other tservers) to retrieve or write data
- Merging of the batch scanner and scanner implementations
  - a batch scanner with 1 thread have the same behavior as a scanner
  - scanners have a close() method on them
- Adding some builder interface for creating and introspecting iterator
stacks
- Clients being able to scan to specific keys using the scan command

"- Allowing iterators to launch connections to other services (caching,
other tservers) to retrieve or write data"

  What does allow mean in this context? I don't think its disallowed (I know
of an iterator that does this).

-----Original Message-----
From: William Slacum [mailto:[EMAIL PROTECTED]]
Sent: Monday, January 28, 2013 7:13 PM
To: [EMAIL PROTECTED]
Subject: Re: Accumulo 1.6 and beyond feature summit

I'd like to see:

- Data triggers on insertion
- REST interface for looking up ranges of keys
- A DSL or some other interpreted language for crafting iterators
  - there's the clojure iterator, but something like python (via jython) or
javascript (via rhino) would be more adoptable
- Adding a clean up hook to iterators
- Allowing iterators to launch connections to other services (caching, other
tservers) to retrieve or write data
- Merging of the batch scanner and scanner implementations
  - a batch scanner with 1 thread have the same behavior as a scanner
  - scanners have a close() method on them
- Adding some builder interface for creating and introspecting iterator
stacks
- Clients being able to scan to specific keys using the scan command

Currently it's not recommended to launch a batch scanner from an iterator
and retrieve new information, due to the possibility of a dead lock. Other
services may alleviate that concern, but due to lifecycle management issues
(related to the "add a clean up method to iterators"), it's not fool proof
to clean up connections from it.

On Mon, Jan 28, 2013 at 7:21 PM, Dave Marion <[EMAIL PROTECTED]> wrote:

> "- Allowing iterators to launch connections to other services (caching,
> other tservers) to retrieve or write data"
>
>   What does allow mean in this context? I don't think its disallowed (I
> know
> of an iterator that does this).
>
> -----Original Message-----
> From: William Slacum [mailto:[EMAIL PROTECTED]]
> Sent: Monday, January 28, 2013 7:13 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Accumulo 1.6 and beyond feature summit
>
> I'd like to see:
>
> - Data triggers on insertion
> - REST interface for looking up ranges of keys
> - A DSL or some other interpreted language for crafting iterators
>   - there's the clojure iterator, but something like python (via jython) or
> javascript (via rhino) would be more adoptable
> - Adding a clean up hook to iterators
> - Allowing iterators to launch connections to other services (caching,
> other
> tservers) to retrieve or write data
> - Merging of the batch scanner and scanner implementations
>   - a batch scanner with 1 thread have the same behavior as a scanner
>   - scanners have a close() method on them
> - Adding some builder interface for creating and introspecting iterator
> stacks
> - Clients being able to scan to specific keys using the scan command
>
>

Ok. That one that I know of doesn't spin up a batch scanner, it makes a call
to another program.

-----Original Message-----
From: William Slacum [mailto:[EMAIL PROTECTED]]
Sent: Monday, January 28, 2013 7:33 PM
To: [EMAIL PROTECTED]
Subject: Re: Accumulo 1.6 and beyond feature summit

Currently it's not recommended to launch a batch scanner from an iterator
and retrieve new information, due to the possibility of a dead lock. Other
services may alleviate that concern, but due to lifecycle management issues
(related to the "add a clean up method to iterators"), it's not fool proof
to clean up connections from it.

On Mon, Jan 28, 2013 at 7:21 PM, Dave Marion <[EMAIL PROTECTED]> wrote:

> "- Allowing iterators to launch connections to other services
> (caching, other tservers) to retrieve or write data"
>
>   What does allow mean in this context? I don't think its disallowed
> (I know of an iterator that does this).
>
> -----Original Message-----
> From: William Slacum [mailto:[EMAIL PROTECTED]]
> Sent: Monday, January 28, 2013 7:13 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Accumulo 1.6 and beyond feature summit
>
> I'd like to see:
>
> - Data triggers on insertion
> - REST interface for looking up ranges of keys
> - A DSL or some other interpreted language for crafting iterators
>   - there's the clojure iterator, but something like python (via
> jython) or javascript (via rhino) would be more adoptable
> - Adding a clean up hook to iterators
> - Allowing iterators to launch connections to other services (caching,
> other
> tservers) to retrieve or write data
> - Merging of the batch scanner and scanner implementations
>   - a batch scanner with 1 thread have the same behavior as a scanner
>   - scanners have a close() method on them
> - Adding some builder interface for creating and introspecting
> iterator stacks
> - Clients being able to scan to specific keys using the scan command
>
>

On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
<[EMAIL PROTECTED]> wrote:
> I'd like to see:
>
> - Data triggers on insertion
> - REST interface for looking up ranges of keys
> - A DSL or some other interpreted language for crafting iterators
>   - there's the clojure iterator, but something like python (via jython) or
> javascript (via rhino) would be more adoptable
> - Adding a clean up hook to iterators

I was thinking about this.   If we added a close() method to the SKVI
interface then it would break existing iterators.  Another option
would be to support closing iterators that implement Closeable.  So if
in iterators is an intstanceof Closeable then the framework could
close it when its finished with the iterator.   I wish there had been
a 1.5 ticket for this, I think it would have been fairly simple to
implement.

> - Allowing iterators to launch connections to other services (caching,
> other tservers) to retrieve or write data
> - Merging of the batch scanner and scanner implementations
>   - a batch scanner with 1 thread have the same behavior as a scanner
>   - scanners have a close() method on them
> - Adding some builder interface for creating and introspecting iterator
> stacks
> - Clients being able to scan to specific keys using the scan command

Features I'd like to see:

1. Better user experience for configuration: separate, *non-xml*
configuration files for separate services, and for clients.
2. Separate packaging for separate services (tservers, monitor, master,
trace), especially for RPMs and DEBs to make provisioning easier.
3. A suite of command-line utilities based on the shell's commands, that
work in Bash, rather than be restricted to the limitations of the shell to
issue these commands.
4. Provide a script maintained and supported by the community to provision
an Accumulo cloud instance in EC2 or OpenStack.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
> <[EMAIL PROTECTED]> wrote:
> > I'd like to see:
> >
> > - Data triggers on insertion
> > - REST interface for looking up ranges of keys
> > - A DSL or some other interpreted language for crafting iterators
> >   - there's the clojure iterator, but something like python (via jython)
> or
> > javascript (via rhino) would be more adoptable
> > - Adding a clean up hook to iterators
>
> I was thinking about this.   If we added a close() method to the SKVI
> interface then it would break existing iterators.  Another option
> would be to support closing iterators that implement Closeable.  So if
> in iterators is an intstanceof Closeable then the framework could
> close it when its finished with the iterator.   I wish there had been
> a 1.5 ticket for this, I think it would have been fairly simple to
> implement.
>
> > - Allowing iterators to launch connections to other services (caching,
> > other tservers) to retrieve or write data
> > - Merging of the batch scanner and scanner implementations
> >   - a batch scanner with 1 thread have the same behavior as a scanner
> >   - scanners have a close() method on them
> > - Adding some builder interface for creating and introspecting iterator
> > stacks
> > - Clients being able to scan to specific keys using the scan command
>

On Tue, Jan 29, 2013 at 12:34 PM, Christopher <[EMAIL PROTECTED]> wrote:
> Features I'd like to see:
>
> 1. Better user experience for configuration: separate, *non-xml*
> configuration files for separate services, and for clients.
I assume you mean tserver, master, gc etc. when you say separate services?

> 2. Separate packaging for separate services (tservers, monitor, master,
> trace), especially for RPMs and DEBs to make provisioning easier.
> 3. A suite of command-line utilities based on the shell's commands, that
> work in Bash, rather than be restricted to the limitations of the shell to
> issue these commands.
> 4. Provide a script maintained and supported by the community to provision
> an Accumulo cloud instance in EC2 or OpenStack.
>
> --
> Christopher L Tubbs II
> http://gravatar.com/ctubbsii
>
>
> On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>
>> On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
>> <[EMAIL PROTECTED]> wrote:
>> > I'd like to see:
>> >
>> > - Data triggers on insertion
>> > - REST interface for looking up ranges of keys
>> > - A DSL or some other interpreted language for crafting iterators
>> >   - there's the clojure iterator, but something like python (via jython)
>> or
>> > javascript (via rhino) would be more adoptable
>> > - Adding a clean up hook to iterators
>>
>> I was thinking about this.   If we added a close() method to the SKVI
>> interface then it would break existing iterators.  Another option
>> would be to support closing iterators that implement Closeable.  So if
>> in iterators is an intstanceof Closeable then the framework could
>> close it when its finished with the iterator.   I wish there had been
>> a 1.5 ticket for this, I think it would have been fairly simple to
>> implement.
>>
>> > - Allowing iterators to launch connections to other services (caching,
>> > other tservers) to retrieve or write data
>> > - Merging of the batch scanner and scanner implementations
>> >   - a batch scanner with 1 thread have the same behavior as a scanner
>> >   - scanners have a close() method on them
>> > - Adding some builder interface for creating and introspecting iterator
>> > stacks
>> > - Clients being able to scan to specific keys using the scan command
>>

Yes.

--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Tue, Jan 29, 2013 at 12:38 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> On Tue, Jan 29, 2013 at 12:34 PM, Christopher <[EMAIL PROTECTED]> wrote:
> > Features I'd like to see:
> >
> > 1. Better user experience for configuration: separate, *non-xml*
> > configuration files for separate services, and for clients.
> I assume you mean tserver, master, gc etc. when you say separate services?
>
> > 2. Separate packaging for separate services (tservers, monitor, master,
> > trace), especially for RPMs and DEBs to make provisioning easier.
> > 3. A suite of command-line utilities based on the shell's commands, that
> > work in Bash, rather than be restricted to the limitations of the shell
> to
> > issue these commands.
> > 4. Provide a script maintained and supported by the community to
> provision
> > an Accumulo cloud instance in EC2 or OpenStack.
> >
> > --
> > Christopher L Tubbs II
> > http://gravatar.com/ctubbsii
> >
> >
> > On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
> >
> >> On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
> >> <[EMAIL PROTECTED]> wrote:
> >> > I'd like to see:
> >> >
> >> > - Data triggers on insertion
> >> > - REST interface for looking up ranges of keys
> >> > - A DSL or some other interpreted language for crafting iterators
> >> >   - there's the clojure iterator, but something like python (via
> jython)
> >> or
> >> > javascript (via rhino) would be more adoptable
> >> > - Adding a clean up hook to iterators
> >>
> >> I was thinking about this.   If we added a close() method to the SKVI
> >> interface then it would break existing iterators.  Another option
> >> would be to support closing iterators that implement Closeable.  So if
> >> in iterators is an intstanceof Closeable then the framework could
> >> close it when its finished with the iterator.   I wish there had been
> >> a 1.5 ticket for this, I think it would have been fairly simple to
> >> implement.
> >>
> >> > - Allowing iterators to launch connections to other services (caching,
> >> > other tservers) to retrieve or write data
> >> > - Merging of the batch scanner and scanner implementations
> >> >   - a batch scanner with 1 thread have the same behavior as a scanner
> >> >   - scanners have a close() method on them
> >> > - Adding some builder interface for creating and introspecting
> iterator
> >> > stacks
> >> > - Clients being able to scan to specific keys using the scan command
> >>
>

On Tue, Jan 29, 2013 at 12:34 PM, Christopher <[EMAIL PROTECTED]> wrote:
> 3. A suite of command-line utilities based on the shell's commands, that
> work in Bash, rather than be restricted to the limitations of the shell to
> issue these commands.

Can we send a set of command to the Accumulo shell via stdin?

Yes, that's already possible... but it's very messy.
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Tue, Jan 29, 2013 at 1:21 PM, David Medinets <[EMAIL PROTECTED]>wrote:

> On Tue, Jan 29, 2013 at 12:34 PM, Christopher <[EMAIL PROTECTED]> wrote:
> > 3. A suite of command-line utilities based on the shell's commands, that
> > work in Bash, rather than be restricted to the limitations of the shell
> to
> > issue these commands.
>
> Can we send a set of command to the Accumulo shell via stdin?
>

I gave this a bit of thought too, and I think the easiest thing is to break
the interface and wrap all instances of non-closable iterators in a
closable one. That way we can delegate close down to the sources like
deepCopy does. I think Josh created a ticket for this; if not I will so we
don't derail this.

Also, in regards to the doodle thing, are we trying to set up like a cam
show or something? Personally I don't see the issue with us just listing
stuff here and having a discussion about it.

On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
> <[EMAIL PROTECTED]> wrote:
> > I'd like to see:
> >
> > - Data triggers on insertion
> > - REST interface for looking up ranges of keys
> > - A DSL or some other interpreted language for crafting iterators
> >   - there's the clojure iterator, but something like python (via jython)
> or
> > javascript (via rhino) would be more adoptable
> > - Adding a clean up hook to iterators
>
> I was thinking about this.   If we added a close() method to the SKVI
> interface then it would break existing iterators.  Another option
> would be to support closing iterators that implement Closeable.  So if
> in iterators is an intstanceof Closeable then the framework could
> close it when its finished with the iterator.   I wish there had been
> a 1.5 ticket for this, I think it would have been fairly simple to
> implement.
>
> > - Allowing iterators to launch connections to other services (caching,
> > other tservers) to retrieve or write data
> > - Merging of the batch scanner and scanner implementations
> >   - a batch scanner with 1 thread have the same behavior as a scanner
> >   - scanners have a close() method on them
> > - Adding some builder interface for creating and introspecting iterator
> > stacks
> > - Clients being able to scan to specific keys using the scan command
>

Warning about Closable with Java 7 try-with-resources....

----- Original Message -----
From: "William Slacum" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, January 29, 2013 12:40:00 PM
Subject: Re: Accumulo 1.6 and beyond feature summit

I gave this a bit of thought too, and I think the easiest thing is to break
the interface and wrap all instances of non-closable iterators in a
closable one. That way we can delegate close down to the sources like
deepCopy does. I think Josh created a ticket for this; if not I will so we
don't derail this.

Also, in regards to the doodle thing, are we trying to set up like a cam
show or something? Personally I don't see the issue with us just listing
stuff here and having a discussion about it.

On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
> <[EMAIL PROTECTED]> wrote:
> > I'd like to see:
> >
> > - Data triggers on insertion
> > - REST interface for looking up ranges of keys
> > - A DSL or some other interpreted language for crafting iterators
> >   - there's the clojure iterator, but something like python (via jython)
> or
> > javascript (via rhino) would be more adoptable
> > - Adding a clean up hook to iterators
>
> I was thinking about this.   If we added a close() method to the SKVI
> interface then it would break existing iterators.  Another option
> would be to support closing iterators that implement Closeable.  So if
> in iterators is an intstanceof Closeable then the framework could
> close it when its finished with the iterator.   I wish there had been
> a 1.5 ticket for this, I think it would have been fairly simple to
> implement.
>
> > - Allowing iterators to launch connections to other services (caching,
> > other tservers) to retrieve or write data
> > - Merging of the batch scanner and scanner implementations
> >   - a batch scanner with 1 thread have the same behavior as a scanner
> >   - scanners have a close() method on them
> > - Adding some builder interface for creating and introspecting iterator
> > stacks
> > - Clients being able to scan to specific keys using the scan command
>

Features I'd like to see-
SSL
Encryption at rest
locality-group level configurations
Either migrating our monitor dependency to another project or making the
monitor support real time configurations
read-only tablet mirroring
a configuration script to generate appropriate sized files based on general
memory footprint, and gathered information (# disks, # cores, read vs.
write vs. balanced performance)

more comments-
Packaging is something that I'm currently working on integrating into
Bigtop, with some help from those folks.
We're still java 6 based, not 7. I don't think we should concern ourselves
with java 7 discrepencies unless we switch to java 7.

On Tue, Jan 29, 2013 at 12:52 PM, <[EMAIL PROTECTED]> wrote:

> Warning about Closable with Java 7 try-with-resources....
>
> ----- Original Message -----
> From: "William Slacum" <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Tuesday, January 29, 2013 12:40:00 PM
> Subject: Re: Accumulo 1.6 and beyond feature summit
>
> I gave this a bit of thought too, and I think the easiest thing is to break
> the interface and wrap all instances of non-closable iterators in a
> closable one. That way we can delegate close down to the sources like
> deepCopy does. I think Josh created a ticket for this; if not I will so we
> don't derail this.
>
> Also, in regards to the doodle thing, are we trying to set up like a cam
> show or something? Personally I don't see the issue with us just listing
> stuff here and having a discussion about it.
>
> On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>
> > On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
> > <[EMAIL PROTECTED]> wrote:
> > > I'd like to see:
> > >
> > > - Data triggers on insertion
> > > - REST interface for looking up ranges of keys
> > > - A DSL or some other interpreted language for crafting iterators
> > >   - there's the clojure iterator, but something like python (via
> jython)
> > or
> > > javascript (via rhino) would be more adoptable
> > > - Adding a clean up hook to iterators
> >
> > I was thinking about this.   If we added a close() method to the SKVI
> > interface then it would break existing iterators.  Another option
> > would be to support closing iterators that implement Closeable.  So if
> > in iterators is an intstanceof Closeable then the framework could
> > close it when its finished with the iterator.   I wish there had been
> > a 1.5 ticket for this, I think it would have been fairly simple to
> > implement.
> >
> > > - Allowing iterators to launch connections to other services (caching,
> > > other tservers) to retrieve or write data
> > > - Merging of the batch scanner and scanner implementations
> > >   - a batch scanner with 1 thread have the same behavior as a scanner
> > >   - scanners have a close() method on them
> > > - Adding some builder interface for creating and introspecting iterator
> > > stacks
> > > - Clients being able to scan to specific keys using the scan command
> >
>

Java 6 End of Public Updates is slated for Feb 2013 [1], so I'm sure there
will be a lot of interest in being Java 7 compatible very soon.

+1 for Java 7, bash commands, and Percolator

[1]: http://www.oracle.com/technetwork/java/eol-135779.html
On Tue, Jan 29, 2013 at 1:11 PM, John Vines <[EMAIL PROTECTED]> wrote:

> Features I'd like to see-
> SSL
> Encryption at rest
> locality-group level configurations
> Either migrating our monitor dependency to another project or making the
> monitor support real time configurations
> read-only tablet mirroring
> a configuration script to generate appropriate sized files based on general
> memory footprint, and gathered information (# disks, # cores, read vs.
> write vs. balanced performance)
>
> more comments-
> Packaging is something that I'm currently working on integrating into
> Bigtop, with some help from those folks.
> We're still java 6 based, not 7. I don't think we should concern ourselves
> with java 7 discrepencies unless we switch to java 7.
>
>
>
> On Tue, Jan 29, 2013 at 12:52 PM, <[EMAIL PROTECTED]> wrote:
>
> > Warning about Closable with Java 7 try-with-resources....
> >
> > ----- Original Message -----
> > From: "William Slacum" <[EMAIL PROTECTED]>
> > To: [EMAIL PROTECTED]
> > Sent: Tuesday, January 29, 2013 12:40:00 PM
> > Subject: Re: Accumulo 1.6 and beyond feature summit
> >
> > I gave this a bit of thought too, and I think the easiest thing is to
> break
> > the interface and wrap all instances of non-closable iterators in a
> > closable one. That way we can delegate close down to the sources like
> > deepCopy does. I think Josh created a ticket for this; if not I will so
> we
> > don't derail this.
> >
> > Also, in regards to the doodle thing, are we trying to set up like a cam
> > show or something? Personally I don't see the issue with us just listing
> > stuff here and having a discussion about it.
> >
> > On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
> >
> > > On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
> > > <[EMAIL PROTECTED]> wrote:
> > > > I'd like to see:
> > > >
> > > > - Data triggers on insertion
> > > > - REST interface for looking up ranges of keys
> > > > - A DSL or some other interpreted language for crafting iterators
> > > >   - there's the clojure iterator, but something like python (via
> > jython)
> > > or
> > > > javascript (via rhino) would be more adoptable
> > > > - Adding a clean up hook to iterators
> > >
> > > I was thinking about this.   If we added a close() method to the SKVI
> > > interface then it would break existing iterators.  Another option
> > > would be to support closing iterators that implement Closeable.  So if
> > > in iterators is an intstanceof Closeable then the framework could
> > > close it when its finished with the iterator.   I wish there had been
> > > a 1.5 ticket for this, I think it would have been fairly simple to
> > > implement.
> > >
> > > > - Allowing iterators to launch connections to other services
> (caching,
> > > > other tservers) to retrieve or write data
> > > > - Merging of the batch scanner and scanner implementations
> > > >   - a batch scanner with 1 thread have the same behavior as a scanner
> > > >   - scanners have a close() method on them
> > > > - Adding some builder interface for creating and introspecting
> iterator
> > > > stacks
> > > > - Clients being able to scan to specific keys using the scan command
> > >
> >
>

On Tue, Jan 29, 2013 at 12:40 PM, William Slacum <
[EMAIL PROTECTED]> wrote:

> Also, in regards to the doodle thing, are we trying to set up like a cam
> show or something? Personally I don't see the issue with us just listing
> stuff here and having a discussion about it.
>
> I would like to have a live meeting is to support more interactive
communication with a more complete scope of features in one discussion. You
can think of this as a sort of mini-conference for Accumulo features. This
is an opportunity to get people excited about the projects that you are
interested in as well as to establish some big picture identity for the
next releases of Accumulo.

Adam

Can we use the Accumulo wiki to kind of collect these feature requests into a place where they are easier to see, avoid duplication, and identify overlaps / convergent feature requests?
On Jan 30, 2013, at 12:22 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:

> On Tue, Jan 29, 2013 at 12:40 PM, William Slacum <
> [EMAIL PROTECTED]> wrote:
>
>> Also, in regards to the doodle thing, are we trying to set up like a cam
>> show or something? Personally I don't see the issue with us just listing
>> stuff here and having a discussion about it.
>>
>> I would like to have a live meeting is to support more interactive
> communication with a more complete scope of features in one discussion. You
> can think of this as a sort of mini-conference for Accumulo features. This
> is an opportunity to get people excited about the projects that you are
> interested in as well as to establish some big picture identity for the
> next releases of Accumulo.
>
> Adam

On Wed, Jan 30, 2013 at 1:20 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
> Can we use the Accumulo wiki to kind of collect these feature requests into a place where they are easier to see, avoid duplication, and identify overlaps / convergent feature requests?

Why not use jira?   I would encourage people to create tickets for
things they want to see in 1.6

>
>
> On Jan 30, 2013, at 12:22 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>
>> On Tue, Jan 29, 2013 at 12:40 PM, William Slacum <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Also, in regards to the doodle thing, are we trying to set up like a cam
>>> show or something? Personally I don't see the issue with us just listing
>>> stuff here and having a discussion about it.
>>>
>>> I would like to have a live meeting is to support more interactive
>> communication with a more complete scope of features in one discussion. You
>> can think of this as a sort of mini-conference for Accumulo features. This
>> is an opportunity to get people excited about the projects that you are
>> interested in as well as to establish some big picture identity for the
>> next releases of Accumulo.
>>
>> Adam
>

That works too. It'd be much better than distributing the features throughout the dev list amidst lots of text about where to put the list of features.

On Jan 30, 2013, at 1:28 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> On Wed, Jan 30, 2013 at 1:20 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>> Can we use the Accumulo wiki to kind of collect these feature requests into a place where they are easier to see, avoid duplication, and identify overlaps / convergent feature requests?
>
> Why not use jira?   I would encourage people to create tickets for
> things they want to see in 1.6
>
>>
>>
>> On Jan 30, 2013, at 12:22 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>>
>>> On Tue, Jan 29, 2013 at 12:40 PM, William Slacum <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Also, in regards to the doodle thing, are we trying to set up like a cam
>>>> show or something? Personally I don't see the issue with us just listing
>>>> stuff here and having a discussion about it.
>>>>
>>>> I would like to have a live meeting is to support more interactive
>>> communication with a more complete scope of features in one discussion. You
>>> can think of this as a sort of mini-conference for Accumulo features. This
>>> is an opportunity to get people excited about the projects that you are
>>> interested in as well as to establish some big picture identity for the
>>> next releases of Accumulo.
>>>
>>> Adam
>>

On Wed, Jan 30, 2013 at 1:30 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
> That works too. It'd be much better than distributing the features throughout the dev list amidst lots of text about where to put the list of features.

Agreed.  Also, if you create new tickets for things you want in 1.6
then please mark tickets as fix for 1.6 so we can easily find them
with a query.

>
> On Jan 30, 2013, at 1:28 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>
>> On Wed, Jan 30, 2013 at 1:20 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>>> Can we use the Accumulo wiki to kind of collect these feature requests into a place where they are easier to see, avoid duplication, and identify overlaps / convergent feature requests?
>>
>> Why not use jira?   I would encourage people to create tickets for
>> things they want to see in 1.6
>>
>>>
>>>
>>> On Jan 30, 2013, at 12:22 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>>>
>>>> On Tue, Jan 29, 2013 at 12:40 PM, William Slacum <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> Also, in regards to the doodle thing, are we trying to set up like a cam
>>>>> show or something? Personally I don't see the issue with us just listing
>>>>> stuff here and having a discussion about it.
>>>>>
>>>>> I would like to have a live meeting is to support more interactive
>>>> communication with a more complete scope of features in one discussion. You
>>>> can think of this as a sort of mini-conference for Accumulo features. This
>>>> is an opportunity to get people excited about the projects that you are
>>>> interested in as well as to establish some big picture identity for the
>>>> next releases of Accumulo.
>>>>
>>>> Adam
>>>
>

+1 for creating feature tickets marked as fix for 1.6.0 for
requested/desired features.
--
Christopher L Tubbs II
http://gravatar.com/ctubbsii
On Wed, Jan 30, 2013 at 1:36 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> On Wed, Jan 30, 2013 at 1:30 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
> > That works too. It'd be much better than distributing the features
> throughout the dev list amidst lots of text about where to put the list of
> features.
>
> Agreed.  Also, if you create new tickets for things you want in 1.6
> then please mark tickets as fix for 1.6 so we can easily find them
> with a query.
>
> >
> > On Jan 30, 2013, at 1:28 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
> >
> >> On Wed, Jan 30, 2013 at 1:20 PM, Aaron Cordova <[EMAIL PROTECTED]>
> wrote:
> >>> Can we use the Accumulo wiki to kind of collect these feature requests
> into a place where they are easier to see, avoid duplication, and identify
> overlaps / convergent feature requests?
> >>
> >> Why not use jira?   I would encourage people to create tickets for
> >> things they want to see in 1.6
> >>
> >>>
> >>>
> >>> On Jan 30, 2013, at 12:22 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
> >>>
> >>>> On Tue, Jan 29, 2013 at 12:40 PM, William Slacum <
> >>>> [EMAIL PROTECTED]> wrote:
> >>>>
> >>>>> Also, in regards to the doodle thing, are we trying to set up like a
> cam
> >>>>> show or something? Personally I don't see the issue with us just
> listing
> >>>>> stuff here and having a discussion about it.
> >>>>>
> >>>>> I would like to have a live meeting is to support more interactive
> >>>> communication with a more complete scope of features in one
> discussion. You
> >>>> can think of this as a sort of mini-conference for Accumulo features.
> This
> >>>> is an opportunity to get people excited about the projects that you
> are
> >>>> interested in as well as to establish some big picture identity for
> the
> >>>> next releases of Accumulo.
> >>>>
> >>>> Adam
> >>>
> >
>

I'd like to see ACCUMULO-118 completed for 1.6

----- Original Message -----
From: "William Slacum" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, January 29, 2013 12:40:00 PM
Subject: Re: Accumulo 1.6 and beyond feature summit

I gave this a bit of thought too, and I think the easiest thing is to break
the interface and wrap all instances of non-closable iterators in a
closable one. That way we can delegate close down to the sources like
deepCopy does. I think Josh created a ticket for this; if not I will so we
don't derail this.

Also, in regards to the doodle thing, are we trying to set up like a cam
show or something? Personally I don't see the issue with us just listing
stuff here and having a discussion about it.

On Tue, Jan 29, 2013 at 12:15 PM, Keith Turner <[EMAIL PROTECTED]> wrote:

> On Mon, Jan 28, 2013 at 7:12 PM, William Slacum
> <[EMAIL PROTECTED]> wrote:
> > I'd like to see:
> >
> > - Data triggers on insertion
> > - REST interface for looking up ranges of keys
> > - A DSL or some other interpreted language for crafting iterators
> >   - there's the clojure iterator, but something like python (via jython)
> or
> > javascript (via rhino) would be more adoptable
> > - Adding a clean up hook to iterators
>
> I was thinking about this.   If we added a close() method to the SKVI
> interface then it would break existing iterators.  Another option
> would be to support closing iterators that implement Closeable.  So if
> in iterators is an intstanceof Closeable then the framework could
> close it when its finished with the iterator.   I wish there had been
> a 1.5 ticket for this, I think it would have been fairly simple to
> implement.
>
> > - Allowing iterators to launch connections to other services (caching,
> > other tservers) to retrieve or write data
> > - Merging of the batch scanner and scanner implementations
> >   - a batch scanner with 1 thread have the same behavior as a scanner
> >   - scanners have a close() method on them
> > - Adding some builder interface for creating and introspecting iterator
> > stacks
> > - Clients being able to scan to specific keys using the scan command
>

I think shifting all monitoring tools to something like Ambari would be the
best route to go. It offloads a majority of our development onto another
project while still providing monitoring, but also provided a command and
control interface.
On Mon, Jan 28, 2013 at 12:03 PM, Josh Elser <[EMAIL PROTECTED]> wrote:

> A big concern with the monitor has always been data spillage due to tablet
> split points being only base64 encoded.
>
> Perhaps wrapping the currents calls with some authentication would make the
> current monitor more extendable in addition to the ease of other monitor
> processes being created.
>
> Regardless, yes to think about the monitor :)
>
> On Monday, January 28, 2013, Jim Klucar <[EMAIL PROTECTED]> wrote:
> > I think new monitor features could use a whole separate discussion. Being
> > able to do everything from the browser implies an officially supported
> > webservice of sorts.
> >
> > Seems like monitor upgrades should be on the list though.
> >
> >
> > On Mon, Jan 28, 2013 at 10:46 AM, David Medinets
> > <[EMAIL PROTECTED]>wrote:
> >
> >> How about letting users see and restart processes from the Monitor page?
> >> Splitting on tablet record count as well as tablet size?
> >>
> >
>

I'd argue refactoring the monitor to fit inside of Ambari, in addition
to whatever else, would be the ideal way to go. But that's why we can
"summit" together :P

On 01/28/2013 12:20 PM, John Vines wrote:
> I think shifting all monitoring tools to something like Ambari would be the
> best route to go. It offloads a majority of our development onto another
> project while still providing monitoring, but also provided a command and
> control interface.
>
>
> On Mon, Jan 28, 2013 at 12:03 PM, Josh Elser <[EMAIL PROTECTED]> wrote:
>
>> A big concern with the monitor has always been data spillage due to tablet
>> split points being only base64 encoded.
>>
>> Perhaps wrapping the currents calls with some authentication would make the
>> current monitor more extendable in addition to the ease of other monitor
>> processes being created.
>>
>> Regardless, yes to think about the monitor :)
>>
>> On Monday, January 28, 2013, Jim Klucar <[EMAIL PROTECTED]> wrote:
>>> I think new monitor features could use a whole separate discussion. Being
>>> able to do everything from the browser implies an officially supported
>>> webservice of sorts.
>>>
>>> Seems like monitor upgrades should be on the list though.
>>>
>>>
>>> On Mon, Jan 28, 2013 at 10:46 AM, David Medinets
>>> <[EMAIL PROTECTED]>wrote:
>>>
>>>> How about letting users see and restart processes from the Monitor page?
>>>> Splitting on tablet record count as well as tablet size?
>>>>

On Sun, Jan 27, 2013 at 12:43 PM, Eric Newton <[EMAIL PROTECTED]> wrote:

> What is CAS?
>
>    - Switch to Curator.
>    - Monitoring.  I'm hoping this will just be integration with an existing
>    tool.
>    - Namespaces.
>    - A simple query language.  Perhaps there's something we can collaborate
>    with some of the other NoSQL folks.
>    - Update the wiki search example.
>    - Automatic tablet merge.
>    - Scalable NN
>

Is scalable NN == ACCUMULO-722?  I'd be excited about that for 1.6.

Billie

>
> Some of these don't need design documents, but some discussion of
> requirements would be nice.
>
> -Eric
>
>
>
> On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>
> > This will be very useful.
> >
> > Some new features I am thinking about for 1.6 are replication, CAS,
> > and Percolator.  I would like to put together a bit of an initial
> > design document for each of these.  I'm thinking that a follow up
> > online deep dive on the design of each large new 1.6 feature would
> > also be very useful.
> >
> > In the coming week I plan to spend most of my time testing,
> > documenting, fixing bugs, and polishing 1.5.0.  I do not think this
> > would leave much time to work on 1.6.  So interleaving design
> > discussion about new 1.6 features while 1.5 is being polished would be
> > nice.  So once 1.5.0 ships, I would have peer reviewed designs to
> > start implementing.
> >
> > Keith
> >
> > On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
> > > Folks,
> > >
> > > Now that we're past feature freeze for 1.5, I'd like to set up a
> > discussion
> > > of the features that Accumulo developers are interested in for 1.6 and
> > > beyond. The purpose of this discussion will be to raise awareness of
> > > projects that you think are cool and important, and to rally the
> > community.
> > > As a proposed format, let's plan on a series of 2-5 minute
> presentations
> > of
> > > individual features -- what it is, why it's important, how should we
> > build
> > > it, and who needs to help. We'll record the presentations for
> posterity.
> > >
> > > Since the community is now distributed throughout several states, let's
> > try
> > > our hand at an online discussion. Depending on participation, we'll
> pick
> > a
> > > forum that can accomodate everyone. If you're interested in attending,
> > > please fill out this doodle with what time you would be available:
> > > http://www.doodle.com/w9zb77ya3ykxr4zv
> > >
> > > If you can, please try to fill out the survey by Tuesday afternoon.
> After
> > > we get a good feel for times and participation, I'll schedule an online
> > > forum and send out a meeting invitation. If you would like to present a
> > > feature, send me an email directly at [EMAIL PROTECTED], and I'll make
> > sure
> > > you get time to present it.
> > >
> > > I look forward to seeing what everyone has up their sleeves!
> > >
> > > Cheers,
> > > Adam
> >
>

I want to caution everyone that this process skates close to an edge
of an Apache invariant. An online forum has the potential to exclude
people in far-away time zones or with other constraints. At Apache, we
try to maximize the use of email to avoid this problem.

Nothing here is an absolute -- I'm not trying to torpedo this plan altogether.

Nothing, that is, except except that _decisions_ must be made on the
mailing list afterwards.

If anyone in the dev community feels excluded by this prospect, I
would encourage you to speak up now, so that the planners can make
adjustments to minimize that.
On Thu, Jan 31, 2013 at 10:10 PM, Billie Rinaldi
<[EMAIL PROTECTED]> wrote:
> On Sun, Jan 27, 2013 at 12:43 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
>
>> What is CAS?
>>
>>    - Switch to Curator.
>>    - Monitoring.  I'm hoping this will just be integration with an existing
>>    tool.
>>    - Namespaces.
>>    - A simple query language.  Perhaps there's something we can collaborate
>>    with some of the other NoSQL folks.
>>    - Update the wiki search example.
>>    - Automatic tablet merge.
>>    - Scalable NN
>>
>
> Is scalable NN == ACCUMULO-722?  I'd be excited about that for 1.6.
>
> Billie
>
>
>
>>
>> Some of these don't need design documents, but some discussion of
>> requirements would be nice.
>>
>> -Eric
>>
>>
>>
>> On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>>
>> > This will be very useful.
>> >
>> > Some new features I am thinking about for 1.6 are replication, CAS,
>> > and Percolator.  I would like to put together a bit of an initial
>> > design document for each of these.  I'm thinking that a follow up
>> > online deep dive on the design of each large new 1.6 feature would
>> > also be very useful.
>> >
>> > In the coming week I plan to spend most of my time testing,
>> > documenting, fixing bugs, and polishing 1.5.0.  I do not think this
>> > would leave much time to work on 1.6.  So interleaving design
>> > discussion about new 1.6 features while 1.5 is being polished would be
>> > nice.  So once 1.5.0 ships, I would have peer reviewed designs to
>> > start implementing.
>> >
>> > Keith
>> >
>> > On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>> > > Folks,
>> > >
>> > > Now that we're past feature freeze for 1.5, I'd like to set up a
>> > discussion
>> > > of the features that Accumulo developers are interested in for 1.6 and
>> > > beyond. The purpose of this discussion will be to raise awareness of
>> > > projects that you think are cool and important, and to rally the
>> > community.
>> > > As a proposed format, let's plan on a series of 2-5 minute
>> presentations
>> > of
>> > > individual features -- what it is, why it's important, how should we
>> > build
>> > > it, and who needs to help. We'll record the presentations for
>> posterity.
>> > >
>> > > Since the community is now distributed throughout several states, let's
>> > try
>> > > our hand at an online discussion. Depending on participation, we'll
>> pick
>> > a
>> > > forum that can accomodate everyone. If you're interested in attending,
>> > > please fill out this doodle with what time you would be available:
>> > > http://www.doodle.com/w9zb77ya3ykxr4zv
>> > >
>> > > If you can, please try to fill out the survey by Tuesday afternoon.
>> After
>> > > we get a good feel for times and participation, I'll schedule an online
>> > > forum and send out a meeting invitation. If you would like to present a
>> > > feature, send me an email directly at [EMAIL PROTECTED], and I'll make
>> > sure
>> > > you get time to present it.
>> > >
>> > > I look forward to seeing what everyone has up their sleeves!
>> > >
>> > > Cheers,
>> > > Adam
>> >
>>

Benson,

Just to be clear (I had to re-read your email a few times to understand) you're saying a real-time online discussion such as Adam is proposing has the potential to exclude some members of the dev community and that we should be sure to stick to things like this email discussion to avoid those problems?
Thanks,

Aaron
On Feb 1, 2013, at 2:36 AM, Benson Margulies <[EMAIL PROTECTED]> wrote:

> I want to caution everyone that this process skates close to an edge
> of an Apache invariant. An online forum has the potential to exclude
> people in far-away time zones or with other constraints. At Apache, we
> try to maximize the use of email to avoid this problem.
>
> Nothing here is an absolute -- I'm not trying to torpedo this plan altogether.
>
> Nothing, that is, except except that _decisions_ must be made on the
> mailing list afterwards.
>
> If anyone in the dev community feels excluded by this prospect, I
> would encourage you to speak up now, so that the planners can make
> adjustments to minimize that.
>
>
> On Thu, Jan 31, 2013 at 10:10 PM, Billie Rinaldi
> <[EMAIL PROTECTED]> wrote:
>> On Sun, Jan 27, 2013 at 12:43 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
>>
>>> What is CAS?
>>>
>>>   - Switch to Curator.
>>>   - Monitoring.  I'm hoping this will just be integration with an existing
>>>   tool.
>>>   - Namespaces.
>>>   - A simple query language.  Perhaps there's something we can collaborate
>>>   with some of the other NoSQL folks.
>>>   - Update the wiki search example.
>>>   - Automatic tablet merge.
>>>   - Scalable NN
>>>
>>
>> Is scalable NN == ACCUMULO-722?  I'd be excited about that for 1.6.
>>
>> Billie
>>
>>
>>
>>>
>>> Some of these don't need design documents, but some discussion of
>>> requirements would be nice.
>>>
>>> -Eric
>>>
>>>
>>>
>>> On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>>>
>>>> This will be very useful.
>>>>
>>>> Some new features I am thinking about for 1.6 are replication, CAS,
>>>> and Percolator.  I would like to put together a bit of an initial
>>>> design document for each of these.  I'm thinking that a follow up
>>>> online deep dive on the design of each large new 1.6 feature would
>>>> also be very useful.
>>>>
>>>> In the coming week I plan to spend most of my time testing,
>>>> documenting, fixing bugs, and polishing 1.5.0.  I do not think this
>>>> would leave much time to work on 1.6.  So interleaving design
>>>> discussion about new 1.6 features while 1.5 is being polished would be
>>>> nice.  So once 1.5.0 ships, I would have peer reviewed designs to
>>>> start implementing.
>>>>
>>>> Keith
>>>>
>>>> On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>>>>> Folks,
>>>>>
>>>>> Now that we're past feature freeze for 1.5, I'd like to set up a
>>>> discussion
>>>>> of the features that Accumulo developers are interested in for 1.6 and
>>>>> beyond. The purpose of this discussion will be to raise awareness of
>>>>> projects that you think are cool and important, and to rally the
>>>> community.
>>>>> As a proposed format, let's plan on a series of 2-5 minute
>>> presentations
>>>> of
>>>>> individual features -- what it is, why it's important, how should we
>>>> build
>>>>> it, and who needs to help. We'll record the presentations for
>>> posterity.
>>>>>
>>>>> Since the community is now distributed throughout several states, let's
>>>> try
>>>>> our hand at an online discussion. Depending on participation, we'll
>>> pick
>>>> a
>>>>> forum that can accomodate everyone. If you're interested in attending,
>>>>> please fill out this doodle with what time you would be available:
>>>>> http://www.doodle.com/w9zb77ya3ykxr4zv
>>>>>
>>>>> If you can, please try to fill out the survey by Tuesday afternoon.
>>> After
>>>>> we get a good feel for times and participation, I'll schedule an online
>>>>> forum and send out a meeting invitation. If you would like to present a
>>>>> feature, send me an email directly at [EMAIL PROTECTED], and I'll make

On Fri, Feb 1, 2013 at 3:04 PM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
> Benson,
>
> Just to be clear (I had to re-read your email a few times to understand) you're saying a real-time online discussion such as Adam is proposing has the potential to exclude some members of the dev community and that we should be sure to stick to things like this email discussion to avoid those problems?

Small adjustment: email avoids these problems, but I'm not telling you
that online is 'forbidden'. Just that, as a community, you should be
careful to avoid excluding people. One schema: publish minutes, and be
sure that you don't consider a decision final until it's ratified
on-list.
>
>
> Thanks,
>
> Aaron
>
>
> On Feb 1, 2013, at 2:36 AM, Benson Margulies <[EMAIL PROTECTED]> wrote:
>
>> I want to caution everyone that this process skates close to an edge
>> of an Apache invariant. An online forum has the potential to exclude
>> people in far-away time zones or with other constraints. At Apache, we
>> try to maximize the use of email to avoid this problem.
>>
>> Nothing here is an absolute -- I'm not trying to torpedo this plan altogether.
>>
>> Nothing, that is, except except that _decisions_ must be made on the
>> mailing list afterwards.
>>
>> If anyone in the dev community feels excluded by this prospect, I
>> would encourage you to speak up now, so that the planners can make
>> adjustments to minimize that.
>>
>>
>> On Thu, Jan 31, 2013 at 10:10 PM, Billie Rinaldi
>> <[EMAIL PROTECTED]> wrote:
>>> On Sun, Jan 27, 2013 at 12:43 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
>>>
>>>> What is CAS?
>>>>
>>>>   - Switch to Curator.
>>>>   - Monitoring.  I'm hoping this will just be integration with an existing
>>>>   tool.
>>>>   - Namespaces.
>>>>   - A simple query language.  Perhaps there's something we can collaborate
>>>>   with some of the other NoSQL folks.
>>>>   - Update the wiki search example.
>>>>   - Automatic tablet merge.
>>>>   - Scalable NN
>>>>
>>>
>>> Is scalable NN == ACCUMULO-722?  I'd be excited about that for 1.6.
>>>
>>> Billie
>>>
>>>
>>>
>>>>
>>>> Some of these don't need design documents, but some discussion of
>>>> requirements would be nice.
>>>>
>>>> -Eric
>>>>
>>>>
>>>>
>>>> On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> This will be very useful.
>>>>>
>>>>> Some new features I am thinking about for 1.6 are replication, CAS,
>>>>> and Percolator.  I would like to put together a bit of an initial
>>>>> design document for each of these.  I'm thinking that a follow up
>>>>> online deep dive on the design of each large new 1.6 feature would
>>>>> also be very useful.
>>>>>
>>>>> In the coming week I plan to spend most of my time testing,
>>>>> documenting, fixing bugs, and polishing 1.5.0.  I do not think this
>>>>> would leave much time to work on 1.6.  So interleaving design
>>>>> discussion about new 1.6 features while 1.5 is being polished would be
>>>>> nice.  So once 1.5.0 ships, I would have peer reviewed designs to
>>>>> start implementing.
>>>>>
>>>>> Keith
>>>>>
>>>>> On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>>>>>> Folks,
>>>>>>
>>>>>> Now that we're past feature freeze for 1.5, I'd like to set up a
>>>>> discussion
>>>>>> of the features that Accumulo developers are interested in for 1.6 and
>>>>>> beyond. The purpose of this discussion will be to raise awareness of
>>>>>> projects that you think are cool and important, and to rally the
>>>>> community.
>>>>>> As a proposed format, let's plan on a series of 2-5 minute
>>>> presentations
>>>>> of
>>>>>> individual features -- what it is, why it's important, how should we
>>>>> build
>>>>>> it, and who needs to help. We'll record the presentations for
>>>> posterity.
>>>>>>
>>>>>> Since the community is now distributed throughout several states, let's
>>>>> try
>>>>>> our hand at an online discussion. Depending on participation, we'll
>>>> pick
>>>>> a
>>>>>> forum that can accomodate everyone. If you're interested in attending,

How I read Benson's comment is that if anyone feels excluded by the online
forum and voices that objection then we should not do it. This is not to
say that the community has any power to forbid communication about Accumulo
outside of email channels, but we should make things as accessible as
possible and only make official decisions via approved channels (do we have
by-laws to approve channels yet?).

Another way to think of this online forum is like a meetup. We're proposing
to do a couple of things to be as inclusive as possible -- one is to record
the presentations so that everyone can see them on their own schedule. The
other is to refrain from any official decision making at this event. This
is only intended to facilitate information exchange.

Adam

On Fri, Feb 1, 2013 at 10:04 AM, Aaron Cordova <[EMAIL PROTECTED]> wrote:

> Benson,
>
> Just to be clear (I had to re-read your email a few times to understand)
> you're saying a real-time online discussion such as Adam is proposing has
> the potential to exclude some members of the dev community and that we
> should be sure to stick to things like this email discussion to avoid those
> problems?
>
>
> Thanks,
>
> Aaron
>
>
> On Feb 1, 2013, at 2:36 AM, Benson Margulies <[EMAIL PROTECTED]>
> wrote:
>
> > I want to caution everyone that this process skates close to an edge
> > of an Apache invariant. An online forum has the potential to exclude
> > people in far-away time zones or with other constraints. At Apache, we
> > try to maximize the use of email to avoid this problem.
> >
> > Nothing here is an absolute -- I'm not trying to torpedo this plan
> altogether.
> >
> > Nothing, that is, except except that _decisions_ must be made on the
> > mailing list afterwards.
> >
> > If anyone in the dev community feels excluded by this prospect, I
> > would encourage you to speak up now, so that the planners can make
> > adjustments to minimize that.
> >
> >
> > On Thu, Jan 31, 2013 at 10:10 PM, Billie Rinaldi
> > <[EMAIL PROTECTED]> wrote:
> >> On Sun, Jan 27, 2013 at 12:43 PM, Eric Newton <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> What is CAS?
> >>>
> >>>   - Switch to Curator.
> >>>   - Monitoring.  I'm hoping this will just be integration with an
> existing
> >>>   tool.
> >>>   - Namespaces.
> >>>   - A simple query language.  Perhaps there's something we can
> collaborate
> >>>   with some of the other NoSQL folks.
> >>>   - Update the wiki search example.
> >>>   - Automatic tablet merge.
> >>>   - Scalable NN
> >>>
> >>
> >> Is scalable NN == ACCUMULO-722?  I'd be excited about that for 1.6.
> >>
> >> Billie
> >>
> >>
> >>
> >>>
> >>> Some of these don't need design documents, but some discussion of
> >>> requirements would be nice.
> >>>
> >>> -Eric
> >>>
> >>>
> >>>
> >>> On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> This will be very useful.
> >>>>
> >>>> Some new features I am thinking about for 1.6 are replication, CAS,
> >>>> and Percolator.  I would like to put together a bit of an initial
> >>>> design document for each of these.  I'm thinking that a follow up
> >>>> online deep dive on the design of each large new 1.6 feature would
> >>>> also be very useful.
> >>>>
> >>>> In the coming week I plan to spend most of my time testing,
> >>>> documenting, fixing bugs, and polishing 1.5.0.  I do not think this
> >>>> would leave much time to work on 1.6.  So interleaving design
> >>>> discussion about new 1.6 features while 1.5 is being polished would be
> >>>> nice.  So once 1.5.0 ships, I would have peer reviewed designs to
> >>>> start implementing.
> >>>>
> >>>> Keith
> >>>>
> >>>> On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]>
> wrote:
> >>>>> Folks,
> >>>>>
> >>>>> Now that we're past feature freeze for 1.5, I'd like to set up a
> >>>> discussion
> >>>>> of the features that Accumulo developers are interested in for 1.6
> and
> >>>>> beyond. The purpose of this discussion will be to raise awareness of

On Fri, Feb 1, 2013 at 7:22 AM, Adam Fuchs <[EMAIL PROTECTED]> wrote:

> How I read Benson's comment is that if anyone feels excluded by the online
> forum and voices that objection then we should not do it. This is not to
> say that the community has any power to forbid communication about Accumulo
> outside of email channels, but we should make things as accessible as
> possible and only make official decisions via approved channels (do we have
> by-laws to approve channels yet?).
>

We should work on our by-laws.  I believe the only channel we have to make
decisions is through the mailing list (and by extension JIRA, since it is
logged to the mailing list).

Billie

>
> Another way to think of this online forum is like a meetup. We're proposing
> to do a couple of things to be as inclusive as possible -- one is to record
> the presentations so that everyone can see them on their own schedule. The
> other is to refrain from any official decision making at this event. This
> is only intended to facilitate information exchange.
>
> Adam
>
>
>
> On Fri, Feb 1, 2013 at 10:04 AM, Aaron Cordova <[EMAIL PROTECTED]> wrote:
>
> > Benson,
> >
> > Just to be clear (I had to re-read your email a few times to understand)
> > you're saying a real-time online discussion such as Adam is proposing has
> > the potential to exclude some members of the dev community and that we
> > should be sure to stick to things like this email discussion to avoid
> those
> > problems?
> >
> >
> > Thanks,
> >
> > Aaron
> >
> >
> > On Feb 1, 2013, at 2:36 AM, Benson Margulies <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I want to caution everyone that this process skates close to an edge
> > > of an Apache invariant. An online forum has the potential to exclude
> > > people in far-away time zones or with other constraints. At Apache, we
> > > try to maximize the use of email to avoid this problem.
> > >
> > > Nothing here is an absolute -- I'm not trying to torpedo this plan
> > altogether.
> > >
> > > Nothing, that is, except except that _decisions_ must be made on the
> > > mailing list afterwards.
> > >
> > > If anyone in the dev community feels excluded by this prospect, I
> > > would encourage you to speak up now, so that the planners can make
> > > adjustments to minimize that.
> > >
> > >
> > > On Thu, Jan 31, 2013 at 10:10 PM, Billie Rinaldi
> > > <[EMAIL PROTECTED]> wrote:
> > >> On Sun, Jan 27, 2013 at 12:43 PM, Eric Newton <[EMAIL PROTECTED]>
> > wrote:
> > >>
> > >>> What is CAS?
> > >>>
> > >>>   - Switch to Curator.
> > >>>   - Monitoring.  I'm hoping this will just be integration with an
> > existing
> > >>>   tool.
> > >>>   - Namespaces.
> > >>>   - A simple query language.  Perhaps there's something we can
> > collaborate
> > >>>   with some of the other NoSQL folks.
> > >>>   - Update the wiki search example.
> > >>>   - Automatic tablet merge.
> > >>>   - Scalable NN
> > >>>
> > >>
> > >> Is scalable NN == ACCUMULO-722?  I'd be excited about that for 1.6.
> > >>
> > >> Billie
> > >>
> > >>
> > >>
> > >>>
> > >>> Some of these don't need design documents, but some discussion of
> > >>> requirements would be nice.
> > >>>
> > >>> -Eric
> > >>>
> > >>>
> > >>>
> > >>> On Sun, Jan 27, 2013 at 12:26 PM, Keith Turner <[EMAIL PROTECTED]>
> > wrote:
> > >>>
> > >>>> This will be very useful.
> > >>>>
> > >>>> Some new features I am thinking about for 1.6 are replication, CAS,
> > >>>> and Percolator.  I would like to put together a bit of an initial
> > >>>> design document for each of these.  I'm thinking that a follow up
> > >>>> online deep dive on the design of each large new 1.6 feature would
> > >>>> also be very useful.
> > >>>>
> > >>>> In the coming week I plan to spend most of my time testing,
> > >>>> documenting, fixing bugs, and polishing 1.5.0.  I do not think this
> > >>>> would leave much time to work on 1.6.  So interleaving design
> > >>>> discussion about new 1.6 features while 1.5 is being polished would
> be
>

As a reminder, please fill out the doodle (
http://www.doodle.com/w9zb77ya3ykxr4zv) if you want me to take your
schedule into account. We've seen a lot of great ideas on this thread so
far -- it would be a shame if Billie and I are the only ones who can attend.

Adam
On Sat, Jan 26, 2013 at 8:10 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:

> Folks,
>
> Now that we're past feature freeze for 1.5, I'd like to set up a
> discussion of the features that Accumulo developers are interested in for
> 1.6 and beyond. The purpose of this discussion will be to raise awareness
> of projects that you think are cool and important, and to rally the
> community. As a proposed format, let's plan on a series of 2-5 minute
> presentations of individual features -- what it is, why it's important, how
> should we build it, and who needs to help. We'll record the presentations
> for posterity.
>
> Since the community is now distributed throughout several states, let's
> try our hand at an online discussion. Depending on participation, we'll
> pick a forum that can accomodate everyone. If you're interested in
> attending, please fill out this doodle with what time you would be
> available: http://www.doodle.com/w9zb77ya3ykxr4zv
>
> If you can, please try to fill out the survey by Tuesday afternoon. After
> we get a good feel for times and participation, I'll schedule an online
> forum and send out a meeting invitation. If you would like to present a
> feature, send me an email directly at [EMAIL PROTECTED], and I'll make
> sure you get time to present it.
>
> I look forward to seeing what everyone has up their sleeves!
>
> Cheers,
> Adam
>
>