-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/
-----------------------------------------------------------

Review request for accumulo.
Description
-------

Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
This addresses bug ACCUMULO-482.
    https://issues.apache.org/jira/browse/ACCUMULO-482
Diffs
-----

  /trunk/pom.xml 1406772
  /trunk/proxy/README PRE-CREATION
  /trunk/proxy/examples/python/README PRE-CREATION
  /trunk/proxy/examples/python/TestClient.py PRE-CREATION
  /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
  /trunk/proxy/examples/python/data/constants.py PRE-CREATION
  /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
  /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
  /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
  /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
  /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
  /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
  /trunk/proxy/examples/ruby/README PRE-CREATION
  /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
  /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
  /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
  /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION
  /trunk/proxy/examples/ruby/proxy_types.rb PRE-CREATION
  /trunk/proxy/examples/ruby/test_client.rb PRE-CREATION
  /trunk/proxy/examples/ruby/thrift.rb PRE-CREATION
  /trunk/proxy/pom.xml PRE-CREATION
  /trunk/proxy/proxy.properties PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyHarness.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/TestProxyClient.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/Util.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloException.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloProxy.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloSecurityException.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/IOException.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/KeyValueAndPeek.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/NoMoreEntriesException.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyIteratorSetting.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxySystemPermission.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyTablePermission.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/TableExistsException.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/TableNotFoundException.java PRE-CREATION
  /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/UserPass.java PRE-CREATION
  /trunk/proxy/src/main/thrift/data.thrift PRE-CREATION
  /trunk/proxy/src/main/thrift/proxy.thrift PRE-CREATION
  /trunk/proxy/src/test/java/org/apache/accumulo/TestProxyInstanceOperations.java PRE-CREATION
  /trunk/proxy/src/test/java/org/apache/accumulo/TestProxyReadWrite.java PRE-CREATION
  /trunk/proxy/src/test/java/org/apache/accumulo/TestProxySecurityOperations.java PRE-CREATION
  /trunk/proxy/src/test/java/org/apache/accumulo/TestProxyTableOperations.java PRE-CREATION

Diff: https://reviews.apache.org/r/7936/diff/
Testing
-------
Thanks,

kturner

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13220
-----------------------------------------------------------

/trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java
<https://reviews.apache.org/r/7936/#comment28444>

    An explicit NoSuchScanner exception may be useful.

/trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java
<https://reviews.apache.org/r/7936/#comment28438>

    sync on batchScanner before checking if its null

/trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java
<https://reviews.apache.org/r/7936/#comment28443>

    In past experience with Thrift throwing a TException in server side code woudl result in client seeing nothing or seeing something like an IOException instead of a TApplicationException.   But this was a while ago, not sure what current behavior of thrift is.

/trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java
<https://reviews.apache.org/r/7936/#comment28442>

    If user does not scan until end, then will scanner always be present in scannerMap and iteratorMap?  If so this could lead to memory leaks.  I think its a commom use case that users do not read all data from a scanner.

/trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java
<https://reviews.apache.org/r/7936/#comment28445>

    would probably be more efficient to get the scanner once from the iteratorMap, rather than doing a lookup in the map for each loop iteration

/trunk/proxy/src/main/thrift/proxy.thrift
<https://reviews.apache.org/r/7936/#comment28441>

    can anything be done to make methods shorter?  Is it possible to get rid of all of the tableOperation_ and instanceOperation_ prefixes?  Does having different services (ie. AccumuloInstanceProxy, AccumloTableOpProxy) force us to run those services on separate ports?  

/trunk/proxy/src/main/thrift/proxy.thrift
<https://reviews.apache.org/r/7936/#comment28440>

    the batch scanner will need to be closed in some way.  I did not see this being done on the server side.
    
    The thrift API could provide a close method.  Or the server side could time out idle batch scanner and close them.

/trunk/proxy/src/main/thrift/proxy.thrift
<https://reviews.apache.org/r/7936/#comment28439>

    Why not a list of Mutations?  
- kturner
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482
>
>
> Diffs
> -----
>
>   /trunk/pom.xml 1406772
>   /trunk/proxy/README PRE-CREATION
>   /trunk/proxy/examples/python/README PRE-CREATION
>   /trunk/proxy/examples/python/TestClient.py PRE-CREATION
>   /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/data/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/ruby/README PRE-CREATION
>   /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION

> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/thrift/proxy.thrift, line 133
> > <https://reviews.apache.org/r/7936/diff/1/?file=186436#file186436line133>
> >
> >     the batch scanner will need to be closed in some way.  I did not see this being done on the server side.
> >    
> >     The thrift API could provide a close method.  Or the server side could time out idle batch scanner and close them.
>
> Chris McCubbin wrote:
>     Good point. I had this on my list to do but didn't get it in there.

Thats annoying.   So would multiple ports and services be preferable to the long method names?  I dislike both options, but I Am not sure which I dislike more at this point.
- kturner
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13220
-----------------------------------------------------------
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482
>
>
> Diffs
> -----
>
>   /trunk/pom.xml 1406772
>   /trunk/proxy/README PRE-CREATION
>   /trunk/proxy/examples/python/README PRE-CREATION
>   /trunk/proxy/examples/python/TestClient.py PRE-CREATION
>   /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/data/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/ruby/README PRE-CREATION
>   /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/test_client.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/thrift.rb PRE-CREATION
>   /trunk/proxy/pom.xml PRE-CREATION
>   /trunk/proxy/proxy.properties PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyHarness.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/TestProxyClient.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/Util.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloProxy.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloSecurityException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/IOException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/KeyValueAndPeek.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/NoMoreEntriesException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyIteratorSetting.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxySystemPermission.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyTablePermission.java PRE-CREATION

> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/thrift/proxy.thrift, line 72
> > <https://reviews.apache.org/r/7936/diff/1/?file=186436#file186436line72>
> >
> >     can anything be done to make methods shorter?  Is it possible to get rid of all of the tableOperation_ and instanceOperation_ prefixes?  Does having different services (ie. AccumuloInstanceProxy, AccumloTableOpProxy) force us to run those services on separate ports?

Different services must be run on separate ports, yes. That was the reason I included the prefixes. See this really old thrift ticket where they basically say they aren't going to change that: https://issues.apache.org/jira/browse/THRIFT-66
> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/thrift/proxy.thrift, line 140
> > <https://reviews.apache.org/r/7936/diff/1/?file=186436#file186436line140>
> >
> >     Why not a list of Mutations?

TMutation uses serialized ColumnUpdates which would be very confusing to generate. I could make a proxy Mutation structure but this is fairly redundant with a list of TKeyValues.

The first cut at the implementation of insert grouped the keyvalues into Mutations by row. I took that out because it wasn't more performant than one Mutation per keyvalue. Adam just brought up a good point that we might want the assumption of atomicity for these entries, so I could change the server back to doing row grouping.
> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/thrift/proxy.thrift, line 133
> > <https://reviews.apache.org/r/7936/diff/1/?file=186436#file186436line133>
> >
> >     the batch scanner will need to be closed in some way.  I did not see this being done on the server side.
> >    
> >     The thrift API could provide a close method.  Or the server side could time out idle batch scanner and close them.

Good point. I had this on my list to do but didn't get it in there.
> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java, line 540
> > <https://reviews.apache.org/r/7936/diff/1/?file=186420#file186420line540>
> >
> >     sync on batchScanner before checking if its null

Good catch
> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java, line 543
> > <https://reviews.apache.org/r/7936/diff/1/?file=186420#file186420line543>
> >
> >     In past experience with Thrift throwing a TException in server side code woudl result in client seeing nothing or seeing something like an IOException instead of a TApplicationException.   But this was a while ago, not sure what current behavior of thrift is.

This isn't the case currently. An exception is definitely generated on the client side.
> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java, line 547
> > <https://reviews.apache.org/r/7936/diff/1/?file=186420#file186420line547>
> >
> >     If user does not scan until end, then will scanner always be present in scannerMap and iteratorMap?  If so this could lead to memory leaks.  I think its a commom use case that users do not read all data from a scanner.

Yes, this should be addressed.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13220
-----------------------------------------------------------
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482

> On Nov. 7, 2012, 9:14 p.m., kturner wrote:
> > /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java, line 567
> > <https://reviews.apache.org/r/7936/diff/1/?file=186420#file186420line567>
> >
> >     would probably be more efficient to get the scanner once from the iteratorMap, rather than doing a lookup in the map for each loop iteration

We just discussed a different design where the scanner_next_k is the "base" method and scanner_next calls scanner_next_k with k=1. This way around (while a little unintuitive) may actually result in the best overall performance since we are almost always going to call next_k when performance matters, since it maximizes scanner reuse and minimizes function calls in the server.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13220
-----------------------------------------------------------
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482
>
>
> Diffs
> -----
>
>   /trunk/pom.xml 1406772
>   /trunk/proxy/README PRE-CREATION
>   /trunk/proxy/examples/python/README PRE-CREATION
>   /trunk/proxy/examples/python/TestClient.py PRE-CREATION
>   /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/data/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/ruby/README PRE-CREATION
>   /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/test_client.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/thrift.rb PRE-CREATION
>   /trunk/proxy/pom.xml PRE-CREATION
>   /trunk/proxy/proxy.properties PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyHarness.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/TestProxyClient.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/Util.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloProxy.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloSecurityException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/IOException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/KeyValueAndPeek.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/NoMoreEntriesException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyIteratorSetting.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxySystemPermission.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyTablePermission.java PRE-CREATION

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13224
-----------------------------------------------------------

/trunk/proxy/README
<https://reviews.apache.org/r/7936/#comment28447>

    May be useful to provide guidance on how proxies should be run in a cluster.  For example run one proxy for each client node.
    
    So what should this guidance be?  Whats the best practice?
    
    Running a proxy process on every node where a client runs may be the most efficient.  But if the user has to manage this it could be cumbersome.  We could help by having each tserver run a proxy process may be the most convienet for users.  Could do this and provide a little bit of code for each language that helps a user find proxy.   This utility could prefer a local tserver if there is one and otherwise choose a random one.  Having tservers run a proxy service may make them less stable.
    

/trunk/proxy/src/main/thrift/proxy.thrift
<https://reviews.apache.org/r/7936/#comment28452>

    Having a createBatchWriter(), flushBatchWriter(), closeBatchWriter() methods will allow for a high performance write pipeline.  The single method makes the process synchronous, the user has to wait for writes to complete before sending more.  With multiple methods, the user could be sending a batch to the batch writer while the batch writer is sending the previous batch.
    
    Could possibly offer both.  The single method that does everything in one shot and multiple methods that allow a write pipeline.
    
    
- kturner
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482
>
>
> Diffs
> -----
>
>   /trunk/pom.xml 1406772
>   /trunk/proxy/README PRE-CREATION
>   /trunk/proxy/examples/python/README PRE-CREATION
>   /trunk/proxy/examples/python/TestClient.py PRE-CREATION
>   /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/data/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/ruby/README PRE-CREATION
>   /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/test_client.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/thrift.rb PRE-CREATION
>   /trunk/proxy/pom.xml PRE-CREATION
>   /trunk/proxy/proxy.properties PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyHarness.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/TestProxyClient.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/Util.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloProxy.java PRE-CREATION

> On Nov. 7, 2012, 9:41 p.m., kturner wrote:
> > /trunk/proxy/src/main/thrift/proxy.thrift, line 140
> > <https://reviews.apache.org/r/7936/diff/1/?file=186436#file186436line140>
> >
> >     Having a createBatchWriter(), flushBatchWriter(), closeBatchWriter() methods will allow for a high performance write pipeline.  The single method makes the process synchronous, the user has to wait for writes to complete before sending more.  With multiple methods, the user could be sending a batch to the batch writer while the batch writer is sending the previous batch.
> >    
> >     Could possibly offer both.  The single method that does everything in one shot and multiple methods that allow a write pipeline.
> >    
> >

Good idea, I can take a stab at this.
- Chris
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13224
-----------------------------------------------------------
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482
>
>
> Diffs
> -----
>
>   /trunk/pom.xml 1406772
>   /trunk/proxy/README PRE-CREATION
>   /trunk/proxy/examples/python/README PRE-CREATION
>   /trunk/proxy/examples/python/TestClient.py PRE-CREATION
>   /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/data/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/ruby/README PRE-CREATION
>   /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/test_client.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/thrift.rb PRE-CREATION
>   /trunk/proxy/pom.xml PRE-CREATION
>   /trunk/proxy/proxy.properties PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyHarness.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/TestProxyClient.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/Util.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloProxy.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloSecurityException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/IOException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/KeyValueAndPeek.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/NoMoreEntriesException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyIteratorSetting.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxySystemPermission.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/ProxyTablePermission.java PRE-CREATION

> On Nov. 7, 2012, 9:41 p.m., kturner wrote:
> > /trunk/proxy/src/main/thrift/proxy.thrift, line 140
> > <https://reviews.apache.org/r/7936/diff/1/?file=186436#file186436line140>
> >
> >     Having a createBatchWriter(), flushBatchWriter(), closeBatchWriter() methods will allow for a high performance write pipeline.  The single method makes the process synchronous, the user has to wait for writes to complete before sending more.  With multiple methods, the user could be sending a batch to the batch writer while the batch writer is sending the previous batch.
> >    
> >     Could possibly offer both.  The single method that does everything in one shot and multiple methods that allow a write pipeline.
> >    
> >
>
> Chris McCubbin wrote:
>     Good idea, I can take a stab at this.

I was wondering if the reverse might be true for the scanner... a single shot method for the case where I know I just want the first 100 key values.... I am not sure if its worth it... it saves the user one method call...
- kturner
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13224
-----------------------------------------------------------
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482
>
>
> Diffs
> -----
>
>   /trunk/pom.xml 1406772
>   /trunk/proxy/README PRE-CREATION
>   /trunk/proxy/examples/python/README PRE-CREATION
>   /trunk/proxy/examples/python/TestClient.py PRE-CREATION
>   /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/data/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/ruby/README PRE-CREATION
>   /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/test_client.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/thrift.rb PRE-CREATION
>   /trunk/proxy/pom.xml PRE-CREATION
>   /trunk/proxy/proxy.properties PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyHarness.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/TestProxyClient.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/Util.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloProxy.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloSecurityException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/IOException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/KeyValueAndPeek.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/NoMoreEntriesException.java PRE-CREATION

> On Nov. 7, 2012, 9:41 p.m., kturner wrote:
> > /trunk/proxy/README, line 13
> > <https://reviews.apache.org/r/7936/diff/1/?file=186398#file186398line13>
> >
> >     May be useful to provide guidance on how proxies should be run in a cluster.  For example run one proxy for each client node.
> >    
> >     So what should this guidance be?  Whats the best practice?
> >    
> >     Running a proxy process on every node where a client runs may be the most efficient.  But if the user has to manage this it could be cumbersome.  We could help by having each tserver run a proxy process may be the most convienet for users.  Could do this and provide a little bit of code for each language that helps a user find proxy.   This utility could prefer a local tserver if there is one and otherwise choose a random one.  Having tservers run a proxy service may make them less stable.
> >

Just realized that running on a proxy on each tserver is probably a bad idea.  It make the user of the thrift API have to handle failure cases (i.e. remote proxy I was using failed, now I need to select another).   If you are running a local proxy process, there could be failures but I think its much less likely.  Using a local proxy and letting the Accumulo client code handle remote failures seems like a good way to go.
- kturner
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7936/#review13224
-----------------------------------------------------------
On Nov. 7, 2012, 8:33 p.m., kturner wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7936/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2012, 8:33 p.m.)
>
>
> Review request for accumulo.
>
>
> Description
> -------
>
> Patch submitted by Chris McCubbin to add thrift proxy to Accumulo
>
>
> This addresses bug ACCUMULO-482.
>     https://issues.apache.org/jira/browse/ACCUMULO-482
>
>
> Diffs
> -----
>
>   /trunk/pom.xml 1406772
>   /trunk/proxy/README PRE-CREATION
>   /trunk/proxy/examples/python/README PRE-CREATION
>   /trunk/proxy/examples/python/TestClient.py PRE-CREATION
>   /trunk/proxy/examples/python/data/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/data/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/data/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy-remote PRE-CREATION
>   /trunk/proxy/examples/python/proxy/AccumuloProxy.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/__init__.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/constants.py PRE-CREATION
>   /trunk/proxy/examples/python/proxy/ttypes.py PRE-CREATION
>   /trunk/proxy/examples/ruby/README PRE-CREATION
>   /trunk/proxy/examples/ruby/accumulo_proxy.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/data_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_constants.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/proxy_types.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/test_client.rb PRE-CREATION
>   /trunk/proxy/examples/ruby/thrift.rb PRE-CREATION
>   /trunk/proxy/pom.xml PRE-CREATION
>   /trunk/proxy/proxy.properties PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyHarness.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/ProxyServer.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/TestProxyClient.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/Util.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloException.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloProxy.java PRE-CREATION
>   /trunk/proxy/src/main/java/org/apache/accumulo/proxy/api/AccumuloSecurityException.java PRE-CREATION