Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Mapreduce output format killing tablet servers


Copy link to this message
-
Re: Mapreduce output format killing tablet servers
I had a similar thread going on and am currently rummaging through the
batch writer code (as well as pontificating on how the tablet server
handles multiple write clients for the tablet).

What is your ingest skew like? Is it uniform? How quickly do splits occur?
I've seen, at relatively low scale, doing "live" ingest become problematic.

Have you looked into using file output? One of our committers, Cory, has a
library that can handle writing to multiple tables/files. You can peek
here: https://github.com/calrissian/accumulo-recipes (doing a `find . -name
'Group*'` will give you the classes you need). I had to do some massaging
to get them to work properly and am happy to share what I had to do if this
becomes a route you're interested in.
On Wed, Jun 25, 2014 at 2:10 PM, Sean Busbey <[EMAIL PROTECTED]> wrote: