Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> Accumulo 1.4 Memory Issues


+
Matt Parker 2012-08-02, 17:16
+
Marc Parisi 2012-08-02, 17:25
+
Matt Parker 2012-08-02, 17:34
+
John Vines 2012-08-02, 17:44
+
Matt Parker 2012-08-04, 20:37
Copy link to this message
-
Re: Accumulo 1.4 Memory Issues
Is it the TServer bombing out or your client or both? How often are you
flushing your writer?

On Thu, Aug 2, 2012 at 10:34 AM, Matt Parker <[EMAIL PROTECTED]> wrote:

> for my small test case, I'm storing some basic data in three tables:
>
> nodes - spatial index (id, list of child nodes, whether it's a leaf node )
> image metadata - (id, bounding box coordinates, a text string of the
> bounding box)
> link - linking table that tells which images correspond to specific nodes.
>
> The image data isn't being stored in Accumulo, yet.
>
>
>
> On Thu, Aug 2, 2012 at 1:25 PM, Marc Parisi <[EMAIL PROTECTED]> wrote:
>
>> are you using native maps? if so, are they being used?
>>
>>
>> On Thu, Aug 2, 2012 at 1:16 PM, Matt Parker <[EMAIL PROTECTED]>wrote:
>>
>>> I setup a single instance Accumulo server.
>>>
>>> I can load 32K rows of image metadata without issue.
>>>
>>> I have another set of routines that build a dynamic spatial index, where
>>> nodes are inserted/updated/deleted over time.
>>> These operations are typically done one at a time, where each
>>> batchwriter are closed after use.
>>>
>>> It loads maybe a couple hundred operations, and then it dies with an
>>> OutOfMemory error when trying to close a batchwriter.
>>>
>>> I tried uping the memery settings on my client and on the tserver, but
>>> the results were the same.
>>>
>>> Outside of Accumulo, I can build the whole index in memory without any
>>> special JVM memory settings. I was wondering whether anyone else had run
>>> into a similar issue?
>>>
>>
>>
>
+
Matt Parker 2012-08-03, 15:04
+
Marc Parisi 2012-08-02, 17:53
+
Matt Parker 2012-08-02, 17:54
+
David Medinets 2012-08-03, 00:46
+
Josh Elser 2012-08-03, 02:02
+
Adam Fuchs 2012-08-03, 11:45
+
Matt Parker 2012-08-02, 17:51
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB