Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> avro object reuse


Copy link to this message
-
Re: avro object reuse
On Wed, Jun 1, 2011 at 5:45 PM, Scott Carey <[EMAIL PROTECTED]> wrote:
> It would be useful to get a 'jmap -histo:live' report as well, which will
> only have items that remain after a full GC.
>
> However, a high churn of short lived Jackson objects is not expected here
> unless the user is reading Json serialized files and not Avro binary.
> Avro Data Files only contain binary encoded Avro content.
>
> It would be surprising to see many Jackson objects here if reading Avro
> Data Files, because we expect to use Jackson to parse an Avro schema from
> json only once or twice per file.  After the schema is parsed, Jackson
> shouldn't be used.   A hundred thousand DeserializationConfig instances
> means that isn't the case.

Right -- it indicates that something (else) is using Jackson; and
there will typically be one instance of DeserializationConfig for each
data-binding call (ObjectMapper.readValue()), as a read-only copy is
made for operation.
... or if something is reading schema that many times, that sounds
like a problem in itself.

-+ Tatu +-
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB