Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> avro object reuse

Copy link to this message
Re: avro object reuse
On Wed, Jun 1, 2011 at 5:45 PM, Scott Carey <[EMAIL PROTECTED]> wrote:
> It would be useful to get a 'jmap -histo:live' report as well, which will
> only have items that remain after a full GC.
> However, a high churn of short lived Jackson objects is not expected here
> unless the user is reading Json serialized files and not Avro binary.
> Avro Data Files only contain binary encoded Avro content.
> It would be surprising to see many Jackson objects here if reading Avro
> Data Files, because we expect to use Jackson to parse an Avro schema from
> json only once or twice per file.  After the schema is parsed, Jackson
> shouldn't be used.   A hundred thousand DeserializationConfig instances
> means that isn't the case.

Right -- it indicates that something (else) is using Jackson; and
there will typically be one instance of DeserializationConfig for each
data-binding call (ObjectMapper.readValue()), as a read-only copy is
made for operation.
... or if something is reading schema that many times, that sounds
like a problem in itself.

-+ Tatu +-