Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> avro object reuse


+
ey-chih chow 2011-05-31, 17:38
+
ey-chih chow 2011-06-01, 00:40
+
Scott Carey 2011-06-01, 01:26
+
ey-chih chow 2011-06-01, 18:34
+
Scott Carey 2011-06-01, 20:38
+
ey-chih chow 2011-06-01, 22:14
+
Scott Carey 2011-06-01, 22:27
+
ey-chih chow 2011-06-09, 22:08
+
ey-chih chow 2011-06-09, 22:16
+
ey-chih chow 2011-06-09, 22:26
+
Scott Carey 2011-06-09, 22:45
+
Scott Carey 2011-06-09, 22:42
+
ey-chih chow 2011-06-09, 23:45
+
Scott Carey 2011-06-09, 23:58
+
Scott Carey 2011-06-10, 00:43
+
ey-chih chow 2011-06-10, 17:53
+
Scott Carey 2011-06-10, 18:11
+
ey-chih chow 2011-06-10, 19:07
+
Scott Carey 2011-06-10, 22:58
+
Scott Carey 2011-06-01, 20:45
+
ey-chih chow 2011-06-01, 22:21
+
Tatu Saloranta 2011-06-02, 00:13
+
Scott Carey 2011-06-02, 00:45
Copy link to this message
-
Re: avro object reuse
On Wed, Jun 1, 2011 at 5:45 PM, Scott Carey <[EMAIL PROTECTED]> wrote:
> It would be useful to get a 'jmap -histo:live' report as well, which will
> only have items that remain after a full GC.
>
> However, a high churn of short lived Jackson objects is not expected here
> unless the user is reading Json serialized files and not Avro binary.
> Avro Data Files only contain binary encoded Avro content.
>
> It would be surprising to see many Jackson objects here if reading Avro
> Data Files, because we expect to use Jackson to parse an Avro schema from
> json only once or twice per file.  After the schema is parsed, Jackson
> shouldn't be used.   A hundred thousand DeserializationConfig instances
> means that isn't the case.

Right -- it indicates that something (else) is using Jackson; and
there will typically be one instance of DeserializationConfig for each
data-binding call (ObjectMapper.readValue()), as a read-only copy is
made for operation.
... or if something is reading schema that many times, that sounds
like a problem in itself.

-+ Tatu +-
+
Scott Carey 2011-06-02, 01:48
+
ey-chih chow 2011-06-02, 17:23
+
Scott Carey 2011-06-02, 21:14