| clear query|facets|time |
Search criteria: .
Results from 11 to 20 from
40 (2.122s).
|
|
|
Loading phrases to help you
refine your search...
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...Since this was in production, we did not turn on stack trace. Also, it was highly unlikely that there was any data corrupted because, if one ma pper failed due to out of memory, ...
|
|
|
Author: ey-chih chow,
2011-06-10, 17:53
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...We configure more than 100MB for MapReduce to do sorting. Memory we alloca te for doing other things in the mapper actually is larger, but, for th is job, we always get out-of-meory ex...
|
|
|
Author: ey-chih chow,
2011-06-09, 23:45
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...In addition, we ran the same MR job once again and got the following hist ogram. Whey this is different from the previous one? Thanks. Ey-Chih Chow Object Histogram:...
|
|
|
Author: ey-chih chow,
2011-06-09, 22:26
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...I forgot to mention that the histogram in my previous message was extracted from a mapper of one of our MR job. Ey-Chih Chow From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Subject...
|
|
|
Author: ey-chih chow,
2011-06-09, 22:16
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...We did more monitoring. At one instance, we got the following histogram via Jmap. The question is why there are so many instances of BinaryDecoder $BufferAccessor and Binar...
|
|
|
Author: ey-chih chow,
2011-06-09, 22:08
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...We create GenericData.Record a lot in our code via new GenericData.Record(s chema). Will this generates Jackson calls? Thanks. Ey-Chih Chow se, ; be om> o on s &nb...
|
|
|
Author: ey-chih chow,
2011-06-02, 17:23
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...What follows is the whole output of our jmap. Hope this can help you ident ify the problem. num #instances #bytes Class description--------------------------------- 4022...
|
|
|
Author: ey-chih chow,
2011-06-01, 22:21
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...We use a lot of toString() call on the avro Utf8 object. Will this cause J ackson call? Thanks. Ey-Chih From: [EMAIL PROTECTED] To: [EMAIL PROTECTED] Date: Wed, 1 Jun 2011...
|
|
|
Author: ey-chih chow,
2011-06-01, 22:14
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...We ran jmap on one of our mapper and found the top usage as follows: num #instances #bytes Class description--------------------------------- 40228984 int[]3: 388799 19966776 char[...
|
|
|
Author: ey-chih chow,
2011-06-01, 18:34
|
|
|
RE: avro object reuse - Avro - [mail # user]
|
|
...I actually looked into Avro code to find out how Avro does object reuse. I looked at AvroUtf8InputFormat and got the following question. Why a new U tf8 object has to be cr...
|
|
|
Author: ey-chih chow,
2011-06-01, 00:40
|
|
|
|
