| clear query|facets|time |
Search criteria: .
Results from 11 to 20 from
21 (0.088s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...Hi Scott, The problem is found. In the reduce job when there were too many values for some key, I stopped reading values from an iterator. So apparently the rest of the val...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-18, 12:50
|
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...There is a possible reason: It seems that there is an upper limit of 10,001 records per reduce input group. (or is there a setting?) If I output one million rows with the same k...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-17, 22:59
|
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...Hi Scott, thanks for all the suggestions. I really appreciate your support. Unfortunately, I could not solve the problem so far. That's what I have tried: 1. Switche...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-17, 22:02
|
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...On Aug 17, 2011, at 7:18 PM, Scott Carey wrote: wrote: wanted to reuse the same Utf8 object instead of creating new from String upon each map() call. String object? since the pe...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-17, 18:09
|
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...One more update: running the job with the -XX:-UseLoopPredicate option gave the same results. The difference between mapper output records and reducer input records is per...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-17, 15:49
|
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...btw, I was thinking to try it with Utf8 objects instead of strings and I wanted to reuse the same Utf8 object instead of creating new from String upon each map() call. Why ...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-17, 12:02
|
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...Hi Scott, The pair types are Pair, but in essence when I call "collect()" then I always provide a java.lang.String object. The reduce method is reduce(CharSequence k...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-17, 08:32
|
|
|
Re: Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...Hi, Scott, thanks for your reply. We are using Avro 1.5.1 and Sun JDK 6, but the exact version I will have to look up. args if affect I will have to collect it...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-16, 22:56
|
|
|
Map output records/reducer input records mismatch - Avro - [mail # user]
|
|
...Hi, I'm having multiple hadoop jobs that use the avro mapred API. Only in one of the jobs I have a visible mismatch between a number of map output records and reducer input recor...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-16, 15:39
|
|
|
Re: Java Example of writing a union - Avro - [mail # user]
|
|
...I'm assuming for now that you are using a specific writer and you have a union schema with two records FOO and BAR (you should get two classes FOO and BAR generated by avro tools...
|
|
|
Author: Vyacheslav Zholudev,
2011-08-08, 16:52
|
|
|
|
