Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro + MRUnit


Copy link to this message
-
Re: Avro + MRUnit
Tanya Bansal 2013-01-08, 22:55
Has anyone else faced the same problem?
On Tue, Dec 11, 2012 at 10:38 AM, Tanya Bansal <[EMAIL PROTECTED]>wrote:

> Thanks for the quick response.
>
> Here is the source code : http://pastebin.com/6t12ZZ5T
>
> I have created the Count class on my own.
>
> Thanks
> -Tanya
>
>
>
>
>
>
> On Tue, Dec 11, 2012 at 1:58 AM, Dave Beech <[EMAIL PROTECTED]> wrote:
>
>> Hi Tanya
>>
>> How was your Count class created? Did you write it yourself or did you
>> use Avro tools to generate code from a schema? Something is mixed up
>> because Avro is trying to write a "generic" record whereas I'd expect it to
>> be using "specific" records in your case.
>>
>> It's hard to work out the problem from just the stacktrace alone, but if
>> you could share your code (using github gists or pastebin.com), I'd be
>> happy to take a look.
>>
>> Cheers
>> Dave
>>
>>
>> On 11 December 2012 03:53, Tanya Bansal <[EMAIL PROTECTED]> wrote:
>>
>>> Are there any examples out there using Apache and MRUnit?
>>>
>>> I have configured the AvroSerialization class using the
>>> io.serializations string
>>> I am getting the following exception when I call mapDriver.run():
>>>
>>> java.lang.ClassCastException:
>>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Count cannot be
>>> cast to org.apache.avro.generic.IndexedRecord
>>> at org.apache.avro.generic.GenericData.getField(GenericData.java:518)
>>>  at org.apache.avro.generic.GenericData.getField(GenericData.java:533)
>>> at
>>> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:103)
>>>  at
>>> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
>>> at
>>> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
>>>  at
>>> org.apache.avro.hadoop.io.AvroSerializer.serialize(AvroSerializer.java:103)
>>> at
>>> org.apache.avro.hadoop.io.AvroSerializer.serialize(AvroSerializer.java:45)
>>>  at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:62)
>>> at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:81)
>>>  at
>>> org.apache.hadoop.mrunit.mapreduce.mock.MockContextWrapper$4.answer(MockContextWrapper.java:78)
>>> at
>>> org.mockito.internal.stubbing.StubbedInvocationMatcher.answer(StubbedInvocationMatcher.java:34)
>>>  at org.mockito.internal.MockHandler.handle(MockHandler.java:99)
>>> at
>>> org.mockito.internal.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:36)
>>>  at
>>> org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:48)
>>> at
>>> org.apache.hadoop.mapreduce.Mapper$Context$$EnhancerByMockitoWithCGLIB$$4ae1b65a.write(<generated>)
>>>  at
>>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Map.map(AvroWordCount.java:76)
>>> at
>>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Map.map(AvroWordCount.java:1)
>>>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>>> at org.apache.hadoop.mrunit.mapreduce.MapDriver.run(MapDriver.java:221)
>>>  at
>>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCountTest.TestMrUnitMapper(AvroWordCountTest.java:61)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>  at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>  at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>>  at
>>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>> at
>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>>  at
>>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>> at
>>> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>>>  at
>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)