Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Avro + MRUnit


+
Tanya Bansal 2012-12-11, 03:53
Copy link to this message
-
Re: Avro + MRUnit
Dave Beech 2012-12-11, 09:58
Hi Tanya

How was your Count class created? Did you write it yourself or did you use
Avro tools to generate code from a schema? Something is mixed up because
Avro is trying to write a "generic" record whereas I'd expect it to be
using "specific" records in your case.

It's hard to work out the problem from just the stacktrace alone, but if
you could share your code (using github gists or pastebin.com), I'd be
happy to take a look.

Cheers
Dave
On 11 December 2012 03:53, Tanya Bansal <[EMAIL PROTECTED]> wrote:

> Are there any examples out there using Apache and MRUnit?
>
> I have configured the AvroSerialization class using the io.serializations
> string
> I am getting the following exception when I call mapDriver.run():
>
> java.lang.ClassCastException:
> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Count cannot be
> cast to org.apache.avro.generic.IndexedRecord
> at org.apache.avro.generic.GenericData.getField(GenericData.java:518)
>  at org.apache.avro.generic.GenericData.getField(GenericData.java:533)
> at
> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:103)
>  at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
>  at
> org.apache.avro.hadoop.io.AvroSerializer.serialize(AvroSerializer.java:103)
> at
> org.apache.avro.hadoop.io.AvroSerializer.serialize(AvroSerializer.java:45)
>  at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:62)
> at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:81)
>  at
> org.apache.hadoop.mrunit.mapreduce.mock.MockContextWrapper$4.answer(MockContextWrapper.java:78)
> at
> org.mockito.internal.stubbing.StubbedInvocationMatcher.answer(StubbedInvocationMatcher.java:34)
>  at org.mockito.internal.MockHandler.handle(MockHandler.java:99)
> at
> org.mockito.internal.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:36)
>  at
> org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:48)
> at
> org.apache.hadoop.mapreduce.Mapper$Context$$EnhancerByMockitoWithCGLIB$$4ae1b65a.write(<generated>)
>  at
> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Map.map(AvroWordCount.java:76)
> at
> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Map.map(AvroWordCount.java:1)
>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mrunit.mapreduce.MapDriver.run(MapDriver.java:221)
>  at
> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCountTest.TestMrUnitMapper(AvroWordCountTest.java:61)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>  at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>  at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>  at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>  at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>  at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>  at
> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
+
Tanya Bansal 2012-12-11, 18:38
+
Tanya Bansal 2013-01-08, 22:55
+
Dave Beech 2013-01-09, 15:13