Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro + MRUnit


Copy link to this message
-
Re: Avro + MRUnit
Thanks for the quick response.

Here is the source code : http://pastebin.com/6t12ZZ5T

I have created the Count class on my own.

Thanks
-Tanya
On Tue, Dec 11, 2012 at 1:58 AM, Dave Beech <[EMAIL PROTECTED]> wrote:

> Hi Tanya
>
> How was your Count class created? Did you write it yourself or did you use
> Avro tools to generate code from a schema? Something is mixed up because
> Avro is trying to write a "generic" record whereas I'd expect it to be
> using "specific" records in your case.
>
> It's hard to work out the problem from just the stacktrace alone, but if
> you could share your code (using github gists or pastebin.com), I'd be
> happy to take a look.
>
> Cheers
> Dave
>
>
> On 11 December 2012 03:53, Tanya Bansal <[EMAIL PROTECTED]> wrote:
>
>> Are there any examples out there using Apache and MRUnit?
>>
>> I have configured the AvroSerialization class using the io.serializations
>> string
>> I am getting the following exception when I call mapDriver.run():
>>
>> java.lang.ClassCastException:
>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Count cannot be
>> cast to org.apache.avro.generic.IndexedRecord
>> at org.apache.avro.generic.GenericData.getField(GenericData.java:518)
>>  at org.apache.avro.generic.GenericData.getField(GenericData.java:533)
>> at
>> org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:103)
>>  at
>> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
>> at
>> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
>>  at
>> org.apache.avro.hadoop.io.AvroSerializer.serialize(AvroSerializer.java:103)
>> at
>> org.apache.avro.hadoop.io.AvroSerializer.serialize(AvroSerializer.java:45)
>>  at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:62)
>> at org.apache.hadoop.mrunit.Serialization.copy(Serialization.java:81)
>>  at
>> org.apache.hadoop.mrunit.mapreduce.mock.MockContextWrapper$4.answer(MockContextWrapper.java:78)
>> at
>> org.mockito.internal.stubbing.StubbedInvocationMatcher.answer(StubbedInvocationMatcher.java:34)
>>  at org.mockito.internal.MockHandler.handle(MockHandler.java:99)
>> at
>> org.mockito.internal.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:36)
>>  at
>> org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:48)
>> at
>> org.apache.hadoop.mapreduce.Mapper$Context$$EnhancerByMockitoWithCGLIB$$4ae1b65a.write(<generated>)
>>  at
>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Map.map(AvroWordCount.java:76)
>> at
>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCount$Map.map(AvroWordCount.java:1)
>>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>> at org.apache.hadoop.mrunit.mapreduce.MapDriver.run(MapDriver.java:221)
>>  at
>> amazon.fclm.labormetrics.mapreduce.sample.AvroWordCountTest.TestMrUnitMapper(AvroWordCountTest.java:61)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>  at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>  at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>>  at
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> at
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>>  at
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> at
>> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>>  at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
>> at
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>>  at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB