Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> AVRO_EVENT problem


+
DeCarlo, Thom 2012-12-06, 14:59
+
Brock Noland 2012-12-06, 15:06
+
DeCarlo, Thom 2012-12-06, 15:09
+
Brock Noland 2012-12-06, 18:57
+
DeCarlo, Thom 2012-12-06, 19:29
Copy link to this message
-
Re: AVRO_EVENT problem
Hmm, looks like I missed something..Not familiar with this code.

On Thu, Dec 6, 2012 at 1:29 PM, DeCarlo, Thom <[EMAIL PROTECTED]> wrote:
> Hmmm... How does one define/check/set the schema? I lifted my configuration straight from the on-line documentation and it only mentions a schema in the JDBC Channel section.
>
> --
> Thom DeCarlo
>
>
> -----Original Message-----
> From: Brock Noland [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, December 06, 2012 1:57 PM
> To: [EMAIL PROTECTED]
> Subject: Re: AVRO_EVENT problem
>
> It seems to me like the object you are trying to write doesn't match
> the Schema? I'd check your convert() method.
>
> On Thu, Dec 6, 2012 at 9:09 AM, DeCarlo, Thom <[EMAIL PROTECTED]> wrote:
>> Oh, right... forgot the version. I'm using the latest (as of yesterday) from the git repository. It gets built as 1.4-SNAPSHOT.
>>
>> --
>> Thom DeCarlo
>>
>>
>> -----Original Message-----
>> From: Brock Noland [mailto:[EMAIL PROTECTED]]
>> Sent: Thursday, December 06, 2012 10:06 AM
>> To: [EMAIL PROTECTED]
>> Subject: Re: AVRO_EVENT problem
>>
>> Hi,
>>
>> Hopefully someone will be able to answer the AVRO issue, in order to
>> help them, what version of Flume are you running?
>>
>> Brock
>>
>> On Thu, Dec 6, 2012 at 8:59 AM, DeCarlo, Thom <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>> I'm just getting started with flume, so I apologize if this is an already known problem.
>>>
>>> I'm trying to set up a FILE_ROLL sink that uses the AVRO_EVENT serializer. But, when I start the agent I get an exception thrown from within the AVRO DataFileWriter class. It is failing when trying to cast a java.util.ArrayList to java.util.Map.
>>>
>>> Can anyone tell me how this is supposed to work?
>>>
>>> OBTW, I'm running on Windows 7 (64-bit), and I've tried both the 32- and 64-bit versions of both Java 1.6.0_37 and 1.7.0_09, with the same results.
>>>
>>> Thanks,
>>> Thom
>>> --
>>> The console output looks like this:
>>>
>>> 2012-12-04 14:24:25,111 (lifecycleSupervisor-1-4) [INFO - org.apache.flume.sink.RollingFileSink.start(RollingFileSink.java:135)] RollingFileSink fileSink-1 started.
>>> 2012-12-04 14:24:25,145 (lifecycleSupervisor-1-2) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:164)] Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:33333]
>>> 2012-12-04 14:24:25,593 (SinkRunner-PollingRunner-DefaultSinkProcessor) [WARN - org.apache.flume.channel.jdbc.impl.JdbcTransactionImpl.rollback(JdbcTransactionImpl.java:102)] Marking transaction for rollback
>>> 2012-12-04 14:24:25,594 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.channel.jdbc.impl.JdbcTransactionImpl.close(JdbcTransactionImpl.java:118)] Attempting transaction roll-back
>>> 2012-12-04 14:24:25,604 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event. Exception follows.
>>> org.apache.flume.EventDeliveryException: Failed to process transaction
>>>         at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:218)
>>>         at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>>>         at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>>>         at java.lang.Thread.run(Thread.java:722)
>>> Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException: java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map
>>>         at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:263)
>>>         at org.apache.flume.serialization.AbstractAvroEventSerializer.write(AbstractAvroEventSerializer.java:108)
>>>         at org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:195)
>>>         ... 3 more
>>> Caused by: java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.util.Map
>>>         at org.apache.avro.generic.GenericDatumWriter.getMapSize(GenericDatumWriter.java:174)
>>>         at org.apache.avro.generic.GenericDatumWriter.writeMap(GenericDatumWriter.java:159)

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
+
Brock Noland 2012-12-06, 19:51
+
DeCarlo, Thom 2012-12-06, 20:37
+
Brock Noland 2012-12-06, 20:42
+
DeCarlo, Thom 2012-12-06, 20:48
+
DeCarlo, Thom 2012-12-07, 19:47
+
Brock Noland 2012-12-07, 19:50