Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Re: Avro remote exception


+
shekhar sharma 2012-07-17, 06:01
Copy link to this message
-
Re: Avro remote exception
Mike Percy 2012-07-17, 06:19
Som, on Flume NG you must run a source, channel, and sink all in the same
JVM. An Avro source accepts connections and events from an Avro sink.

Try taking a look @ the user guide:
https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/FlumeUserGuide.html

Regards,
Mike

On Mon, Jul 16, 2012 at 11:01 PM, shekhar sharma <[EMAIL PROTECTED]>wrote:

> Hello,
> I am running on flume-ng (flume-1.2.0.SNAPSHOT) and i am trying to see
> whether the events are being captured by avrosource or not!!!..
>
> When i am running avro source only programmitically it is throwing me this
> error...( I am trying to imitate TestAvroSource.java test)
>
> But if i configure both source and sink in the agent file and then send
> the events, i am not getting any error
>
>
> Well my actuall use case is like this:
>
> machine1---> avro source and memory channel
> machine2--->sink and memory channel
>
> i am trying to achieve this.,
>
> Regards,
> Som Shekhar
>
>
>
>
>
> On Tue, Jul 17, 2012 at 3:53 AM, Mike Percy <[EMAIL PROTECTED]> wrote:
>
>> (adding flume users list, bcc: avro users list)
>>
>> Shekhar,
>> Can you please provide more information about what you are doing when you
>> see this message and the version of Flume that you are running?
>>
>> Regards,
>> Mike
>>
>>
>>
>> On Mon, Jul 16, 2012 at 3:12 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>>
>>> From the stack trace you sent this looks like an error thrown by
>>> Flume, not by Avro.
>>>
>>> Doug
>>>
>>> On Fri, Jul 13, 2012 at 11:53 AM, shekhar sharma <[EMAIL PROTECTED]>
>>> wrote:
>>> > Hello,
>>> > i am using avro source to send the events to Flume Avro source.
>>> Connection
>>> > is fine but when i am sending the events it is throwing Avro remote
>>> > exception.
>>> > I  have an application which uses RPC client factory get
>>> > connected to the avro source. RpcClient is successfuly connected to
>>> > the Avro Source but when i am sending the events to the Avro source i
>>> > am getting AvroRemoteException.
>>> >
>>> > This is the following code snippet  am using:
>>> > AvroSource source = new AvroSource();
>>> >                 Channel channel = new MemoryChannel();
>>> >                 List<Channel> ch = new ArrayList();
>>> >                 ch.add(channel);
>>> >                 ChannelSelector cs =  new ReplicatingChannelSelector();
>>> >                 cs.setChannels(ch);
>>> >                 Configurables.configure(cs, new Context());
>>> >
>>> >                 ChannelProcessor cp = new ChannelProcessor(cs);
>>> >                 Context context = new Context();
>>> >                 context.put("bind","localhost"
>>> > );
>>> >                 context.put("port", "12345");
>>> >                 source.configure(context);
>>> >                 source.setChannelProcessor(cp);
>>> >                 source.start();
>>> >                 //Server server = new NettyServer(new
>>> > SpecificResponder(AvroSource.class, source), new
>>> > InetSocketAddress("localhost",12345));
>>> >                 //server.start();
>>> >
>>> >                 AvroSourceProtocol client >>> > SpecificRequestor.getClient(AvroSourceProtocol.class, new
>>> > NettyTransceiver(new InetSocketAddress(12345)));
>>> >         AvroFlumeEvent avroEvent = new AvroFlumeEvent();
>>> >
>>> >         avroEvent.setHeaders(new HashMap<CharSequence,
>>> CharSequence>());
>>> >         avroEvent.setBody(ByteBuffer.wrap("Hello avro".getBytes()));
>>> >         Status status = client.append(avroEvent);
>>> >
>>> >
>>> >                 Transaction transaction = channel.getTransaction();
>>> >         transaction.begin();
>>> >
>>> >         Event event1 = channel.take();
>>> >         transaction.commit();
>>> >         transaction.close();
>>> >
>>> > And i am getting the following exception:
>>> >
>>> > [New I/O server worker #1-1] WARN org.apache.avro.ipc.Responder - user
>>> error
>>> > java.lang.NullPointerException
>>> >         at
>>> >
>>> org.apache.flume.channel.ChannelProcessor.processEvent(ChannelProcessor.java:183)
+
shekhar sharma 2012-07-13, 18:53
+
Doug Cutting 2012-07-16, 22:12