Subject: Re: Reading Kafka directly from Pig?
Right now it only terminates if SimpleConsumer hits the timeout. So, in
theory it can forever. To bound the InputFormat, I would probably add a
max time or max number of messages to consume (in addition to the timeout).
I started by looking at the Camus code, but it was easier to whip up a
simple InputFormat for testing. If this becomes a real thing, I would
probably figure out how to utilize Camus.
On 8/7/13 10:49 AM, Jun Rao wrote: