Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - 7.1 support for List<ProducerData>


Copy link to this message
-
Re: 7.1 support for List<ProducerData>
Felix GV 2012-08-20, 17:06
This may not be entirely related to what you're talking about, but why
would an async producer not be able to meet your throughput needs, and a
sync producer be able to?

Both sync and async producers can be configured to batch more than one
message together, and that's pretty much the main thing that's required to
be able to achieve good throughput, AFAIK.

...?

--
Felix

On Mon, Aug 20, 2012 at 12:49 PM, will martin <[EMAIL PROTECTED]> wrote:

> Thanks Neha. All my data is of 1 type. The serializer in place doesn't seem
> to handle an array of String.
>
> The ProducerData I use is a collection of same types of data wrapped in a
> single defintion, according to as I read spec.  Am I to understand that,
> having a producer batch records itself is unsupported?  The async producer
> can't meet my throughput needs and as I understand is targetted at implicit
> load balancing among different client machines.
>
> Additionally, the sync producer can meet my needs, but requires more use of
> the lower-level design features. For maintenance, it'd be great if I could
> create a list of Strings, create a ProducerData<String, List<String>> and
> have this be serialized.
>
> It occurs to me that the described  serialization may need my attention?
>
> Thx
>
>
> On Mon, Aug 20, 2012 at 12:06 PM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > The producer takes in a "serializer.class" config that it uses to
> > serialize data sent by the Producer. A Producer instance is tied to
> > the type of data it is sending, so you won't be able to send data
> > belonging to diverse types using the same Producer object.
> >
> > Thanks,
> > Neha
> >
> > On Mon, Aug 20, 2012 at 8:02 AM, will martin <[EMAIL PROTECTED]>
> wrote:
> > > This use case is defined by the following snippet from the Design
> section
> > > of the doc pages.
> > >
> > > class Producer {
> > >
> > > public void send (ProducerData)
> > >
> > > public void send (List<ProducerData>)
> > >
> > > public void close()
> > > }
> > >
> > > I've tried various composites for the List<ProducerData> argument,
> > > including strings and Messages. All of these throw serialization errors
> > > deep in the engine.
> > >
> > > Is the list form of send supported in 7.1?
> > >
> > > Thanks in advance,
> > > mmartin
> >
>