noob question:
What's the best way to determine the offset of the message I just
produced via the java api?

-brian

--
Brian ONeill
Lead Architect, Health Market Science (http://healthmarketscience.com)
mobile:215.588.6024
blog: http://weblogs.java.net/blog/boneill42/
blog: http://brianoneill.blogspot.com/

Hi Brian,

This is not directly available in 0.7 which does not have any producer
acks. The closest I can think of right now is to issue offset requests
after your producer send, but that would make sense if you only have one
producer to that topic

0.8 which is under development has producer acks which does return the next
append offsets.

Joel

On Wed, Aug 8, 2012 at 10:29 AM, Brian O'Neill <[EMAIL PROTECTED]>wrote:

> noob question:
> What's the best way to determine the offset of the message I just
> produced via the java api?
>
> -brian
>
> --
> Brian ONeill
> Lead Architect, Health Market Science (http://healthmarketscience.com)
> mobile:215.588.6024
> blog: http://weblogs.java.net/blog/boneill42/
> blog: http://brianoneill.blogspot.com/
>

Thanks for the response Joel.  That will work for now.

I'm queuing a batch of data. (for a Storm topology)
I really just need to know the consuming offset that I need to wait for to
know that the batch is complete.  As long as I get a offset greater than
that of the last message I queued, I should be good.  (So I should be good
even w/ multiple producers)

Is there an example somewhere for issuing an OffSetRequest?  I grep'd the
codebase, but only see references in the Hadoop contribution.

-brian
--
Brian O'Neill
Lead Architect, Software Development
Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406
p: 215.588.6024
www.healthmarketscience.com

On 8/8/12 2:15 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:

>Hi Brian,
>
>This is not directly available in 0.7 which does not have any producer
>acks. The closest I can think of right now is to issue offset requests
>after your producer send, but that would make sense if you only have one
>producer to that topic
>
>0.8 which is under development has producer acks which does return the
>next
>append offsets.
>
>Joel
>
>On Wed, Aug 8, 2012 at 10:29 AM, Brian O'Neill
><[EMAIL PROTECTED]>wrote:
>
>> noob question:
>> What's the best way to determine the offset of the message I just
>> produced via the java api?
>>
>> -brian
>>
>> --
>> Brian ONeill
>> Lead Architect, Health Market Science (http://healthmarketscience.com)
>> mobile:215.588.6024
>> blog: http://weblogs.java.net/blog/boneill42/
>> blog: http://brianoneill.blogspot.com/
>>

Yes - take a look at the GetOffsetShell tool. It may also help to look at
the SimpleConsumer class' getOffsetsBefore method.

Thanks,

Joel

On Wed, Aug 8, 2012 at 12:42 PM, Brian O'Neill <[EMAIL PROTECTED]> wrote:

>
> Thanks for the response Joel.  That will work for now.
>
> I'm queuing a batch of data. (for a Storm topology)
> I really just need to know the consuming offset that I need to wait for to
> know that the batch is complete.  As long as I get a offset greater than
> that of the last message I queued, I should be good.  (So I should be good
> even w/ multiple producers)
>
> Is there an example somewhere for issuing an OffSetRequest?  I grep'd the
> codebase, but only see references in the Hadoop contribution.
>
> -brian
>
>
> --
> Brian O'Neill
> Lead Architect, Software Development
> Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406
> p: 215.588.6024
> www.healthmarketscience.com
>
>
>
>
>
> On 8/8/12 2:15 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:
>
> >Hi Brian,
> >
> >This is not directly available in 0.7 which does not have any producer
> >acks. The closest I can think of right now is to issue offset requests
> >after your producer send, but that would make sense if you only have one
> >producer to that topic
> >
> >0.8 which is under development has producer acks which does return the
> >next
> >append offsets.
> >
> >Joel
> >
> >On Wed, Aug 8, 2012 at 10:29 AM, Brian O'Neill
> ><[EMAIL PROTECTED]>wrote:
> >
> >> noob question:
> >> What's the best way to determine the offset of the message I just
> >> produced via the java api?
> >>
> >> -brian
> >>
> >> --
> >> Brian ONeill
> >> Lead Architect, Health Market Science (http://healthmarketscience.com)
> >> mobile:215.588.6024
> >> blog: http://weblogs.java.net/blog/boneill42/
> >> blog: http://brianoneill.blogspot.com/
> >>
>
>
>

Thanks. I just got it sorted out.
Its working perfectly.  Thanks for the help.

-brian

--
Brian O'Neill
Lead Architect, Software Development
Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406
p: 215.588.6024
www.healthmarketscience.com

On 8/8/12 3:59 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:

>Yes - take a look at the GetOffsetShell tool. It may also help to look at
>the SimpleConsumer class' getOffsetsBefore method.
>
>Thanks,
>
>Joel
>
>On Wed, Aug 8, 2012 at 12:42 PM, Brian O'Neill <[EMAIL PROTECTED]>
>wrote:
>
>>
>> Thanks for the response Joel.  That will work for now.
>>
>> I'm queuing a batch of data. (for a Storm topology)
>> I really just need to know the consuming offset that I need to wait for
>>to
>> know that the batch is complete.  As long as I get a offset greater than
>> that of the last message I queued, I should be good.  (So I should be
>>good
>> even w/ multiple producers)
>>
>> Is there an example somewhere for issuing an OffSetRequest?  I grep'd
>>the
>> codebase, but only see references in the Hadoop contribution.
>>
>> -brian
>>
>>
>> --
>> Brian O'Neill
>> Lead Architect, Software Development
>> Health Market Science | 2700 Horizon Drive | King of Prussia, PA 19406
>> p: 215.588.6024
>> www.healthmarketscience.com
>>
>>
>>
>>
>>
>> On 8/8/12 2:15 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:
>>
>> >Hi Brian,
>> >
>> >This is not directly available in 0.7 which does not have any producer
>> >acks. The closest I can think of right now is to issue offset requests
>> >after your producer send, but that would make sense if you only have
>>one
>> >producer to that topic
>> >
>> >0.8 which is under development has producer acks which does return the
>> >next
>> >append offsets.
>> >
>> >Joel
>> >
>> >On Wed, Aug 8, 2012 at 10:29 AM, Brian O'Neill
>> ><[EMAIL PROTECTED]>wrote:
>> >
>> >> noob question:
>> >> What's the best way to determine the offset of the message I just
>> >> produced via the java api?
>> >>
>> >> -brian
>> >>
>> >> --
>> >> Brian ONeill
>> >> Lead Architect, Health Market Science
>>(http://healthmarketscience.com)
>> >> mobile:215.588.6024
>> >> blog: http://weblogs.java.net/blog/boneill42/
>> >> blog: http://brianoneill.blogspot.com/
>> >>
>>
>>
>>