Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Re: [jira] [Commented] (FLUME-2333) HTTP source handler doesn't allow for responses


Copy link to this message
-
Re: [jira] [Commented] (FLUME-2333) HTTP source handler doesn't allow for responses
So the interface for BidirectionalHTTPSourceHandler would look like below.
Note that I removed the exception throwing in getEvents(), because I think
it¹s the responsibility of the handler to provide the appropriate response.
Do you agree with this?
Also, for both the HTTPSourceHandler and the BidirectionalHTTPSourceHandler,
I¹d add some code regarding handling null returned from getEvents ­ so we
don¹t try to write it to the channel.

  /**
   * Given a {@linkplain HttpServletRequest} and
   * {@linkplain HttpServletResponse}, returns a list of Flume Events.
   *
   * @param request
   *          The request to be parsed into Flume events.
   * @param response
   *          The response to sent back. Note that the servlet will call
   *          either
   *          {@link #onChannelException(HttpServletRequest,
HttpServletResponse, Exception)}
   *          or
   *          {@link #onSuccesfulCommit(HttpServletRequest,
HttpServletResponse)}
   *          after committing the events to the channel
   * @return List of Flume events generated from the request.
   */
  public List<Event> getEvents(HttpServletRequest request,
      HttpServletResponse response);

  /**
   * Method called to get the HTTP response if writing the events to the
channel
   * failed
   *
   * @param request
   *          The request for which the commit failed
   * @param response
   *          The response to sent back
   * @param exception
   *          The exception thrown while writing the events to the channel
   */
  public void onChannelException(HttpServletRequest request,
      HttpServletResponse response, Exception exception);

  /**
   * @param request
   *          The request successfully committed to the channel
   * @param response
   *          The response to sent back
   */
  public void onSuccesfulCommit(HttpServletRequest request,
      HttpServletResponse response);
From:  Gabriel Commeau <[EMAIL PROTECTED]>
Date:  Friday, February 28, 2014 at 1:54 PM
To:  <[EMAIL PROTECTED]>
Subject:  Re: [jira] [Commented] (FLUME-2333) HTTP source handler doesn't
allow for responses

What Hari proposed is very close to what I wrote. I didn¹t write a specific
method to get the response on successful commit, but I like the idea,
because I think it¹ll make the code more readable. In response to Hari¹s
suggestion, I¹d just add the HttpRequest object as a parameter in the ³on
commit² methods. It may be useful to get the encoding amongst other things.
I understand your suggestion Jeremy, but I would prefer the response to come
from the handler (BidirectionalHTTPSourceHandler) in an error case also ­
and very likely the code will be based on sendError().

From:  "Jeremy Karlson (JIRA)" <[EMAIL PROTECTED]>
Reply-To:  <[EMAIL PROTECTED]>
Date:  Friday, February 28, 2014 at 1:36 PM
To:  <[EMAIL PROTECTED]>
Subject:  [jira] [Commented] (FLUME-2333) HTTP source handler doesn't allow
for responses
    [
https://issues.apache.org/jira/browse/FLUME-2333?page=com.atlassian.jira.plu
gin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916340#comment
-13916340 ]

Jeremy Karlson commented on FLUME-2333:

I understand your concerns.  I did think about the error conditions and
considered something like what you proposed.  In the end I submitted it
as-is because of this:

http://docs.oracle.com/javaee/6/api/javax/servlet/http/HttpServletResponse.h
tml#sendError%28int%29

My understanding of the servlet spec is that the handler can write anything
it wants to the response stream.  In the event sendError() is called, that
response is discarded.  (As long as flush() was not previously called on the
stream, which I don't think they should be doing anyway.)  So the
transactional error cases end up doing this:

* passes the input and output to the handler
* assuming the events are well formed, handler generates a list of events
and writes a success response to the output
* source attempts to write events to the channel and fails
* source calls sendError() on the output, which aborts any response the
handler may have made and returns a HTTP error code

In my mind, this is actually a feature - clearer separation of concerns.
The handler is only concerned with receiving and generating events.  It
does't know, and doesn't have to know, that something went wrong in the
channel writing.  That is returned as a 500 (server error) or 503 (server
unavailable), which I think actually makes sense...  It's not an error
internal to the handler protocol, it's something in the server (Flume).

I suppose I should have mentioned the "don't call flush()" thing in the doc
update.

What do you think?
This message was sent by Atlassian JIRA
(v6.1.5#6160)
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB