Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Custom sink - "close() called when transaction is OPEN" error


+
Andrew Jones 2012-11-14, 16:55
+
Roshan Naik 2012-11-14, 19:57
+
Hari Shreedharan 2012-11-14, 20:02
Copy link to this message
-
Re: Custom sink - "close() called when transaction is OPEN" error
Andrew Jones 2012-11-15, 11:24
We are using Flume 1.2.0. We have a custom source, although it passes
through an Avro Sink and Source before getting to the sink. We are now
using the memory channel, although had just switched from the JDBC channel
when we started seeing these errors, so maybe that's something to do with
it?

I tried wrapping transaction.rollback(); in a try catch and logging in the
catch, but it wasn't called, so I don't think the rollback is throwing an
error.

I think it may have something to do with switching channels, as right after
Flume reloaded the config we started getting errors. I have restarted the
flume node manually and we are still getting the error.

Thanks,
Andrew
On 14 November 2012 20:02, Hari Shreedharan <[EMAIL PROTECTED]>wrote:

> Which version of Flume are you using? It looks like the transaction was
> never rolled back or committed. It is likely that the rollback method
> too threw some exception, and the rollback was not successful. Also, what
> channel are you using?
>
>
> Thanks,
> Hari
>
> --
> Hari Shreedharan
>
> On Wednesday, November 14, 2012 at 8:55 AM, Andrew Jones wrote:
>
> Hi,
>
> I have a custom sink which has been working fine, but recently I have
> started seeing this error in the logs:
>
> Unable to deliver event. Exception follows.
> java.lang.IllegalStateException: close() called when transaction is OPEN -
> you must either commit or rollback first
>         at
> com.google.common.base.Preconditions.checkState(Preconditions.java:176)
> ...
>
>
> After having a google and finding
> https://issues.apache.org/jira/browse/FLUME-1089, I have double checked I
> am using the correct try, catch, finally idiom that other sinks use, and I
> seem to be doing the same. I do the following:
>
> public Status process() throws EventDeliveryException {
> Status status = Status.READY;
>
> Channel channel = getChannel();
>  Transaction transaction = channel.getTransaction();
>
> try {
> transaction.begin();
>
>                         // does a bit of processing and
>                         // writes out the event to MongoDB
>
>                         transaction.commit();
>
> } catch (Throwable t) {
> transaction.rollback();
>
> if (t instanceof Error) {
>  throw (Error) t;
> } else if  (t instanceof EventDeliveryException) {
> throw (EventDeliveryException) t;
>  } else if (t instanceof ChannelException) {
> logger.error("Brodie Log Sink " + getName() + ": Unable to get event from"
> +
>  " channel " + channel.getName() + ". Exception follows.", t);
> status = Status.BACKOFF;
>  } else {
> throw new EventDeliveryException("Failed to send events", t);
> }
>  } finally {
> transaction.close();
> }
>
> return status;
> }
>
> }
>
> All of this code came from looking at other sinks (Avro and HDFS), so I am
> pretty sure its correct.
>
> Can anyone see anything that might be a problem, or is there anything else
> I can do to avoid this error?
>
> Thanks,
> Andrew
>
>
>
+
Brock Noland 2012-11-15, 12:50
+
Roshan Naik 2012-11-15, 19:18
+
Andrew Jones 2012-11-16, 15:21
+
Brock Noland 2012-11-16, 15:32
+
Andrew Jones 2012-11-16, 15:45
+
Brock Noland 2012-11-16, 16:02
+
Andrew Jones 2012-11-16, 16:17