Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - lzo file rolling on Cloudera Flume


Copy link to this message
-
Re: lzo file rolling on Cloudera Flume
Hari Shreedharan 2013-09-25, 18:11
Can you try setting hdfs.idleTimeout to a small value (much smaller than roll timeout) and then see if the file gets rolled correctly?
Thanks,
Hari
On Wednesday, September 25, 2013 at 9:22 AM, Edwin Chiu wrote:

> Hi,
>
> I have also observed this behavior with gzip and bz2. I'm using flume-1.3.1.1.3.0.0-107.el6.noarch under Hortonworks 1.3.
>
> Best,
>
>
> - Edwin
>
> On Wed, Sep 25, 2013 at 2:51 AM, Devendra Tagare <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > Hi,
> >
> > I am using Cloudera Flume (apache-flume-1.3.0-cdh4.3.1)
> >
> > I have observed a large number of *.lzo.tmp files not getting rolled even though their file sizes are non-zero and the roll time has elapsed
> >
> > Regards,
> > Dev