Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - [winows] tracker file sharing voilation wne trying to delete tracker file


Copy link to this message
-
RE: [winows] tracker file sharing voilation wne trying to delete tracker file
Phil Scala 2013-04-13, 00:21
Thanks Hari, will do.

Phil

Phil Scala
Developer / Architect
Global Relay

[EMAIL PROTECTED]

866.484.6630  |  [EMAIL PROTECTED]  |  globalrelay.com 

-----Original Message-----
From: Hari Shreedharan [mailto:[EMAIL PROTECTED]]
Sent: April-12-13 5:20 PM
To: [EMAIL PROTECTED]
Subject: Re: [winows] tracker file sharing voilation wne trying to delete tracker file

Hi Phil,

Thank you for reporting this. I believe Paul Chavez reported the same issue earlier today on the user@ list. Please file a jira and attach the patch.
Someone will review and commit it!

Thanks,
Hari
On Fri, Apr 12, 2013 at 5:07 PM, Phil Scala <[EMAIL PROTECTED]>wrote:

> Hi,
>
> I am new to Flume, was trying out some basic's on my Windows 7 dev
> machine and I ran into problems with the spooling directory source,
> the issue was in the
> org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFi
> le() method... we were getting an instance of a DurablePositionTracker
> and not calling close on it before then creating another instance...
> thus not calling close on the underlying DataFileWriter.  This left
> the file locked in Windows and hence any delete attempts were
> failing...causing the Agent to spin around and around trying to delete
> this file and creating endless numbers of the temporary tracker files.
>
> I have the code change here and can submit a JIRA item, I don't see
> anything similar in JIRA, but before I created something I wanted to
> discuss this in the community (realizing that Windows is not
> supported)
>
> My code change is very simple, I added the close() call in the else block :
>
> org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFi
> le()
> ....
>
> if (!tracker.getTarget().equals(nextPath)) {
>           tracker.close();
>           deleteMetaFile();
>         }
>         else{
>               //Explicitly close the tracker instance received so that
> //there is no sharing violation on Windows.
>               tracker.close();
>         }
>         tracker = DurablePositionTracker.getInstance(metaFile,
> nextPath); ....
>
> Thanks
>   Phil
>
> Phil Scala
> Developer / Architect
> Global Relay
>
> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
>
> 866.484.6630
> New York  |  Chicago  |  Vancouver  |  London  (+44.0800.032.9829)  |  
> Singapore  (+65.3158.1301) Global Relay Archive supports email,
> instant messaging, BlackBerry, Bloomberg, Thomson Reuters, Pivot,
> YellowJacket, LinkedIn, Twitter, Facebook and more.
>
> Ask about Global Relay
> Message<http://www.globalrelay.com/services/message>
> - The Future of Collaboration in the Financial Services World
>
> All email sent to or from this address will be retained by Global
> Relay's email archiving system. This message is intended only for the
> use of the individual or entity to which it is addressed, and may
> contain information that is privileged, confidential, and exempt from
> disclosure under applicable law.  Global Relay will not be liable for
> any compliance or technical information provided herein.  All
> trademarks are the property of their respective owners.
>
>