Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> C Implementation, missing avro_flush()


+
Niraj Tolia 2010-03-17, 19:21
+
Bruce Mitchener 2010-03-17, 20:47
Copy link to this message
-
Re: C Implementation, missing avro_flush()
Niraj,

So, I've been looking at this.

I have a patch ready to roll for the first thing (dealing with
avro_flush()).  I'll go ahead and open a bug about that and attach the patch
for massie to commit.

However, for fsync(), I think we'd like to wait until 1.4 to address this as
it will lead a confusing API at the moment that we'll just have to remove in
short order.

Right now, the container files are implemented with buffered I/O, so you'd
have to call 3 functions to ensure that it hit disk.  I think we can get
away without buffered I/O and that would make it a more reasonable 2
functions.

Do you have an overwhelming need for fsync functionality on container files
(datafile.c stuff) in the next month or two?

 - Bruce

On Wed, Mar 17, 2010 at 2:47 PM, Bruce Mitchener
<[EMAIL PROTECTED]>wrote:

> Niraj,
>
> The header says avro_flush(), but the implementation says
> avro_writer_flush().  We'll get this addressed shortly.
>
> There isn't currently a way to call fsync() directly ... but since you pass
> the FILE* to the file writer, you could call fsync(fileno(FILE*)) on your
> own, unless you're using the container file.
>
> If you want to open a bug on each of these, I'll work up the patches and
> work with massie to get them into SVN.
>
> Cheers,
>
>  - Bruce
>
>
> On Wed, Mar 17, 2010 at 1:21 PM, Niraj Tolia <[EMAIL PROTECTED]> wrote:
>
>> I was going through the avro.h header file (from the 1.3.0 release)
>> and noticed that the avro_flush() call is defined but has no
>> implementation. If someone does try to use it, compilation will fail
>> with an undefined reference error. I am not sure if this was an
>> accidental oversight but figured I should let someone know.
>>
>> Also, would I be correct in assuming that the C API doesn't allow me
>> to actually call fsync() on a file writer? Digging through the code
>> didn't turn up anything obvious.
>>
>> Cheers,
>> Niraj
>>
>> --
>> http://www.tolia.org/
>>
>
>
+
Bruce Mitchener 2010-03-17, 22:19
+
Matt Massie 2010-03-17, 23:27
+
Niraj Tolia 2010-03-17, 23:35
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB