Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - profiling hdfs write path

Copy link to this message
Re: profiling hdfs write path
Radim Kolar 2012-11-26, 03:07

> Currently it's CPU-intensive for several reasons:
> 1) It doesn't yet use the native CRC code
> 2) It makes several unnecessary copies and byte buffer allocations, both in
> the client and in the DataNode
> There are open JIRAs for these, and I have a preliminary patch which helped
> a lot, but it hasn't been high priority.
can you attach crc path there?
i will finish it.