Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> Query regarding adaptor for binary files

Arati Mahimane 2011-02-04, 14:28
Copy link to this message
Re: Query regarding adaptor for binary files

Yes, the chunk size is configurable. I think
chukwaAgent.fileTailingAdaptor.maxReadSize is the option you want to

Chukwa should not alter any of your data and should send every byte at
least once, appropriately tagged so that you can recover the exact
original file.


On Fri, Feb 4, 2011 at 6:28 AM, Arati Mahimane
> Hi All,
> I want to send Binary files (larger than 10 MB) from Chukwa agent to
> collector.
> I have the following queries:
> 1) I read that the filetailer.LWFTAdaptor  repeatedly tails a file, treating
> the file as a sequence of bytes, ignoring the content. But the Chunk
> boundaries are arbitrary. Is there any adaptor in which we can specify the
> chunk size ? If yes, how can the chunk size be configured?
> (e.g. I have a 10 MB file and need to divide it into 10 chunks of 1 MB each
> and send each chunk from agent to collector)
> 2) When binary files are sent, does chukwa ensure that they are not
> corrupted in anyway (i.e. Do the contents of the file remain unaltered) ?
> Any pointers ?
> Thanks,
> Arati Mahimane
> --
> View this message in context: http://apache-chukwa.679492.n3.nabble.com/Query-regarding-adaptor-for-binary-files-tp2424540p2424540.html
> Sent from the Chukwa - Users mailing list archive at Nabble.com.

UC Berkeley Computer Science Department