Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> Query regarding adaptor for binary files


Copy link to this message
-
Re: Query regarding adaptor for binary files
Howdy.

Yes, the chunk size is configurable. I think
chukwaAgent.fileTailingAdaptor.maxReadSize is the option you want to
change.

Chukwa should not alter any of your data and should send every byte at
least once, appropriately tagged so that you can recover the exact
original file.

--Ari

On Fri, Feb 4, 2011 at 6:28 AM, Arati Mahimane
<[EMAIL PROTECTED]> wrote:
>
> Hi All,
>
> I want to send Binary files (larger than 10 MB) from Chukwa agent to
> collector.
> I have the following queries:
>
> 1) I read that the filetailer.LWFTAdaptor  repeatedly tails a file, treating
> the file as a sequence of bytes, ignoring the content. But the Chunk
> boundaries are arbitrary. Is there any adaptor in which we can specify the
> chunk size ? If yes, how can the chunk size be configured?
> (e.g. I have a 10 MB file and need to divide it into 10 chunks of 1 MB each
> and send each chunk from agent to collector)
>
> 2) When binary files are sent, does chukwa ensure that they are not
> corrupted in anyway (i.e. Do the contents of the file remain unaltered) ?
>
> Any pointers ?
>
> Thanks,
> Arati Mahimane
>
> --
> View this message in context: http://apache-chukwa.679492.n3.nabble.com/Query-regarding-adaptor-for-binary-files-tp2424540p2424540.html
> Sent from the Chukwa - Users mailing list archive at Nabble.com.
>

--
Ari Rabkin [EMAIL PROTECTED]
UC Berkeley Computer Science Department
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB