| clear query|facets|time |
Search criteria: .
Results from 21 to 30 from
53 (0.081s).
|
|
|
Loading phrases to help you
refine your search...
|
|
FileChannel - WARN on restart - Flume - [mail # dev]
|
|
...When I restart a flume agent with a file channel (ctrl^c to stop the process), I get the following warning when the channel is empty (not sure what happens otherwise): Checkpoint for f...
|
|
|
Author: Connor Woodson,
2013-01-19, 06:26
|
|
|
Log4J Appender in Flume - Flume - [mail # user]
|
|
...I just ran into an unfortunate configuration issue; I want to use flume's log4j appender as part of the agent itself in order to send error logs to a monitoring system. However, when that sy...
|
|
|
Author: Connor Woodson,
2013-01-18, 19:13
|
|
|
Re: Returns 127.0.0.1 at HostInterceptor - Flume - [mail # dev]
|
|
...We could use InetAddress.getLocalHost().getCanonicalHostName() to get just the IP, which I feel is good enough; a quick and dirty interceptor could be built that substitutes certain names fo...
|
|
|
Author: Connor Woodson,
2013-01-18, 09:39
|
|
|
Re: Uncaught Exception When Using Spooling Directory Source - Flume - [mail # user]
|
|
...The Spooling Directory Source is best used for sending old data / backups through Flume, as opposed to trying to use it for realtime data (due to, as you discovered, you aren't supposed to w...
|
|
|
Author: Connor Woodson,
2013-01-18, 09:13
|
|
|
Re: hdfs.idleTimeout ,what's it used for ? - Flume - [mail # user]
|
|
...Alright, that makes sense. The takeaway from this conversation for everyone else: If you use idleTimeout, be sure to set the rollInterval to 0. And if you don't use idleTimeout, be sur...
|
|
|
Author: Connor Woodson,
2013-01-18, 04:18
|
|
|
Re: hdfs.idleTimeout ,what's it used for ? - Flume - [mail # user]
|
|
...The way idleTimeout works right now is that it's another rollInterval; it will work best when rollInterval is not set and so it seems that it's use is best for when you don't want to use a r...
|
|
|
Author: Connor Woodson,
2013-01-18, 03:24
|
|
|
Re: hdfs.idleTimeout ,what's it used for ? - Flume - [mail # user]
|
|
...Ya, I read this first; I find the implementation of the idleTimeout slightly odd that it doesn't persist through the file closing. On Thu, Jan 17, 2013 at 6:39 PM, Juhani Connolly &nb...
|
|
|
Author: Connor Woodson,
2013-01-18, 02:42
|
|
|
Re: hdfs.idleTimeout ,what's it used for ? - Flume - [mail # user]
|
|
...And @ my aside: I hadn't realized that the idleTimeout is canceled by the rollInterval occurring. That's annoying. So setting a lower idleTimeout, and drastically decreasing maxOpenFiles to ...
|
|
|
Author: Connor Woodson,
2013-01-18, 02:23
|
|
|
Re: hdfs.idleTimeout ,what's it used for ? - Flume - [mail # user]
|
|
...@Mohit: For the HDFS Sink, the tmp files are placed based on the hadoop.tmp.dir property. The default location is /tmp/hadoop-${user.name} To change this you can add -Dhadoop.tmp.dir= ...
|
|
|
Author: Connor Woodson,
2013-01-18, 02:20
|
|
|
Re: hdfs.idleTimeout ,what's it used for ? - Flume - [mail # user]
|
|
...Whether idleTimeout is lower or higher than rollInterval is a preference; set it before, and assume you get one message right on the turn of the hour, then you will have some part of that ho...
|
|
|
Author: Connor Woodson,
2013-01-18, 02:19
|
|
|
|
