Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Re: Translate this to english :)


Copy link to this message
-
Re: Translate this to english :)
Roshan Naik 2013-08-28, 23:30
1)  flume-ng is  basically all versions of flume >= 1.0.  It is a
completely new architecture compared to older flume (<=1.0).  NG = new
generation.
Older flume, sometimes also called flume OG, is deprecated and unsupported.
As far as i can tell, it was never released as an official apache project.
 flume-ng does not support node_watch option.

2) Looks like an OG thing
On Wed, Aug 28, 2013 at 2:49 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:

> Hi flume  (cc'd bigtop also)....
>
> I've been playing with the bigtop tests, and this snippet is particularly
> difficult to
> understand for a flume novice :
>
>   private void compressionCommonTest(String id, String decompress, String
> glob) {
>     String node_config > "node:text(\"events.txt\")|collectorSink(\"${hdfs_sink_dir}\",\"data\");";
>
>     sh.exec("export FLUME_CONF_DIR=./${id}",
>             "flume node_nowatch -s -1 -n node -c '${node_config}'");
>     assertEquals("Flume failed to accept events",
>                  0, sh.ret);
>
>     sh.exec("hadoop fs -cat ${hdfs_sink_dir}/${glob} | ${decompress} | wc
> -l");
>     assertEquals("Wrong # of lines in output found at ${hdfs_sink_dir}",
>                  "10000", sh.out[0]);
>   }
>
> Can someone explain the logic of it to me? In particular...
>
> 1) What is the difference between flume and flume ng - and does "flume_ng"
> support these options (i.e. node_nowatch) ?
>
> 2) What is the "node_config |" syntax accomplishing?
>
> The entire Flume bigtop smoke test can be found here:
>
>
> https://github.com/apache/bigtop/tree/master/bigtop-tests/test-artifacts/flume/src/main/groovy/org/apache/bigtop/itest/flumesmoke
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.