hi all,

   I tested Flume in last week with ScribeSource(
https://issues.apache.org/jira/browse/FLUME-1382) and HDFS Sink. More
detailed conditions and deployment cases listed below. Too many 'Full GC'
impact the throughput and amount of events promoted into old generation. I
have applied some tuning methods, no much effect.

   Could someone give me your feedback or tip to reduce the GC problem?
Wish your attention.
PS: Using Mike's report template at
https://cwiki.apache.org/FLUME/flume-ng-performance-measurements.html

*
*

*Flume Performance Test 2012-07-25*

*Overview*

The Flume agent was run on its own physical machine in a single JVM. A
separate client machine generated load against the Flume box in
List<LogEntry> format. Flume stored data onto a 4-node HDFS cluster
configured on its own separate hardware. No virtual machines were used in
this test.

*Hardware specs*

CPU: Inter Xeon L5640 2 x quad-core @ 2.27 GHz (12 physical cores)

Memory: 16 GB

OS: CentOS release 5.3 (Final)

*Flume configuration*

JAVA Version: 1.6.0_20 (Java HotSpot 64-Bit Server VM)

JAVA OPTS: -Xms1024m -Xmx4096m -XX:PermSize=256m -XX:NewRatio=1
-XX:SurvivorRatio=5 -XX:InitialTenuringThreshold=15
-XX:MaxTenuringThreshold=31 -XX:PretenureSizeThreshold=4096

Num. agents: 1

Num. parallel flows: 5

Source: ScribeSource

Channel: MemoryChannel

Sink: HDFSEventSink

Selector: RandomSelector

*Config-file*

# list sources, channels, sinks for the agent

agent.sources = seqGenSrc

agent.channels = mc1 mc2 mc3 mc4 mc5

agent.sinks = hdfsSin1 hdfsSin2 hdfsSin3 hdfsSin4 hdfsSin5

# define sources

agent.sources.seqGenSrc.type = org.apache.flume.source.scribe.ScribeSource

agent.sources.seqGenSrc.selector.type = io.flume.RandomSelector

# define sinks

agent.sinks.hdfsSin1.type = hdfs

agent.sinks.hdfsSin1.hdfs.path = /flume_test/data1/

agent.sinks.hdfsSin1.hdfs.rollInterval = 300

agent.sinks.hdfsSin1.hdfs.rollSize = 0

agent.sinks.hdfsSin1.hdfs.rollCount = 1000000

agent.sinks.hdfsSin1.hdfs.batchSize = 10000

agent.sinks.hdfsSin1.hdfs.fileType = DataStream

agent.sinks.hdfsSin1.hdfs.txnEventMax = 1000

# ... define sink #2 #3 #4 #5 ...

# define channels

agent.channels.mc1.type = memory

agent.channels.mc1.capacity = 1000000

agent.channels.mc1.transactionCapacity = 1000

# ... define channel #2 #3 #4 #5 ...

# specify the channel each sink and source should use

agent.sources.seqGenSrc.channels = mc1 mc2 mc3 mc4 mc5

agent.sinks.hdfsSin1.channel = mc1

# ... specify sink #2 #3 #4 #5 ...

*Hadoop configuration*

The HDFS sink was connected to a 4-node Hadoop cluster running CDH3u1. For
different HDFS sink, HDFS wrote data into different path.

*Visualization of test setup*

https://lh3.googleusercontent.com/dGumq1pu1Wr3Bj8WJmRHOoLWmUlGqxC4wW7_XCNO9R1wuh15LRXaKKxGoccpjBXtgqcdSVW-vtg

There are 10 Scribe Clients and each client send 20 million LogEntry
objects to ScribleSource.

*Data description*

List<LogEntry> entries containing a string category and a ByteArray body.
The ByteArray body size is 500 bytes.

*Results*

Throughput:

Average:       Source: 46.4 MB/s, Sink: 45.2 MB/s

Maximum:    Source: 67.1 MB/s, Sink: 88.3 MB/s

CPU:       Average: 196%, Maximum: 440%

GC:         Young GC: 1636 times,      Full GC: 384 times
No data loss.

*Heap and GC*

By analyzing JVM Heap, we found that there are many LogEntry objects in
OldGen. We have tried to carry out some optimizations, but the results are
not satisfactory. We will continue to track this limitation.

FullGC Log examples:

[Full GC [PSYoungGen: 1497984K->0K(1797568K)] [PSOldGen:
1720643K->1693741K(2097152K)] 3218627K->1693741K(3894720K) [PSPermGen:
14566K->14566K(262144K)], 5.0027700 secs] [Times: user=5.01 sys=0.00,
real=5.00 secs]

[Full GC [PSYoungGen: 1497960K->0K(1797568K)] [PSOldGen:
1693805K->1752540K(2097152K)] 3191765K->1752540K(3894720K) [PSPermGen:
14571K->14571K(262144K)], 5.0732570 secs] [Times: user=5.07 sys=0.00,
real=5.07 secs]

[Full GC [PSYoungGen: 1497984K->0K(1797568K)] [PSOldGen:
1752540K->1642553K(2097152K)] 3250524K->1642553K(3894720K) [PSPermGen:
14572K->14568K(262144K)], 5.0710730 secs] [Times: user=5.07 sys=0.01,
real=5.08 secs]

－Regards

Denny Ye

Hey Denny,

thanks for the report.

Can you please try to rerun with:

JAVA_OPTS="-Xms200m -Xmx200m -Xmn32m -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -Xss128k
-XX:+UseMembar -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-Xloggc:/var/log/flume/gc.log"

Plese attach the gc.log after.

cheers,
Alex

Am 25.07.2012 10:35, schrieb Denny Ye:
> hi all,
>
>    I tested Flume in last week with ScribeSource(
> https://issues.apache.org/jira/browse/FLUME-1382) and HDFS Sink. More
> detailed conditions and deployment cases listed below. Too many 'Full GC'
> impact the throughput and amount of events promoted into old generation. I
> have applied some tuning methods, no much effect.
>
>    Could someone give me your feedback or tip to reduce the GC problem?
> Wish your attention.
>
>
> PS: Using Mike's report template at
> https://cwiki.apache.org/FLUME/flume-ng-performance-measurements.html
>
> *
> *
>
> *Flume Performance Test 2012-07-25*
>
> *Overview*
>
> The Flume agent was run on its own physical machine in a single JVM. A
> separate client machine generated load against the Flume box in
> List<LogEntry> format. Flume stored data onto a 4-node HDFS cluster
> configured on its own separate hardware. No virtual machines were used in
> this test.
>
> *Hardware specs*
>
> CPU: Inter Xeon L5640 2 x quad-core @ 2.27 GHz (12 physical cores)
>
> Memory: 16 GB
>
> OS: CentOS release 5.3 (Final)
>
> *Flume configuration*
>
> JAVA Version: 1.6.0_20 (Java HotSpot 64-Bit Server VM)
>
> JAVA OPTS: -Xms1024m -Xmx4096m -XX:PermSize=256m -XX:NewRatio=1
> -XX:SurvivorRatio=5 -XX:InitialTenuringThreshold=15
> -XX:MaxTenuringThreshold=31 -XX:PretenureSizeThreshold=4096
>
> Num. agents: 1
>
> Num. parallel flows: 5
>
> Source: ScribeSource
>
> Channel: MemoryChannel
>
> Sink: HDFSEventSink
>
> Selector: RandomSelector
>
> *Config-file*
>
> # list sources, channels, sinks for the agent
>
> agent.sources = seqGenSrc
>
> agent.channels = mc1 mc2 mc3 mc4 mc5
>
> agent.sinks = hdfsSin1 hdfsSin2 hdfsSin3 hdfsSin4 hdfsSin5
>
>
>
> # define sources
>
> agent.sources.seqGenSrc.type = org.apache.flume.source.scribe.ScribeSource
>
> agent.sources.seqGenSrc.selector.type = io.flume.RandomSelector
>
>
>
> # define sinks
>
> agent.sinks.hdfsSin1.type = hdfs
>
> agent.sinks.hdfsSin1.hdfs.path = /flume_test/data1/
>
> agent.sinks.hdfsSin1.hdfs.rollInterval = 300
>
> agent.sinks.hdfsSin1.hdfs.rollSize = 0
>
> agent.sinks.hdfsSin1.hdfs.rollCount = 1000000
>
> agent.sinks.hdfsSin1.hdfs.batchSize = 10000
>
> agent.sinks.hdfsSin1.hdfs.fileType = DataStream
>
> agent.sinks.hdfsSin1.hdfs.txnEventMax = 1000
>
> # ... define sink #2 #3 #4 #5 ...
>
>
>
> # define channels
>
> agent.channels.mc1.type = memory
>
> agent.channels.mc1.capacity = 1000000
>
> agent.channels.mc1.transactionCapacity = 1000
>
> # ... define channel #2 #3 #4 #5 ...
>
>
>
> # specify the channel each sink and source should use
>
> agent.sources.seqGenSrc.channels = mc1 mc2 mc3 mc4 mc5
>
> agent.sinks.hdfsSin1.channel = mc1
>
> # ... specify sink #2 #3 #4 #5 ...
>
> *Hadoop configuration*
>
> The HDFS sink was connected to a 4-node Hadoop cluster running CDH3u1. For
> different HDFS sink, HDFS wrote data into different path.
>
> *Visualization of test setup*
>
> https://lh3.googleusercontent.com/dGumq1pu1Wr3Bj8WJmRHOoLWmUlGqxC4wW7_XCNO9R1wuh15LRXaKKxGoccpjBXtgqcdSVW-vtg
>
> There are 10 Scribe Clients and each client send 20 million LogEntry
> objects to ScribleSource.
>
> *Data description*
>
> List<LogEntry> entries containing a string category and a ByteArray body.
> The ByteArray body size is 500 bytes.
>
> *Results*
>
> Throughput:
>
> Average:       Source: 46.4 MB/s, Sink: 45.2 MB/s
>
> Maximum:    Source: 67.1 MB/s, Sink: 88.3 MB/s
>
>
>
> CPU:       Average: 196%, Maximum: 440%
>
>
>
> GC:         Young GC: 1636 times,      Full GC: 384 times
>
>
> No data loss.
>
> *Heap and GC*
>
> By analyzing JVM Heap, we found that there are many LogEntry objects in

hi Alex,
    Attachment is preparing for you!
    Long term pause may be the critical problem for us. Do you agree me ?
    Wish your response, thanks!

-Regards
Denny Ye

2012/7/25 alo.alt <[EMAIL PROTECTED]>

> Hey Denny,
>
> thanks for the report.
>
> Can you please try to rerun with:
>
> JAVA_OPTS="-Xms200m -Xmx200m -Xmn32m -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -Xss128k
> -XX:+UseMembar -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
> -Xloggc:/var/log/flume/gc.log"
>
> Plese attach the gc.log after.
>
> cheers,
> Alex
>
> Am 25.07.2012 10:35, schrieb Denny Ye:
> > hi all,
> >
> >    I tested Flume in last week with ScribeSource(
> > https://issues.apache.org/jira/browse/FLUME-1382) and HDFS Sink. More
> > detailed conditions and deployment cases listed below. Too many 'Full GC'
> > impact the throughput and amount of events promoted into old generation.
> I
> > have applied some tuning methods, no much effect.
> >
> >    Could someone give me your feedback or tip to reduce the GC problem?
> > Wish your attention.
> >
> >
> > PS: Using Mike's report template at
> > https://cwiki.apache.org/FLUME/flume-ng-performance-measurements.html
> >
> > *
> > *
> >
> > *Flume Performance Test 2012-07-25*
> >
> > *Overview*
> >
> > The Flume agent was run on its own physical machine in a single JVM. A
> > separate client machine generated load against the Flume box in
> > List<LogEntry> format. Flume stored data onto a 4-node HDFS cluster
> > configured on its own separate hardware. No virtual machines were used in
> > this test.
> >
> > *Hardware specs*
> >
> > CPU: Inter Xeon L5640 2 x quad-core @ 2.27 GHz (12 physical cores)
> >
> > Memory: 16 GB
> >
> > OS: CentOS release 5.3 (Final)
> >
> > *Flume configuration*
> >
> > JAVA Version: 1.6.0_20 (Java HotSpot 64-Bit Server VM)
> >
> > JAVA OPTS: -Xms1024m -Xmx4096m -XX:PermSize=256m -XX:NewRatio=1
> > -XX:SurvivorRatio=5 -XX:InitialTenuringThreshold=15
> > -XX:MaxTenuringThreshold=31 -XX:PretenureSizeThreshold=4096
> >
> > Num. agents: 1
> >
> > Num. parallel flows: 5
> >
> > Source: ScribeSource
> >
> > Channel: MemoryChannel
> >
> > Sink: HDFSEventSink
> >
> > Selector: RandomSelector
> >
> > *Config-file*
> >
> > # list sources, channels, sinks for the agent
> >
> > agent.sources = seqGenSrc
> >
> > agent.channels = mc1 mc2 mc3 mc4 mc5
> >
> > agent.sinks = hdfsSin1 hdfsSin2 hdfsSin3 hdfsSin4 hdfsSin5
> >
> >
> >
> > # define sources
> >
> > agent.sources.seqGenSrc.type > org.apache.flume.source.scribe.ScribeSource
> >
> > agent.sources.seqGenSrc.selector.type = io.flume.RandomSelector
> >
> >
> >
> > # define sinks
> >
> > agent.sinks.hdfsSin1.type = hdfs
> >
> > agent.sinks.hdfsSin1.hdfs.path = /flume_test/data1/
> >
> > agent.sinks.hdfsSin1.hdfs.rollInterval = 300
> >
> > agent.sinks.hdfsSin1.hdfs.rollSize = 0
> >
> > agent.sinks.hdfsSin1.hdfs.rollCount = 1000000
> >
> > agent.sinks.hdfsSin1.hdfs.batchSize = 10000
> >
> > agent.sinks.hdfsSin1.hdfs.fileType = DataStream
> >
> > agent.sinks.hdfsSin1.hdfs.txnEventMax = 1000
> >
> > # ... define sink #2 #3 #4 #5 ...
> >
> >
> >
> > # define channels
> >
> > agent.channels.mc1.type = memory
> >
> > agent.channels.mc1.capacity = 1000000
> >
> > agent.channels.mc1.transactionCapacity = 1000
> >
> > # ... define channel #2 #3 #4 #5 ...
> >
> >
> >
> > # specify the channel each sink and source should use
> >
> > agent.sources.seqGenSrc.channels = mc1 mc2 mc3 mc4 mc5
> >
> > agent.sinks.hdfsSin1.channel = mc1
> >
> > # ... specify sink #2 #3 #4 #5 ...
> >
> > *Hadoop configuration*
> >
> > The HDFS sink was connected to a 4-node Hadoop cluster running CDH3u1.
> For
> > different HDFS sink, HDFS wrote data into different path.
> >
> > *Visualization of test setup*
> >
> >
> https://lh3.googleusercontent.com/dGumq1pu1Wr3Bj8WJmRHOoLWmUlGqxC4wW7_XCNO9R1wuh15LRXaKKxGoccpjBXtgqcdSVW-vtg
> >
> > There are 10 Scribe Clients and each client send 20 million LogEntry
> > objects to ScribleSource.

Hey Denny,

Please set XMX and XMS as the same (4GB here), because on high traffic
sinks it could be that the allocation process can cause a gc sweep.
Also I would recommend to define the gen sizes with:
-XX:NewSize=64m -XX:MaxNewSize=64m

cheers,
 Alex

On Mittwoch, 25. Juli 2012 11:48:22, Denny Ye wrote:
> hi Alex,
>     Attachment is preparing for you!
>     Long term pause may be the critical problem for us. Do you agree me ?
>     Wish your response, thanks!
>
> -Regards
> Denny Ye
>
> 2012/7/25 alo.alt <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
>
>     Hey Denny,
>
>     thanks for the report.
>
>     Can you please try to rerun with:
>
>     JAVA_OPTS="-Xms200m -Xmx200m -Xmn32m -XX:+UseParNewGC
>     -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -Xss128k
>     -XX:+UseMembar -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
>     -Xloggc:/var/log/flume/gc.log"
>
>     Plese attach the gc.log after.
>
>     cheers,
>     Alex
>
>     Am 25.07.2012 10 <tel:25.07.2012%2010>:35, schrieb Denny Ye:
>     > hi all,
>     >
>     >    I tested Flume in last week with ScribeSource(
>     > https://issues.apache.org/jira/browse/FLUME-1382) and HDFS Sink.
>     More
>     > detailed conditions and deployment cases listed below. Too many
>     'Full GC'
>     > impact the throughput and amount of events promoted into old
>     generation. I
>     > have applied some tuning methods, no much effect.
>     >
>     >    Could someone give me your feedback or tip to reduce the GC
>     problem?
>     > Wish your attention.
>     >
>     >
>     > PS: Using Mike's report template at
>     >
>     https://cwiki.apache.org/FLUME/flume-ng-performance-measurements.html
>     >
>     > *
>     > *
>     >
>     > *Flume Performance Test 2012-07-25*
>     >
>     > *Overview*
>     >
>     > The Flume agent was run on its own physical machine in a single
>     JVM. A
>     > separate client machine generated load against the Flume box in
>     > List<LogEntry> format. Flume stored data onto a 4-node HDFS cluster
>     > configured on its own separate hardware. No virtual machines
>     were used in
>     > this test.
>     >
>     > *Hardware specs*
>     >
>     > CPU: Inter Xeon L5640 2 x quad-core @ 2.27 GHz (12 physical cores)
>     >
>     > Memory: 16 GB
>     >
>     > OS: CentOS release 5.3 (Final)
>     >
>     > *Flume configuration*
>     >
>     > JAVA Version: 1.6.0_20 (Java HotSpot 64-Bit Server VM)
>     >
>     > JAVA OPTS: -Xms1024m -Xmx4096m -XX:PermSize=256m -XX:NewRatio=1
>     > -XX:SurvivorRatio=5 -XX:InitialTenuringThreshold=15
>     > -XX:MaxTenuringThreshold=31 -XX:PretenureSizeThreshold=4096
>     >
>     > Num. agents: 1
>     >
>     > Num. parallel flows: 5
>     >
>     > Source: ScribeSource
>     >
>     > Channel: MemoryChannel
>     >
>     > Sink: HDFSEventSink
>     >
>     > Selector: RandomSelector
>     >
>     > *Config-file*
>     >
>     > # list sources, channels, sinks for the agent
>     >
>     > agent.sources = seqGenSrc
>     >
>     > agent.channels = mc1 mc2 mc3 mc4 mc5
>     >
>     > agent.sinks = hdfsSin1 hdfsSin2 hdfsSin3 hdfsSin4 hdfsSin5
>     >
>     >
>     >
>     > # define sources
>     >
>     > agent.sources.seqGenSrc.type >     org.apache.flume.source.scribe.ScribeSource
>     >
>     > agent.sources.seqGenSrc.selector.type = io.flume.RandomSelector
>     >
>     >
>     >
>     > # define sinks
>     >
>     > agent.sinks.hdfsSin1.type = hdfs
>     >
>     > agent.sinks.hdfsSin1.hdfs.path = /flume_test/data1/
>     >
>     > agent.sinks.hdfsSin1.hdfs.rollInterval = 300
>     >
>     > agent.sinks.hdfsSin1.hdfs.rollSize = 0
>     >
>     > agent.sinks.hdfsSin1.hdfs.rollCount = 1000000
>     >
>     > agent.sinks.hdfsSin1.hdfs.batchSize = 10000
>     >
>     > agent.sinks.hdfsSin1.hdfs.fileType = DataStream
>     >
>     > agent.sinks.hdfsSin1.hdfs.txnEventMax = 1000
>     >
>     > # ... define sink #2 #3 #4 #5 ...
>     >
>     >
>     >
>     > # define channels

hi Alex
   What you got has been set with 4GB heap

-Regards
Denny Ye

2012/7/25 alo.alt <[EMAIL PROTECTED]>

> Hey Denny,
>
> Please set XMX and XMS as the same (4GB here), because on high traffic
> sinks it could be that the allocation process can cause a gc sweep. Also I
> would recommend to define the gen sizes with:
> -XX:NewSize=64m -XX:MaxNewSize=64m
>
> cheers,
> Alex
>
>
>
>
> On Mittwoch, 25. Juli 2012 11:48:22, Denny Ye wrote:
>
>> hi Alex,
>>     Attachment is preparing for you!
>>     Long term pause may be the critical problem for us. Do you agree me ?
>>     Wish your response, thanks!
>>
>> -Regards
>> Denny Ye
>>
>> 2012/7/25 alo.alt <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
>>
>>
>>     Hey Denny,
>>
>>     thanks for the report.
>>
>>     Can you please try to rerun with:
>>
>>     JAVA_OPTS="-Xms200m -Xmx200m -Xmn32m -XX:+UseParNewGC
>>     -XX:+UseConcMarkSweepGC -XX:**CMSInitiatingOccupancyFraction**=70
>> -Xss128k
>>     -XX:+UseMembar -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
>>     -Xloggc:/var/log/flume/gc.log"
>>
>>     Plese attach the gc.log after.
>>
>>     cheers,
>>     Alex
>>
>>     Am 25.07.2012 10 <tel:25.07.2012%2010>:35, schrieb Denny Ye:
>>
>>     > hi all,
>>     >
>>     >    I tested Flume in last week with ScribeSource(
>>     > https://issues.apache.org/**jira/browse/FLUME-1382<https://issues.apache.org/jira/browse/FLUME-1382>)
>> and HDFS Sink.
>>     More
>>     > detailed conditions and deployment cases listed below. Too many
>>     'Full GC'
>>     > impact the throughput and amount of events promoted into old
>>     generation. I
>>     > have applied some tuning methods, no much effect.
>>     >
>>     >    Could someone give me your feedback or tip to reduce the GC
>>     problem?
>>     > Wish your attention.
>>     >
>>     >
>>     > PS: Using Mike's report template at
>>     >
>>     https://cwiki.apache.org/**FLUME/flume-ng-performance-**
>> measurements.html<https://cwiki.apache.org/FLUME/flume-ng-performance-measurements.html>
>>     >
>>     > *
>>     > *
>>     >
>>     > *Flume Performance Test 2012-07-25*
>>     >
>>     > *Overview*
>>     >
>>     > The Flume agent was run on its own physical machine in a single
>>     JVM. A
>>     > separate client machine generated load against the Flume box in
>>     > List<LogEntry> format. Flume stored data onto a 4-node HDFS cluster
>>     > configured on its own separate hardware. No virtual machines
>>     were used in
>>     > this test.
>>     >
>>     > *Hardware specs*
>>     >
>>     > CPU: Inter Xeon L5640 2 x quad-core @ 2.27 GHz (12 physical cores)
>>     >
>>     > Memory: 16 GB
>>     >
>>     > OS: CentOS release 5.3 (Final)
>>     >
>>     > *Flume configuration*
>>     >
>>     > JAVA Version: 1.6.0_20 (Java HotSpot 64-Bit Server VM)
>>     >
>>     > JAVA OPTS: -Xms1024m -Xmx4096m -XX:PermSize=256m -XX:NewRatio=1
>>     > -XX:SurvivorRatio=5 -XX:InitialTenuringThreshold=**15
>>     > -XX:MaxTenuringThreshold=31 -XX:PretenureSizeThreshold=**4096
>>     >
>>     > Num. agents: 1
>>     >
>>     > Num. parallel flows: 5
>>     >
>>     > Source: ScribeSource
>>     >
>>     > Channel: MemoryChannel
>>     >
>>     > Sink: HDFSEventSink
>>     >
>>     > Selector: RandomSelector
>>     >
>>     > *Config-file*
>>     >
>>     > # list sources, channels, sinks for the agent
>>     >
>>     > agent.sources = seqGenSrc
>>     >
>>     > agent.channels = mc1 mc2 mc3 mc4 mc5
>>     >
>>     > agent.sinks = hdfsSin1 hdfsSin2 hdfsSin3 hdfsSin4 hdfsSin5
>>     >
>>     >
>>     >
>>     > # define sources
>>     >
>>     > agent.sources.seqGenSrc.type >>     org.apache.flume.source.**scribe.ScribeSource
>>     >
>>     > agent.sources.seqGenSrc.**selector.type = io.flume.RandomSelector
>>     >
>>     >
>>     >
>>     > # define sinks
>>     >
>>     > agent.sinks.hdfsSin1.type = hdfs
>>     >
>>     > agent.sinks.hdfsSin1.hdfs.path = /flume_test/data1/
>>     >
>>     > agent.sinks.hdfsSin1.hdfs.**rollInterval = 300