Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Re: Issue with HBase Sink in Flume ( 1.3.0)


Copy link to this message
-
Re: Issue with HBase Sink in Flume ( 1.3.0)
Hello Jeff,

Please find below requested logs.. Initiation part of the logs were
unfortunately not included. I can run these again if necessary but the
Zookeeper connection is included in the logs.
14/02/17 10:26:12 INFO properties.PropertiesFileConfigurationProvider:
created channel ch2
14/02/17 10:26:13 INFO sink.DefaultSinkFactory: Creating instance of sink:
hbase-sink, type: org.apache.flume.sink.hbase.HBaseSink
14/02/17 10:26:13 INFO sink.DefaultSinkFactory: Creating instance of sink:
hdfs-sink, type: hdfs
14/02/17 10:26:14 INFO hdfs.HDFSEventSink: Hadoop Security enabled: false
14/02/17 10:26:14 INFO nodemanager.DefaultLogicalNodeManager: Starting new
configuration:{ sourceRunners:{exec-source=EventDrivenSourceRunner: {
source:org.apache.flume.source.ExecSource{name:exec-source,state:IDLE} }}
sinkRunners:{hbase-sink=SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@4c004c counterGroup:{
name:null counters:{} } }, hdfs-sink=SinkRunner: {
policy:org.apache.flume.sink.DefaultSinkProcessor@7b017b01 counterGroup:{
name:null counters:{} } }} channels:{ch1=FileChannel ch1 { dataDirs:
[/home/biadmin/.flume/file-channel/data] }, ch2=FileChannel ch2 { dataDirs:
[/home/biadmin/.flume/file-channel2/data] }} }
14/02/17 10:26:14 INFO nodemanager.DefaultLogicalNodeManager: Starting
Channel ch1
14/02/17 10:26:14 INFO file.FileChannel: Starting FileChannel ch1 {
dataDirs: [/home/biadmin/.flume/file-channel/data] }...
14/02/17 10:26:14 INFO nodemanager.DefaultLogicalNodeManager: Starting
Channel ch2
14/02/17 10:26:14 INFO file.FileChannel: Starting FileChannel ch2 {
dataDirs: [/home/biadmin/.flume/file-channel2/data] }...
14/02/17 10:26:14 INFO file.Log: Encryption is not enabled
14/02/17 10:26:14 INFO file.Log: Replay started
14/02/17 10:26:14 INFO file.Log: Encryption is not enabled
14/02/17 10:26:14 INFO file.Log: Replay started
14/02/17 10:26:14 INFO file.Log: Found NextFileID 7, from
[/home/biadmin/.flume/file-channel/data/log-7,
/home/biadmin/.flume/file-channel/data/log-6]
14/02/17 10:26:14 INFO file.Log: Found NextFileID 6, from
[/home/biadmin/.flume/file-channel2/data/log-6,
/home/biadmin/.flume/file-channel2/data/log-4,
/home/biadmin/.flume/file-channel2/data/log-5]
14/02/17 10:26:14 INFO file.EventQueueBackingStoreFileV3: Starting up with
/home/biadmin/.flume/file-channel2/checkpoint/checkpoint and
/home/biadmin/.flume/file-channel2/checkpoint/checkpoint.meta
14/02/17 10:26:14 INFO file.EventQueueBackingStoreFileV3: Reading
checkpoint metadata from
/home/biadmin/.flume/file-channel2/checkpoint/checkpoint.meta
14/02/17 10:26:14 INFO file.EventQueueBackingStoreFileV3: Starting up with
/home/biadmin/.flume/file-channel/checkpoint/checkpoint and
/home/biadmin/.flume/file-channel/checkpoint/checkpoint.meta
14/02/17 10:26:14 INFO file.EventQueueBackingStoreFileV3: Reading
checkpoint metadata from
/home/biadmin/.flume/file-channel/checkpoint/checkpoint.meta
14/02/17 10:26:14 INFO file.Log: Last Checkpoint Mon Feb 17 10:21:35 EST
2014, queue depth = 0
14/02/17 10:26:14 INFO file.Log: Last Checkpoint Mon Feb 17 10:21:31 EST
2014, queue depth = 0
14/02/17 10:26:14 INFO file.Log: Replaying logs with v2 replay logic
14/02/17 10:26:14 INFO file.Log: Replaying logs with v2 replay logic
14/02/17 10:26:14 INFO file.ReplayHandler: Starting replay of
[/home/biadmin/.flume/file-channel/data/log-6,
/home/biadmin/.flume/file-channel/data/log-7]
14/02/17 10:26:14 INFO file.ReplayHandler: Starting replay of
[/home/biadmin/.flume/file-channel2/data/log-4,
/home/biadmin/.flume/file-channel2/data/log-5,
/home/biadmin/.flume/file-channel2/data/log-6]
14/02/17 10:26:14 INFO file.ReplayHandler: Replaying
/home/biadmin/.flume/file-channel/data/log-6
14/02/17 10:26:14 INFO file.ReplayHandler: Replaying
/home/biadmin/.flume/file-channel2/data/log-4
14/02/17 10:26:14 INFO tools.DirectMemoryUtils: Unable to get
maxDirectMemory from VM: NoSuchMethodException:
sun.misc.VM.maxDirectMemory(null)
14/02/17 10:26:14 INFO tools.DirectMemoryUtils: Direct Memory Allocation:
Allocation = 1048576, Allocated = 0, MaxDirectMemorySize = 20971520,
Remaining = 20971520
14/02/17 10:26:16 INFO file.LogFile: fast-forward to checkpoint position:
32040
14/02/17 10:26:16 INFO file.ReplayHandler: Replaying
/home/biadmin/.flume/file-channel/data/log-7
14/02/17 10:26:16 INFO file.LogFile: fast-forward to checkpoint position:
2496
14/02/17 10:26:16 WARN file.LogFile: Checkpoint for
file(/home/biadmin/.flume/file-channel2/data/log-4) is: 1392407375821,
which is beyond the requested checkpoint time: 1392650490155 and position 0
14/02/17 10:26:16 INFO file.ReplayHandler: Replaying
/home/biadmin/.flume/file-channel2/data/log-5
14/02/17 10:26:16 INFO file.LogFile: fast-forward to checkpoint position:
22843
14/02/17 10:26:16 INFO file.LogFile: Encountered EOF at 22843 in
/home/biadmin/.flume/file-channel2/data/log-5
14/02/17 10:26:16 INFO file.ReplayHandler: Replaying
/home/biadmin/.flume/file-channel2/data/log-6
14/02/17 10:26:16 WARN file.LogFile: Checkpoint for
file(/home/biadmin/.flume/file-channel2/data/log-6) is: 1392650490155,
which is beyond the requested checkpoint time: 1392650490155 and position 0
14/02/17 10:26:16 INFO file.ReplayHandler: read: 0, put: 0, take: 0,
rollback: 0, commit: 0, skip: 0, eventCount:0
14/02/17 10:26:16 INFO file.Log: Rolling
/home/biadmin/.flume/file-channel2/data
14/02/17 10:26:16 INFO file.Log: Roll start
/home/biadmin/.flume/file-channel2/data
14/02/17 10:26:16 INFO file.LogFile: Opened
/home/biadmin/.flume/file-channel2/data/log-7
14/02/17 10:26:16 INFO file.LogFile: Encountered EOF at 2496 in
/home/biadmin/.flume/file-channel/data/log-7
14/02/17 10:26:16 INFO file.LogFile: Encountered EOF at 32071 in
/home/biadmin/.flume/file-channel/data/log-6
14/02/17 10:26:16 INFO file.ReplayHandler: read: 1, put: 0, take: 0,
rollback: 0, commit: 0, skip: 1, eventCount:0
14/02/17 10:26:16 INFO file.Log: Rolling
/home/biadmin/.flume/file-channel/data
14/02/17 10:26:16 INFO file.Log: Roll start
/home/biadmin/.flu