Flume, mail # user - ERROR connector.DirectDriver: Exiting driver logicalNode agent2-18 in error state TailSource | fail because Attempting to open a stub sink 'fail'!

ERROR connector.DirectDriver: Exiting driver logicalNode agent2-18 in error state TailSource | fail because Attempting to open a stub sink 'fail'!

周梦想 2013-01-23, 09:21

hello,
I add logical config to flume:
agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
collect1: logicalSource |
collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);

map Hadoop46 agent1
map Hadoop47 agent2
map Hadoop48 collect1

===========================I run master on Hadoop48,
it reports:
2013-01-23 16:49:27,153 [main] INFO rolling.RollSink: Created RollSink:
trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@7a960b08]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:27,158 [main] INFO rolling.RollSink: Created RollSink:
trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1d29109e]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:27,207 [main] INFO master.ZooKeeperConfigStore: Created
new config at /flume-cfgs/cfg-0000000104
2013-01-23 16:49:27,207 [main-EventThread] INFO
master.ZooKeeperConfigStore: Config was updated - reloading
2013-01-23 16:49:27,255 [main] INFO master.ZooKeeperConfigStore: Created
new config at /flume-cfgs/cfg-0000000105
2013-01-23 16:49:27,255 [main-EventThread] INFO
master.ZooKeeperConfigStore: Config was updated - reloading
2013-01-23 16:49:27,307 [main] WARN logical.LogicalConfigurationManager:
Logical node 'collect1' has not been assigned to a physical node yet

================run agent1 on Hadoop46, it reports:

2013-01-23 16:49:45,070 [Heartbeat] INFO rolling.RollSink: Created
RollSink: trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1fd54f77]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:45,073 [Heartbeat] INFO agent.LogicalNodeManager: creating
new logical node collect1
2013-01-23 16:49:45,076 [logicalNode collect1-18] ERROR
connector.DirectDriver: Closing down due to exception on open calls
2013-01-23 16:49:45,076 [logicalNode collect1-18] INFO
connector.DirectDriver: Connector logicalNode collect1-18 exited with
error: Attempting to open a Stub Source!
2013-01-23 16:49:45,077 [logicalNode collect1-18] ERROR
connector.DirectDriver: Error closing logicalNode collect1-18 source:
Attempting to close a Stub Source!
2013-01-23 16:49:45,077 [logicalNode collect1-18] INFO rolling.RollSink:
closing RollSink
'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:45,081 [logicalNode collect1-18] INFO rolling.RollSink:
double close
'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'
2013-01-23 16:49:45,081 [logicalNode collect1-18] ERROR
connector.DirectDriver: Exiting driver logicalNode collect1-18 in error
state StubSource | Collector because Attempting to open a Stub Source!
2013-01-23 16:49:45,091 [Check config] INFO rolling.RollSink: Created
RollSink: trigger=[TimeTrigger: maxAge=5000
tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@5780208f]
checkPeriodMs = 250
spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
)'

==============and agent2 on Hadoop47 , also logs error.
[zhouhh@Hadoop47 ~]$ flume node

2013-01-23 17:11:38,454 [main] INFO mortbay.log: Started
SelectChannelConnector@0.0.0.0:35862
2013-01-23 17:11:38,454 [main] INFO util.InternalHttpServer: Server started
2013-01-23 17:11:38,468 [Heartbeat] INFO agent.MultiMasterRPC: No active
master RPC connection
2013-01-23 17:11:38,495 [Heartbeat] INFO agent.ThriftMasterRPC: Connected
to master at hadoop48:35872
2013-01-23 17:11:38,687 [main] INFO agent.LogicalNodeManager: creating new
logical node Hadoop47
2013-01-23 17:11:38,819 [main] WARN agent.FlumeNode: Flume is using Hadoop
core 1.0.4 which does not support Security / Authentication: null
2013-01-23 17:11:43,529 [Heartbeat] INFO agent.LogicalNodeManager: creating
new logical node agent2
2013-01-23 17:11:43,536 [logicalNode agent2-18] ERROR
connector.DirectDriver: Closing down due to exception on open calls
2013-01-23 17:11:43,537 [logicalNode agent2-18] INFO
connector.DirectDriver: Connector logicalNode agent2-18 exited with error:
Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,537 [logicalNode agent2-18] ERROR
connector.DirectDriver: Exiting driver logicalNode agent2-18 in error state
TailSource | fail because Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,540 [TailThread-0] INFO text.TailSource: TailThread has
exited
2013-01-23 17:11:43,553 [Check config] INFO agent.LogicalNode: Node config
successfully set to com.cloudera.flume.conf.FlumeConfigData@76a0f9cb
2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
connector.DirectDriver: Closing down due to exception on open calls
2013-01-23 17:11:43,554 [logicalNode agent2-20] INFO
connector.DirectDriver: Connector logicalNode agent2-20 exited with error:
Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,554 [logicalNode agent2-20] ERROR
connector.DirectDriver: Exiting driver logicalNode agent2-20 in error state
TailSource | fail because Attempting to open a stub sink 'fail'!
2013-01-23 17:11:43,554 [TailThread-1] INFO text.TailSource: TailThread has
exited

http://hadoop48:35871/flumemaster.jsp
Node status logical node physical nodehost name statusversionlast seen
delta (s) last seenHadoop46 Hadoop46Hadoop46IDLEnone 2Wed Jan 23 17:07:05
CST 2013 Hadoop47Hadoop47Hadoop47 IDLEnone1Wed Jan 23 17:07:06 CST 2013
Hadoop48Hadoop48Hadoop48 IDLEWed Jan 23 16:49:27 CST 2013 0Wed Jan 23
17:07:08 CST 2013WX-wuwb.TKOffice.local WX-wuwb.TKOffice.local
WX-wuwb.TKOffice.localERROR Wed Jan 23 16:49:27 CST 20130Wed Jan 23
17:07:07 CST 2013 agent2Hadoop47Hadoop47ERROR Wed Jan 23 16:49:27 CST
20131Wed Jan 23 17:07:06 CST 2013
aggent1Hadoop46 Hadoop46IDLEnone2 Wed Jan 23 17:07:05 CST 2013co1co1Hadoop48
LOSTWed Jan 23 09:56:

Re: ERROR connector.DirectDriver: Exiting driver logicalNode agent2-18 in error state TailSource | fail because Attempting to open a stub sink 'fail'!

Jeong-shik Jang 2013-01-23, 14:36

Hi Andy,

How about trying "refreshAll" command to see it helps?

It seems that logicalSource and logicalSink are not working perfectly
especially in finding and mapping logical node to physical node during
translation process in master node. I think "map Hadoop48 collect1"
should be done earlier so that master has mapping information for
collector1 to Hadoop48(physical node) so that translation can be done
properly when related configs were created.

for example,
(config) collect1: null | null

map Hadoop48 collect1

(config) collect1: logicalSource |
collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);

(config) agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
(config) agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");

map Hadoop46 agent1
map Hadoop47 agent2

and, there seems typo in node name; you mapped "aggent1" to Hadoop46 but
name of the config you had created was "agent1"; it is why aggent1 on
Hadoop46 is "IDLE".

- JS

On 1/23/13 6:21 PM, ??? wrote:
> hello,
> I add logical config to flume:
> agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> collect1: logicalSource |
> collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
>
> map Hadoop46 agent1
> map Hadoop47 agent2
> map Hadoop48 collect1
>
> ===========================> I run master on Hadoop48,
> it reports:
> 2013-01-23 16:49:27,153 [main] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@7a960b08]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,158 [main] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1d29109e]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,207 [main] INFO master.ZooKeeperConfigStore:
> Created new config at /flume-cfgs/cfg-0000000104
> 2013-01-23 16:49:27,207 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,255 [main] INFO master.ZooKeeperConfigStore:
> Created new config at /flume-cfgs/cfg-0000000105
> 2013-01-23 16:49:27,255 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,307 [main] WARN
> logical.LogicalConfigurationManager: Logical node 'collect1' has not
> been assigned to a physical node yet
>
> ================> run agent1 on Hadoop46, it reports:
>
> 2013-01-23 16:49:45,070 [Heartbeat] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1fd54f77]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,073 [Heartbeat] INFO agent.LogicalNodeManager:
> creating new logical node collect1
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] INFO
> connector.DirectDriver: Connector logicalNode collect1-18 exited with
> error: Attempting to open a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Error closing logicalNode collect1-18 source:
> Attempting to close a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] INFO
> rolling.RollSink: closing RollSink
> 'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,081 [logicalNode collect1-18] INFO
> rolling.RollSink: double close
> 'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,081 [logicalNode collect1-18] ERROR
Jeong-shik Jang / [EMAIL PROTECTED]
Gruter, Inc., R&D Team Leader
www.gruter.com
Enjoy Connecting

Re: ERROR connector.DirectDriver: Exiting driver logicalNode agent2-18 in error state TailSource | fail because Attempting to open a stub sink 'fail'!

周梦想 2013-01-25, 07:32

thank you jeong-shik, I will try this:)
Andy

2013/1/23 Jeong-shik Jang <[EMAIL PROTECTED]>

> Hi Andy,
>
> How about trying "refreshAll" command to see it helps?
>
> It seems that logicalSource and logicalSink are not working perfectly
> especially in finding and mapping logical node to physical node during
> translation process in master node. I think "map Hadoop48 collect1" should
> be done earlier so that master has mapping information for collector1 to
> Hadoop48(physical node) so that translation can be done properly when
> related configs were created.
>
> for example,
> (config) collect1: null | null
>
> map Hadoop48 collect1
>
> (config) collect1: logicalSource |
> collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
>
> (config) agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> (config) agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
>
> map Hadoop46 agent1
> map Hadoop47 agent2
>
> and, there seems typo in node name; you mapped "aggent1" to Hadoop46 but
> name of the config you had created was "agent1"; it is why aggent1 on
> Hadoop46 is "IDLE".
>
> - JS
>
> On 1/23/13 6:21 PM, 周梦想 wrote:
>
> hello,
> I add logical config to flume:
> agent1: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> agent2: tail("/home/zhouhh/cars.csv") | logicalSink("collect1");
> collect1: logicalSource |
> collectorSink("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-",5000,seqfile);
>
> map Hadoop46 agent1
> map Hadoop47 agent2
> map Hadoop48 collect1
>
> ===========================> I run master on Hadoop48,
> it reports:
> 2013-01-23 16:49:27,153 [main] INFO rolling.RollSink: Created RollSink:
> trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@7a960b08]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,158 [main] INFO rolling.RollSink: Created RollSink:
> trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1d29109e]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:27,207 [main] INFO master.ZooKeeperConfigStore: Created
> new config at /flume-cfgs/cfg-0000000104
> 2013-01-23 16:49:27,207 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,255 [main] INFO master.ZooKeeperConfigStore: Created
> new config at /flume-cfgs/cfg-0000000105
> 2013-01-23 16:49:27,255 [main-EventThread] INFO
> master.ZooKeeperConfigStore: Config was updated - reloading
> 2013-01-23 16:49:27,307 [main] WARN logical.LogicalConfigurationManager:
> Logical node 'collect1' has not been assigned to a physical node yet
>
> ================> run agent1 on Hadoop46, it reports:
>
> 2013-01-23 16:49:45,070 [Heartbeat] INFO rolling.RollSink: Created
> RollSink: trigger=[TimeTrigger: maxAge=5000
> tagger=com.cloudera.flume.handlers.rolling.ProcessTagger@1fd54f77]
> checkPeriodMs = 250
> spec='escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,073 [Heartbeat] INFO agent.LogicalNodeManager:
> creating new logical node collect1
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Closing down due to exception on open calls
> 2013-01-23 16:49:45,076 [logicalNode collect1-18] INFO
> connector.DirectDriver: Connector logicalNode collect1-18 exited with
> error: Attempting to open a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] ERROR
> connector.DirectDriver: Error closing logicalNode collect1-18 source:
> Attempting to close a Stub Source!
> 2013-01-23 16:49:45,077 [logicalNode collect1-18] INFO rolling.RollSink:
> closing RollSink
> 'escapedCustomDfs("hdfs://hadoop48:54310/user/flume/%y%m/%d","test-%{rolltag}"
> )'
> 2013-01-23 16:49:45,081 [logicalNode collect1-18] INFO rolling.RollSink:
> double close