|
|
-
RE: Oozie workflow error - renewing token issueCorbett Martin 2013-01-31, 18:30
Thanks Daryn for the reply.
Here's some answers that Alejandro asked for: 1) Versions reported by Cloudera Manager: Flume NG CDH4 1.2.0+121 Hadoop CDH4 2.0.0+545 HBase CDH4 0.92.1+156 HDFS (CDH4 only) CDH4 2.0.0+545 Hive CDH4 0.9.0+150 HttpFS (CDH4 only) CDH4 2.0.0+545 Hue CDH4 2.1.0+210 Hue Plugins CDH4 2.1.0+210 MapReduce 1 (CDH4 only) CDH4 0.20.2+1261 MapReduce 2 (CDH4 only) CDH4 2.0.0+545 Oozie CDH4 3.2.0+123 Yarn (CDH4 only) CDH4 2.0.0+545 Zookeeper CDH4 3.4.3+27 2) We didn't specifically enable Kerberos so my guess is its off. 3) Bummer here - seems the logs have been cleaned up. I can repost some new ones when we try to run again. Yes we're running the job as hdfs. Let me know if you have some other advice. Thanks for your feedback. ~Corbett Martin Software Architect AbsoluteAR Accounts Receivable Services - An NHIN Solution From: Daryn Sharp [mailto:[EMAIL PROTECTED]] Sent: Wednesday, January 30, 2013 3:07 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Subject: Re: Oozie workflow error - renewing token issue The token renewer needs to be the job tracker principal. I think oozie had "mr token" hardcoded at one point, but later changed it to use a conf setting. The rest of the log looks very odd - ie. it looks like security is off, but it can't be. It's trying to renew hdfs tokens issued for the "hdfs" superuser. Are you running your job as hdfs? With security enabled, the container executor won't run jobs for hdfs... It's also trying to renew as "mapred (auth:SIMPLE)". SIMPLE means that security is disabled, which doesn't make sense since tokens require kerberos. The cannot find renewer class for MAPREDUCE_DELEGATION_TOKEN is also odd because it did subsequently try to renew the token. Daryn On Jan 30, 2013, at 2:14 PM, Alejandro Abdelnur wrote: Cobert, [Moving thread to user@oozie.a.o<mailto:user@oozie.a.o>, BCCing common-user@hadoop.a.o<mailto:common-user@hadoop.a.o>] * What version of Oozie are you using? * Is the cluster a secure setup (Kerberos enabled)? * Would you mind posting the complete launcher logs? Thx On Wed, Jan 30, 2013 at 6:14 AM, Corbett Martin <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote: Thanks for the tip. The sqoop command listed in the stdout log file is: sqoop import --driver org.apache.derby.jdbc.ClientDriver --connect jdbc:derby://test-server:1527/mondb --username monuser --password xxxxxxxxx --table MONITOR --split-by request_id --target-dir /mon/import --append --incremental append --check-column request_id --last-value 2000000 The following information is from the sterr and stdout log files. /var/log/hadoop-0.20-mapreduce/userlogs/job_201301231648_0029/attempt_201301231648_0029_m_000000_0 # more stderr No such sqoop tool: sqoop. See 'sqoop help'. Intercepting System.exit(1) Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] /var/log/hadoop-0.20-mapreduce/userlogs/job_201301231648_0029/attempt_201301231648_0029_m_000000_0 # more stdout ... ... ... >>> Invoking Sqoop command line now >>> 1598 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration. Intercepting System.exit(1) <<< Invocation of Main class completed <<< Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1] Oozie Launcher failed, finishing Hadoop job gracefully Oozie Launcher ends ~Corbett Martin Software Architect AbsoluteAR Accounts Receivable Services - An NHIN Solution From: Harsh J [mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>] Sent: Tuesday, January 29, 2013 10:11 PM To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> Subject: Re: Oozie workflow error - renewing token issue The job that Oozie launches for your action, which you are observing is failing, does its own logs (task logs) show any errors? On Wed, Jan 30, 2013 at 4:59 AM, Corbett Martin <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote: Harsh J This message and its contents (to include attachments) are the property of National Health Systems, Inc. and may contain confidential and proprietary information. This email and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed. You are hereby notified that any unauthorized disclosure, copying, or distribution of this message, or the taking of any unauthorized action based on information contained herein is strictly prohibited. Unauthorized use of information contained herein may subject you to civil and criminal prosecution and penalties. If you are not the intended recipient, you should delete this message immediately and notify the sender immediately by telephone or by replying to this transmission. Alejandro ________________________________ This message and its contents (to include attachments) are the property of National Health Systems, Inc. and may contain confidential and proprietary information. This email and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed. You are hereby notified that any unauthorized disclosure, copying, or distribution of this message, or the taking of any unauthorized action based on information contained herein is strictly prohibited. Unauthorized use of information contained herein may subject you to civil and criminal prosecution and penalties. I |