Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> ProtocolProvider errors On MRv2 Failed to use org.apache.hadoop.mapred.YarnClientProtocolProvider


+
Stephen Boesch 2011-11-28, 18:40
Copy link to this message
-
Re: ProtocolProvider errors On MRv2 Failed to use org.apache.hadoop.mapred.YarnClientProtocolProvider
2011/11/28 Stephen Boesch <[EMAIL PROTECTED]>:
>
> Hi
>   I set up a pseudo cluster  according to the instructions ��here
>  http://www.cloudera.com/blog/2011/11/building-and-deploying-mr2/.
> Initially the randomwriter example worked. But after a crash on the machine
> and restarting the services I am getting the errors shown below.
> Jps seems to think the processes are running properly:
>
> had@mithril:/shared/hadoop$ jps
> 7980 JobHistoryServer
> 7668 NameNode
> 7821 ResourceManager
> 7748 DataNode
> 8021 Jps
> 7902 NodeManager
>
> $ hadoop jar hadoop-mapreduce-examples-0.23.0.jar  randomwriter
> -Dmapreduce.job.user.name=$USER
> -Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory
> -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=64m
> -Ddfs.block.size=64m -libjars
> $YARN_HOME/modules/hadoop-mapreduce-client-jobclient-0.23.0.jar output
>
> 2011-11-28 10:23:56,102 WARN  conf.Configuration
> (Configuration.java:set(629)) - mapred.used.genericoptionsparser is
> deprecated. Instead, use mapreduce.client.genericoptionsparser.used
> 2011-11-28 10:23:56,158 INFO  ipc.YarnRPC (YarnRPC.java:create(47)) -
> Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 2011-11-28 10:23:56,162 INFO  mapred.ResourceMgrDelegate
> (ResourceMgrDelegate.java:<init>(95)) - Connecting to ResourceManager at
> /0.0.0.0:8040
> 2011-11-28 10:23:56,163 INFO  ipc.HadoopYarnRPC
> (HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc proxy
> for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 2011-11-28 10:23:56,203 INFO  mapred.ResourceMgrDelegate
> (ResourceMgrDelegate.java:<init>(99)) - Connected to ResourceManager at
> /0.0.0.0:8040
> 2011-11-28 10:23:56,248 INFO  mapreduce.Cluster
> (Cluster.java:initialize(116)) - Failed to use
> org.apache.hadoop.mapred.YarnClientProtocolProvider due to error:
> java.lang.reflect.InvocationTargetException
> 2011-11-28 10:23:56,250 INFO  mapreduce.Cluster
> (Cluster.java:initialize(111)) - Cannot pick
> org.apache.hadoop.mapred.LocalClientProtocolProvider as the
> ClientProtocolProvider - returned null protocol
> 2011-11-28 10:23:56,251 INFO  mapreduce.Cluster
> (Cluster.java:initialize(111)) - Cannot pick
> org.apache.hadoop.mapred.JobTrackerClientProtocolProvider as the
> ClientProtocolProvider - returned null protocol
> java.io.IOException: Cannot initialize Cluster. Please check your
> configuration for mapreduce.framework.name and the correspond server
> addresses.
>
> My  *-site.xml files are precisely as shown on the instructions page.
> In any case copying here the one that is most germane - mapred-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet href="configuration.xsl"?>
> <configuration>
> <property>
> <name> mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>

Remember that you have to configure two conf files related to Yarn,
 yarn-site.xml:
 <?xml version=”1.0″?>
<configuration>
<!– Site specific YARN configuration properties –>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce.shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>

and mapred-site.xml

<?xml version=”1.0″?>
<?xml-stylesheet href=”configuration.xsl”?>
<configuration>
<property>
<name> mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

Regards

--
Marcos Luis Ortíz Valmaseda
 Linux Infrastructure Engineer
 Linux User # 418229
 http://marcosluis2186.posterous.com
 http://www.linkedin.com/in/marcosluis2186
 Twitter: @marcosluis2186
+
Stephen Boesch 2011-11-28, 21:19
+
Stephen Boesch 2011-11-28, 21:25
+
Stephen Boesch 2011-11-29, 00:38