Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> ProtocolProvider errors On MRv2 Failed to use org.apache.hadoop.mapred.YarnClientProtocolProvider


Copy link to this message
-
Re: ProtocolProvider errors On MRv2 Failed to use org.apache.hadoop.mapred.YarnClientProtocolProvider
2011/11/28 Stephen Boesch <[EMAIL PROTECTED]>:
>
> Hi
>   I set up a pseudo cluster  according to the instructions ��here
>  http://www.cloudera.com/blog/2011/11/building-and-deploying-mr2/.
> Initially the randomwriter example worked. But after a crash on the machine
> and restarting the services I am getting the errors shown below.
> Jps seems to think the processes are running properly:
>
> had@mithril:/shared/hadoop$ jps
> 7980 JobHistoryServer
> 7668 NameNode
> 7821 ResourceManager
> 7748 DataNode
> 8021 Jps
> 7902 NodeManager
>
> $ hadoop jar hadoop-mapreduce-examples-0.23.0.jar  randomwriter
> -Dmapreduce.job.user.name=$USER
> -Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory
> -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=64m
> -Ddfs.block.size=64m -libjars
> $YARN_HOME/modules/hadoop-mapreduce-client-jobclient-0.23.0.jar output
>
> 2011-11-28 10:23:56,102 WARN  conf.Configuration
> (Configuration.java:set(629)) - mapred.used.genericoptionsparser is
> deprecated. Instead, use mapreduce.client.genericoptionsparser.used
> 2011-11-28 10:23:56,158 INFO  ipc.YarnRPC (YarnRPC.java:create(47)) -
> Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
> 2011-11-28 10:23:56,162 INFO  mapred.ResourceMgrDelegate
> (ResourceMgrDelegate.java:<init>(95)) - Connecting to ResourceManager at
> /0.0.0.0:8040
> 2011-11-28 10:23:56,163 INFO  ipc.HadoopYarnRPC
> (HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc proxy
> for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
> 2011-11-28 10:23:56,203 INFO  mapred.ResourceMgrDelegate
> (ResourceMgrDelegate.java:<init>(99)) - Connected to ResourceManager at
> /0.0.0.0:8040
> 2011-11-28 10:23:56,248 INFO  mapreduce.Cluster
> (Cluster.java:initialize(116)) - Failed to use
> org.apache.hadoop.mapred.YarnClientProtocolProvider due to error:
> java.lang.reflect.InvocationTargetException
> 2011-11-28 10:23:56,250 INFO  mapreduce.Cluster
> (Cluster.java:initialize(111)) - Cannot pick
> org.apache.hadoop.mapred.LocalClientProtocolProvider as the
> ClientProtocolProvider - returned null protocol
> 2011-11-28 10:23:56,251 INFO  mapreduce.Cluster
> (Cluster.java:initialize(111)) - Cannot pick
> org.apache.hadoop.mapred.JobTrackerClientProtocolProvider as the
> ClientProtocolProvider - returned null protocol
> java.io.IOException: Cannot initialize Cluster. Please check your
> configuration for mapreduce.framework.name and the correspond server
> addresses.
>
> My  *-site.xml files are precisely as shown on the instructions page.
> In any case copying here the one that is most germane - mapred-site.xml
> <?xml version="1.0"?>
> <?xml-stylesheet href="configuration.xsl"?>
> <configuration>
> <property>
> <name> mapreduce.framework.name</name>
> <value>yarn</value>
> </property>
> </configuration>
>

Remember that you have to configure two conf files related to Yarn,
 yarn-site.xml:
 <?xml version=”1.0″?>
<configuration>
<!– Site specific YARN configuration properties –>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce.shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>

and mapred-site.xml

<?xml version=”1.0″?>
<?xml-stylesheet href=”configuration.xsl”?>
<configuration>
<property>
<name> mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

Regards

--
Marcos Luis Ortíz Valmaseda
 Linux Infrastructure Engineer
 Linux User # 418229
 http://marcosluis2186.posterous.com
 http://www.linkedin.com/in/marcosluis2186
 Twitter: @marcosluis2186
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB