Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Re: Maven Cloudera Configuration problem


+
Raj K Singh 2013-08-14, 09:05
+
Pavan Sudheendra 2013-08-13, 16:07
+
sandy.ryza@... 2013-08-13, 17:03
+
Pavan Sudheendra 2013-08-13, 17:07
+
Shahab Yunus 2013-08-13, 20:04
+
Brad Cox 2013-08-13, 22:07
+
Suresh Srinivas 2013-08-13, 23:54
Copy link to this message
-
Re: Maven Cloudera Configuration problem
In our Clouder 4.2.0 cluster, Iog-in with *admin* user (do you have
appropriate permissions by the way?) Then I click on any one of the 3
services (hbase, mapred, hdfs and excluding zookeeper) from the top-leftish
menu. Then for each of these I can click the *Configuration* tab which is
in the top-middlish section of the page. Once the configuration page opens
then I click on Action menu on the top-right. One of the sub-menu of this
is *Download Client Configuration* which as the name says downloads the
config files (zip file to be exact) to be used at client machines.

Regards,
Shahab
On Tue, Aug 13, 2013 at 6:07 PM, Brad Cox <[EMAIL PROTECTED]> wrote:

> That link got my hopes up. But Cloudera Manager  (what I'm running; on
> CDH4) does not offer an "Export Client Config" option. What am I missing?
>
> On Aug 13, 2013, at 4:04 PM, Shahab Yunus <[EMAIL PROTECTED]> wrote:
>
> You should not use LocalJobRunner. Make sure that the mapred.job.tracker
> property does not point to 'local' an instead to your job-tracker host and
> port.
>
> *But before that* as Sandy said, your client machine (from where you will
> be kicking of your jobs and apps) should be using config files which will
> have your cluster's configuration. This is the alternative that you should
> follow if you don't want to bundle the configs for your cluster in the
> application itself (either in java code or separate copies of relevant
> properties set of config files.) This was something which I was suggesting
> early on to just to get you started using your cluster instead of local
> mode.
>
> By the way have you seen the following link? It gives you step by step
> information about how to generate config files from your cluster specific
> to your cluster and then how to place them and use the from any machine you
> want to designate as your client. Running your jobs form one of the
> datanodes without proper config would not work.
>
> https://ccp.cloudera.com/display/FREE373/Generating+Client+Configuration
>
> Regards,
> Shahab
>
>
> On Tue, Aug 13, 2013 at 1:07 PM, Pavan Sudheendra <[EMAIL PROTECTED]
> >wrote:
>
> Yes Sandy, I'm referring to LocalJobRunner. I'm actually running the
> job on one datanode..
>
> What changes should i make so that my application would take advantage
> of the cluster as a whole?
>
> On Tue, Aug 13, 2013 at 10:33 PM,  <[EMAIL PROTECTED]> wrote:
>
> Nothing in your pom.xml should affect the configurations your job runs
>
> with.
>
>
> Are you running your job from a node on the cluster? When you say
>
> localhost configurations, do you mean it's using the LocalJobRunner?
>
>
> -sandy
>
> (iphnoe tpying)
>
> On Aug 13, 2013, at 9:07 AM, Pavan Sudheendra <[EMAIL PROTECTED]>
>
> wrote:
>
>
> When i actually run the job on the multi node cluster, logs shows it
> uses localhost configurations which i don't want..
>
> I just have a pom.xml which lists all the dependencies like standard
> hadoop, standard hbase, standard zookeeper etc., Should i remove these
> dependencies?
>
> I want the cluster settings to apply in my map-reduce application..
> So, this is where i'm stuck at..
>
> On Tue, Aug 13, 2013 at 9:30 PM, Pavan Sudheendra <[EMAIL PROTECTED]>
>
> wrote:
>
> Hi Shabab and Sandy,
> The thing is we have a 6 node cloudera cluster running.. For
> development purposes, i was building a map-reduce application on a
> single node apache distribution hadoop with maven..
>
> To be frank, i don't know how to deploy this application on a multi
> node cloudera cluster. I am fairly well versed with Multi Node Apache
> Hadoop Distribution.. So, how can i go forward?
>
> Thanks for all the help :)
>
> On Tue, Aug 13, 2013 at 9:22 PM,  <[EMAIL PROTECTED]> wrote:
>
> Hi Pavan,
>
> Configuration properties generally aren't included in the jar itself
>
> unless you explicitly set them in your java code. Rather they're picked up
> from the mapred-site.xml file located in the Hadoop configuration directory
> on the host you're running your job from.
+
Brad Cox 2013-08-13, 16:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB