Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Dual Hadoop/HBase configuration through same client


Copy link to this message
-
Re: Dual Hadoop/HBase configuration through same client
Thanks Ted for the response. But the issue is that I want to read from one
cluster and write to another. If I will have to clients then how will they
communicate with each other? Essentially what am I trying to do here is
intra-cluster data copy/exchange. Any other ideas or suggestions? Even if
both servers have no security or one has Kerberos or both have
authentication how to exchange data between them?

I was actually not expecting that I cannot load multiple Hadoop or HBase
configurations in 2 different Configuration objects in one application.
As mentioned I have tried overwriting properties as well but
security/authentication properties are overwritten somehow.

Regards,
Shahab
On Fri, Apr 26, 2013 at 7:43 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looks like the easiest solution is to use separate clients, one for each
> cluster you want to connect to.
>
> Cheers
>
> On Sat, Apr 27, 2013 at 6:51 AM, Shahab Yunus <[EMAIL PROTECTED]
> >wrote:
>
> > Hello,
> >
> > This is a follow-up to my previous post a few days back. I am trying to
> > connect to 2 different Hadoop clusters' setups through a same client but
> I
> > am running into the issue that the config of one overwrites the other.
> >
> > The scenario is that I want to read data from an HBase table from one
> > cluster and write it as a file on HDFS on the other. Individually, if I
> try
> > to write to them they both work but when I try this through a same Java
> > client, they fail.
> >
> > I have tried loading the core-site.xml through addResource method of the
> > Configuration class but only the first found config file is picked? I
> have
> > also tried by renaming the config files and then adding them as a
> resource
> > (again through the addResource method).
> >
> > The situation is compounded by the fact that one cluster is using
> Kerberos
> > authentication and the other is not? If the Kerberos server's file is
> found
> > first then authentication failures are faced for the other server when
> > Hadoop tries to find client authentication information. If the 'simple'
> > cluster's config is loaded first then 'Authentication is Required' error
> is
> > encountered against the Kerberos server.
> >
> > I will gladly provide more information. Is it even possible even if let
> us
> > say both servers have same security configuration or none? Any ideas?
> > Thanks a million.
> >
> > Regards,
> > Shahab
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB