Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Dual Hadoop/HBase configuration through same client


+
Shahab Yunus 2013-04-26, 22:51
+
Ted Yu 2013-04-26, 23:43
Copy link to this message
-
Re: Dual Hadoop/HBase configuration through same client
Shahab Yunus 2013-04-27, 05:20
Thanks Ted for the response. But the issue is that I want to read from one
cluster and write to another. If I will have to clients then how will they
communicate with each other? Essentially what am I trying to do here is
intra-cluster data copy/exchange. Any other ideas or suggestions? Even if
both servers have no security or one has Kerberos or both have
authentication how to exchange data between them?

I was actually not expecting that I cannot load multiple Hadoop or HBase
configurations in 2 different Configuration objects in one application.
As mentioned I have tried overwriting properties as well but
security/authentication properties are overwritten somehow.

Regards,
Shahab
On Fri, Apr 26, 2013 at 7:43 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looks like the easiest solution is to use separate clients, one for each
> cluster you want to connect to.
>
> Cheers
>
> On Sat, Apr 27, 2013 at 6:51 AM, Shahab Yunus <[EMAIL PROTECTED]
> >wrote:
>
> > Hello,
> >
> > This is a follow-up to my previous post a few days back. I am trying to
> > connect to 2 different Hadoop clusters' setups through a same client but
> I
> > am running into the issue that the config of one overwrites the other.
> >
> > The scenario is that I want to read data from an HBase table from one
> > cluster and write it as a file on HDFS on the other. Individually, if I
> try
> > to write to them they both work but when I try this through a same Java
> > client, they fail.
> >
> > I have tried loading the core-site.xml through addResource method of the
> > Configuration class but only the first found config file is picked? I
> have
> > also tried by renaming the config files and then adding them as a
> resource
> > (again through the addResource method).
> >
> > The situation is compounded by the fact that one cluster is using
> Kerberos
> > authentication and the other is not? If the Kerberos server's file is
> found
> > first then authentication failures are faced for the other server when
> > Hadoop tries to find client authentication information. If the 'simple'
> > cluster's config is loaded first then 'Authentication is Required' error
> is
> > encountered against the Kerberos server.
> >
> > I will gladly provide more information. Is it even possible even if let
> us
> > say both servers have same security configuration or none? Any ideas?
> > Thanks a million.
> >
> > Regards,
> > Shahab
> >
>
+
Damien Hardy 2013-04-27, 06:51
+
Shahab Yunus 2013-04-27, 16:54
+
Ted Yu 2013-04-27, 22:38
+
Shahab Yunus 2013-04-28, 01:13
+
Michael Segel 2013-04-28, 02:35
+
Shahab Yunus 2013-04-28, 12:43