Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HRegionPartitioner breaks my configuration


Copy link to this message
-
Re: HRegionPartitioner breaks my configuration
Christoph,

I still don't get your issue though. A stack trace of the error thrown
out by the components you used would be good to have :)

On Thu, May 10, 2012 at 3:38 PM, Christoph Bauer
<[EMAIL PROTECTED]> wrote:
> Hi,
> thank you for your input.
>
> I've been doing it exactly that way. Jars appear in the classpath
> without problems.
> But I am unable to transfer hbase-site.xml to the mapper's classpath.
>
> So now I will try adding hbase-site.xml to the CP hadoop-env.sh.
>
> The addDependencyJars mechanism does not work for xml files :(
>
>
>
> 2012/5/9 Harsh J <[EMAIL PROTECTED]>:
>> Could you share your whole stack trace?
>>
>> How do you launch your HBase+MR job? The ideal way is to simply do:
>>
>> HADOOP_CLASSPATH=`hbase classpath` hadoop jar <hbase job jar> <args>
>>
>> And this will take care of hbase-site.xml location appearing in the
>> classpath as well. If you're using a package-installed environment,
>> ensure /etc/hbase/conf/hbase-site.xml is populated with the right
>> settings and if not, make such a file and:
>>
>> export HBASE_CONF_DIR=/dir/that/contains/that/file
>>
>> Before running the former command.
>>
>> Let us know if this helps.
>>
>> On Wed, May 9, 2012 at 9:38 PM, Christoph Bauer
>> <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> first, I'm aware of HBASE-4398 though I don't know how that patch could work.
>>>
>>> I'm on a cdh3u3 cluster with 4 nodes. hbase is 0.90.4.
>>>
>>> The problem is zookeeper is running on port 2222
>>>
>>> The following line results in a NPE when the mappers start:
>>> TableMapReduceUtil.initTableReducerJob("test2",
>>> IdentityTableReducer.class, job, HRegionPartitioner.class);
>>>
>>> HBaseConfiguration.addHbaseResources in HRegionPartitioner.setConf
>>> overwrites quorum and clientPort with hbase-default.xml from hbase
>>> jar, maybe more.
>>> HBaseConfiguration.addHbaseResources also tries to load
>>> hbase-site.xml, but fails silently (not found as resource).
>>>
>>> Can I make my mapreduce jobs aware of this resource. i.e. pass it to
>>> all the mappers or do I have to ask my administrator to make some
>>> changes?
>>>
>>>
>>> Thank you,
>>>
>>> Christoph Bauer
>>
>>
>>
>> --
>> Harsh J

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB