-
Re: HRegionPartitioner breaks my configurationHarsh J 2012-05-10, 10:42
Christoph,
I still don't get your issue though. A stack trace of the error thrown
out by the components you used would be good to have :)
On Thu, May 10, 2012 at 3:38 PM, Christoph Bauer
<[EMAIL PROTECTED]> wrote:
> Hi,
> thank you for your input.
>
> I've been doing it exactly that way. Jars appear in the classpath
> without problems.
> But I am unable to transfer hbase-site.xml to the mapper's classpath.
>
> So now I will try adding hbase-site.xml to the CP hadoop-env.sh.
>
> The addDependencyJars mechanism does not work for xml files :(
>
>
>
> 2012/5/9 Harsh J <[EMAIL PROTECTED]>:
>> Could you share your whole stack trace?
>>
>> How do you launch your HBase+MR job? The ideal way is to simply do:
>>
>> HADOOP_CLASSPATH=`hbase classpath` hadoop jar <hbase job jar> <args>
>>
>> And this will take care of hbase-site.xml location appearing in the
>> classpath as well. If you're using a package-installed environment,
>> ensure /etc/hbase/conf/hbase-site.xml is populated with the right
>> settings and if not, make such a file and:
>>
>> export HBASE_CONF_DIR=/dir/that/contains/that/file
>>
>> Before running the former command.
>>
>> Let us know if this helps.
>>
>> On Wed, May 9, 2012 at 9:38 PM, Christoph Bauer
>> <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> first, I'm aware of HBASE-4398 though I don't know how that patch could work.
>>>
>>> I'm on a cdh3u3 cluster with 4 nodes. hbase is 0.90.4.
>>>
>>> The problem is zookeeper is running on port 2222
>>>
>>> The following line results in a NPE when the mappers start:
>>> TableMapReduceUtil.initTableReducerJob("test2",
>>> IdentityTableReducer.class, job, HRegionPartitioner.class);
>>>
>>> HBaseConfiguration.addHbaseResources in HRegionPartitioner.setConf
>>> overwrites quorum and clientPort with hbase-default.xml from hbase
>>> jar, maybe more.
>>> HBaseConfiguration.addHbaseResources also tries to load
>>> hbase-site.xml, but fails silently (not found as resource).
>>>
>>> Can I make my mapreduce jobs aware of this resource. i.e. pass it to
>>> all the mappers or do I have to ask my administrator to make some
>>> changes?
>>>
>>>
>>> Thank you,
>>>
>>> Christoph Bauer
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
I still don't get your issue though. A stack trace of the error thrown
out by the components you used would be good to have :)
On Thu, May 10, 2012 at 3:38 PM, Christoph Bauer
<[EMAIL PROTECTED]> wrote:
> Hi,
> thank you for your input.
>
> I've been doing it exactly that way. Jars appear in the classpath
> without problems.
> But I am unable to transfer hbase-site.xml to the mapper's classpath.
>
> So now I will try adding hbase-site.xml to the CP hadoop-env.sh.
>
> The addDependencyJars mechanism does not work for xml files :(
>
>
>
> 2012/5/9 Harsh J <[EMAIL PROTECTED]>:
>> Could you share your whole stack trace?
>>
>> How do you launch your HBase+MR job? The ideal way is to simply do:
>>
>> HADOOP_CLASSPATH=`hbase classpath` hadoop jar <hbase job jar> <args>
>>
>> And this will take care of hbase-site.xml location appearing in the
>> classpath as well. If you're using a package-installed environment,
>> ensure /etc/hbase/conf/hbase-site.xml is populated with the right
>> settings and if not, make such a file and:
>>
>> export HBASE_CONF_DIR=/dir/that/contains/that/file
>>
>> Before running the former command.
>>
>> Let us know if this helps.
>>
>> On Wed, May 9, 2012 at 9:38 PM, Christoph Bauer
>> <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> first, I'm aware of HBASE-4398 though I don't know how that patch could work.
>>>
>>> I'm on a cdh3u3 cluster with 4 nodes. hbase is 0.90.4.
>>>
>>> The problem is zookeeper is running on port 2222
>>>
>>> The following line results in a NPE when the mappers start:
>>> TableMapReduceUtil.initTableReducerJob("test2",
>>> IdentityTableReducer.class, job, HRegionPartitioner.class);
>>>
>>> HBaseConfiguration.addHbaseResources in HRegionPartitioner.setConf
>>> overwrites quorum and clientPort with hbase-default.xml from hbase
>>> jar, maybe more.
>>> HBaseConfiguration.addHbaseResources also tries to load
>>> hbase-site.xml, but fails silently (not found as resource).
>>>
>>> Can I make my mapreduce jobs aware of this resource. i.e. pass it to
>>> all the mappers or do I have to ask my administrator to make some
>>> changes?
>>>
>>>
>>> Thank you,
>>>
>>> Christoph Bauer
>>
>>
>>
>> --
>> Harsh J
--
Harsh J
