+
Alberto Cordioli 2012-09-12, 09:35
+
Cheolsoo Park 2012-09-12, 18:11
+
Alberto Cordioli 2012-09-12, 22:10
+
Cheolsoo Park 2012-09-12, 23:38
+
Alberto Cordioli 2012-09-13, 08:22
-
Re: IllegalArgumentException: Not a host:port pair - Pig 0.10.0 with HBaseAlberto Cordioli 2012-09-13, 09:09
Ok, I solved the problem copying the required jars in $HADOOP_HOME/lib folder.
Thanks,
Alberto
On 13 September 2012 10:22, Alberto Cordioli <[EMAIL PROTECTED]> wrote:
> Cheolsoo, doing so I obtain a ClassNotFoundException:
> ERROR 2998: Unhandled internal error.
> org/apache/hadoop/hbase/filter/WritableByteArrayComparable
>
> That's a very strange thing since I'm sure that package is in the CLASSPATH.
> Are you using Hadoop 1.0.0 or 1.0.1?
>
>
> Alberto
>
> On 13 September 2012 01:38, Cheolsoo Park <[EMAIL PROTECTED]> wrote:
>> Hi Alberto,
>>
>> Here is my local setup that works. Please change them accordingly to your
>> environment.
>>
>> 1) I started hbase-0.94 in standalone mode.
>> 2) I downloaded hadoop-1.0.0.
>> 3) I built "pig-withouthadoop.jar" from the source and ran the following
>> commands:
>>
>> export HADOOP_HOME=/home/cheolsoo/workspace/hadoop-1.0.1
>> export HBASE_HOME=/home/cheolsoo/workspace/hbase-0.94.1
>> export ZOOKEEPER_HOME=/home/cheolsoo/workspace/hbase-0.94.1/lib
>> export
>> PIG_CLASSPATH=/home/cheolsoo/workspace/hbase-0.94.1/lib/protobuf-java-2.4.0a.jar
>> ./bin/pig -x local
>>
>> Please note that by setting HBASE_HOME, Pig uses hbase.jar in that
>> directory. I also set ZOOKEEPER_HOME to hbase-0.94.1/lib because that's
>> where zookeeper.jar exists. Lastly, I add protobuf-java-2.4.0a.jar
>> to PIG_CLASSPATH because I found that HBaseStorage fails with a
>> ClassNotFound exception without it. Basically, everything HBaseStorage
>> needs should be present in classpath at runtime. If these jars are
>> available in classpath by default (for example because you installed hbase
>> via rpm, etc), you won't have to set them explicitly.
>>
>> I am able to successfully load columns from a hbase table.
>>
>> Thanks,
>> Cheolsoo
>>
>> On Wed, Sep 12, 2012 at 3:10 PM, Alberto Cordioli <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Thanks Cheolsoo.
>>> I've already seen that link. But it is not so much clear to me how Pig
>>> makes use of HBase jars.
>>> Even if I use Pig in MapReduce mode (real or pseudo-distributed) it
>>> needs HBase on the client, right?
>>>
>>> In my particular case I use Pig in a real distributed cluster (but I
>>> tested it also in pseudo-distributed env) running pig in interactive
>>> way (using the script pig in bin folder). Should I launch pig with
>>> java command?
>>> Please, could you explain me how to setup correctly Pig for my case
>>> (Hbase 0.94 and hadoop 1.0.3)?
>>>
>>> Thank you very much,
>>> Alberto
>>>
>>> On 12 September 2012 20:11, Cheolsoo Park <[EMAIL PROTECTED]> wrote:
>>> > Hi Alberto,
>>> >
>>> > Are you running in local mode using "pig.jar", or in mapreduce mode using
>>> > "pig-withouthadoop.jar"?
>>> >
>>> > The error that you're seeing can happen when there is a version mismatch
>>> in
>>> > the HBase client and server. The pig.jar by default contains hbase-0.90,
>>> so
>>> > if you use pig.jar against hbase-0.94, it won't work. There was a jira
>>> > about running pig.jar against hbase-0.94:
>>> > https://issues.apache.org/jira/browse/PIG-2891
>>> >
>>> > Thanks,
>>> > Cheolsoo
>>> >
>>> >
>>> > On Wed, Sep 12, 2012 at 2:35 AM, Alberto Cordioli <
>>> > [EMAIL PROTECTED]> wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> I'm currently working with Pig 0.10.0. I'd like to load some data from
>>> >> an HBase table, but I encountered some problems. When I try to load
>>> >> the data it seems to work:
>>> >>
>>> >> grunt> raw = LOAD 'hbase://table_test' USING
>>> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:data1', '-loadKey
>>> >> true -limit 5') as (hash:bytearray, data1:chararray);
>>> >> 2012-09-12 11:27:48,213 [main] INFO
>>> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage - Adding
>>> >> family:descriptor filters with values d:data1
>>> >> 2012-09-12 11:27:48,216 [main] INFO
>>> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage - Adding
>>> >> family:descriptor filters with values d:data1
>>> >> 2012-09-12 11:27:48,264 [main] INFO
Alberto Cordioli
Thanks,
Alberto
On 13 September 2012 10:22, Alberto Cordioli <[EMAIL PROTECTED]> wrote:
> Cheolsoo, doing so I obtain a ClassNotFoundException:
> ERROR 2998: Unhandled internal error.
> org/apache/hadoop/hbase/filter/WritableByteArrayComparable
>
> That's a very strange thing since I'm sure that package is in the CLASSPATH.
> Are you using Hadoop 1.0.0 or 1.0.1?
>
>
> Alberto
>
> On 13 September 2012 01:38, Cheolsoo Park <[EMAIL PROTECTED]> wrote:
>> Hi Alberto,
>>
>> Here is my local setup that works. Please change them accordingly to your
>> environment.
>>
>> 1) I started hbase-0.94 in standalone mode.
>> 2) I downloaded hadoop-1.0.0.
>> 3) I built "pig-withouthadoop.jar" from the source and ran the following
>> commands:
>>
>> export HADOOP_HOME=/home/cheolsoo/workspace/hadoop-1.0.1
>> export HBASE_HOME=/home/cheolsoo/workspace/hbase-0.94.1
>> export ZOOKEEPER_HOME=/home/cheolsoo/workspace/hbase-0.94.1/lib
>> export
>> PIG_CLASSPATH=/home/cheolsoo/workspace/hbase-0.94.1/lib/protobuf-java-2.4.0a.jar
>> ./bin/pig -x local
>>
>> Please note that by setting HBASE_HOME, Pig uses hbase.jar in that
>> directory. I also set ZOOKEEPER_HOME to hbase-0.94.1/lib because that's
>> where zookeeper.jar exists. Lastly, I add protobuf-java-2.4.0a.jar
>> to PIG_CLASSPATH because I found that HBaseStorage fails with a
>> ClassNotFound exception without it. Basically, everything HBaseStorage
>> needs should be present in classpath at runtime. If these jars are
>> available in classpath by default (for example because you installed hbase
>> via rpm, etc), you won't have to set them explicitly.
>>
>> I am able to successfully load columns from a hbase table.
>>
>> Thanks,
>> Cheolsoo
>>
>> On Wed, Sep 12, 2012 at 3:10 PM, Alberto Cordioli <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Thanks Cheolsoo.
>>> I've already seen that link. But it is not so much clear to me how Pig
>>> makes use of HBase jars.
>>> Even if I use Pig in MapReduce mode (real or pseudo-distributed) it
>>> needs HBase on the client, right?
>>>
>>> In my particular case I use Pig in a real distributed cluster (but I
>>> tested it also in pseudo-distributed env) running pig in interactive
>>> way (using the script pig in bin folder). Should I launch pig with
>>> java command?
>>> Please, could you explain me how to setup correctly Pig for my case
>>> (Hbase 0.94 and hadoop 1.0.3)?
>>>
>>> Thank you very much,
>>> Alberto
>>>
>>> On 12 September 2012 20:11, Cheolsoo Park <[EMAIL PROTECTED]> wrote:
>>> > Hi Alberto,
>>> >
>>> > Are you running in local mode using "pig.jar", or in mapreduce mode using
>>> > "pig-withouthadoop.jar"?
>>> >
>>> > The error that you're seeing can happen when there is a version mismatch
>>> in
>>> > the HBase client and server. The pig.jar by default contains hbase-0.90,
>>> so
>>> > if you use pig.jar against hbase-0.94, it won't work. There was a jira
>>> > about running pig.jar against hbase-0.94:
>>> > https://issues.apache.org/jira/browse/PIG-2891
>>> >
>>> > Thanks,
>>> > Cheolsoo
>>> >
>>> >
>>> > On Wed, Sep 12, 2012 at 2:35 AM, Alberto Cordioli <
>>> > [EMAIL PROTECTED]> wrote:
>>> >
>>> >> Hi all,
>>> >>
>>> >> I'm currently working with Pig 0.10.0. I'd like to load some data from
>>> >> an HBase table, but I encountered some problems. When I try to load
>>> >> the data it seems to work:
>>> >>
>>> >> grunt> raw = LOAD 'hbase://table_test' USING
>>> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage('d:data1', '-loadKey
>>> >> true -limit 5') as (hash:bytearray, data1:chararray);
>>> >> 2012-09-12 11:27:48,213 [main] INFO
>>> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage - Adding
>>> >> family:descriptor filters with values d:data1
>>> >> 2012-09-12 11:27:48,216 [main] INFO
>>> >> org.apache.pig.backend.hadoop.hbase.HBaseStorage - Adding
>>> >> family:descriptor filters with values d:data1
>>> >> 2012-09-12 11:27:48,264 [main] INFO
Alberto Cordioli
