Hi,
I have quick questioin, are there any existing tools that are able to export
hive table rows into table database like mysql ? I checked sqoop, but it
seems it's for importing data to hadoop/hive, but not the other way(export).
Any of light you'd like to shed are welcomed, thanks!

Regards,
Peter

Hi Peter,

Sqoop supports both import (RDBMS --> HDFS) and export (HDFS --> RDBMS)
modes of operation. Please consult the user guide in the latest version of
Sqoop for more information:
http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz

Thanks.

Carl

On Mon, Sep 20, 2010 at 1:15 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:

> Hi,
> I have quick questioin, are there any existing tools that are able to
> export hive table rows into table database like mysql ? I checked sqoop, but
> it seems it's for importing data to hadoop/hive, but not the other
> way(export). Any of light you'd like to shed are welcomed, thanks!
>
> Regards,
> Peter
>

Thanks Carl for you hints, let me play with this version of sqoop.

-Peter

On Mon, Sep 20, 2010 at 2:03 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote:

> Hi Peter,
>
> Sqoop supports both import (RDBMS --> HDFS) and export (HDFS --> RDBMS)
> modes of operation. Please consult the user guide in the latest version of
> Sqoop for more information:
> http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz
>
> Thanks.
>
> Carl
>
>
> On Mon, Sep 20, 2010 at 1:15 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>> I have quick questioin, are there any existing tools that are able to
>> export hive table rows into table database like mysql ? I checked sqoop, but
>> it seems it's for importing data to hadoop/hive, but not the other
>> way(export). Any of light you'd like to shed are welcomed, thanks!
>>
>> Regards,
>> Peter
>>
>
>

Hi, Carl,
I tried to compile the package you mentioned, but it said Hbase and
zookeeper are required dependencies, our environment doesn't have these two,
and our hadoop version is from CDH2, which we probably are not convenient to
upgrade to CDH3 in short term, any chance to let the sqoop v1.1.0 run with
CDH2 without hbase and zookeeper installed ?

-Peter

On Mon, Sep 20, 2010 at 2:38 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:

> Thanks Carl for you hints, let me play with this version of sqoop.
>
> -Peter
>
>
> On Mon, Sep 20, 2010 at 2:03 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote:
>
>> Hi Peter,
>>
>> Sqoop supports both import (RDBMS --> HDFS) and export (HDFS --> RDBMS)
>> modes of operation. Please consult the user guide in the latest version of
>> Sqoop for more information:
>> http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz
>>
>> Thanks.
>>
>> Carl
>>
>>
>> On Mon, Sep 20, 2010 at 1:15 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>> I have quick questioin, are there any existing tools that are able to
>>> export hive table rows into table database like mysql ? I checked sqoop, but
>>> it seems it's for importing data to hadoop/hive, but not the other
>>> way(export). Any of light you'd like to shed are welcomed, thanks!
>>>
>>> Regards,
>>> Peter
>>>
>>
>>
>

Hi Peter,

Please try the version of Sqoop that ships with CDH3. This version does not
have dependencies on ZK or HBase. You can get copy of the tarball here:
http://archive.cloudera.com/cdh/3/sqoop-1.0.0+3.tar.gz

Thanks.

Carl

On Mon, Sep 20, 2010 at 5:47 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:

> Hi, Carl,
> I tried to compile the package you mentioned, but it said Hbase and
> zookeeper are required dependencies, our environment doesn't have these two,
> and our hadoop version is from CDH2, which we probably are not convenient to
> upgrade to CDH3 in short term, any chance to let the sqoop v1.1.0 run with
> CDH2 without hbase and zookeeper installed ?
>
> -Peter
>
>
> On Mon, Sep 20, 2010 at 2:38 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:
>
>> Thanks Carl for you hints, let me play with this version of sqoop.
>>
>> -Peter
>>
>>
>> On Mon, Sep 20, 2010 at 2:03 PM, Carl Steinbach <[EMAIL PROTECTED]>wrote:
>>
>>> Hi Peter,
>>>
>>> Sqoop supports both import (RDBMS --> HDFS) and export (HDFS --> RDBMS)
>>> modes of operation. Please consult the user guide in the latest version of
>>> Sqoop for more information:
>>> http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz
>>>
>>> Thanks.
>>>
>>> Carl
>>>
>>>
>>> On Mon, Sep 20, 2010 at 1:15 PM, Tianqiang Li <[EMAIL PROTECTED]>wrote:
>>>
>>>> Hi,
>>>> I have quick questioin, are there any existing tools that are able to
>>>> export hive table rows into table database like mysql ? I checked sqoop, but
>>>> it seems it's for importing data to hadoop/hive, but not the other
>>>> way(export). Any of light you'd like to shed are welcomed, thanks!
>>>>
>>>> Regards,
>>>> Peter
>>>>
>>>
>>>
>>
>

Hi, Carl,
I got sometime today to install sqoop-1.0.0+3 pointed out by you,
installation is successful, thanks, however, when I tried to use it and
type:

sqoop import --connect jdbc::mysql://mysql_host/my_db --table mytable
--username xxx --password yyy

*10/10/01 16:23:32 ERROR sqoop.Sqoop: Got exception running Sqoop:
java.lang.RuntimeException: Could not find appropriate Hadoop shim for
0.20.1+169.113*
java.lang.RuntimeException: Could not find appropriate Hadoop shim for
0.20.1+169.113
        at com.cloudera.sqoop.shims.ShimLoader.loadShim(ShimLoader.java:190)
        at
com.cloudera.sqoop.shims.ShimLoader.getHadoopShim(ShimLoader.java:109)
        at
com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:141)
        at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:66)
        at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:142)
        at com.cloudera.sqoop.Sqoop.run(Sqoop.java:134)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:170)
        at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:196)
        at com.cloudera.sqoop.Sqoop.main(Sqoop.java:205)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:616)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

I looked error msg up in internet, get some very related hit.
https://issues.cloudera.org/browse/SQOOP-82

Our environment is cdh2 hadoop-0.20.1, my question is that is it true there
is no way to let sqoop-1.0.0 work on my current hadoop version, other than
upgrading to chd3 beta-2 ? Currently I am not very tending to upgrade our
system since some important jobs are running on top of it on a daily basis.
Thanks.

-Peter

On Mon, Sep 20, 2010 at 6:03 PM, Carl Steinbach <[EMAIL PROTECTED]> wrote:

> Hi Peter,
>
> Please try the version of Sqoop that ships with CDH3. This version does not
> have dependencies on ZK or HBase. You can get copy of the tarball here:
> http://archive.cloudera.com/cdh/3/sqoop-1.0.0+3.tar.gz
>
> Thanks.
>
> Carl
>
>
> On Mon, Sep 20, 2010 at 5:47 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:
>
>> Hi, Carl,
>> I tried to compile the package you mentioned, but it said Hbase and
>> zookeeper are required dependencies, our environment doesn't have these two,
>> and our hadoop version is from CDH2, which we probably are not convenient to
>> upgrade to CDH3 in short term, any chance to let the sqoop v1.1.0 run with
>> CDH2 without hbase and zookeeper installed ?
>>
>> -Peter
>>
>>
>> On Mon, Sep 20, 2010 at 2:38 PM, Tianqiang Li <[EMAIL PROTECTED]> wrote:
>>
>>> Thanks Carl for you hints, let me play with this version of sqoop.
>>>
>>> -Peter
>>>
>>>
>>> On Mon, Sep 20, 2010 at 2:03 PM, Carl Steinbach <[EMAIL PROTECTED]>wrote:
>>>
>>>> Hi Peter,
>>>>
>>>> Sqoop supports both import (RDBMS --> HDFS) and export (HDFS --> RDBMS)
>>>> modes of operation. Please consult the user guide in the latest version of
>>>> Sqoop for more information:
>>>> http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz
>>>>
>>>> Thanks.
>>>>
>>>> Carl
>>>>
>>>>
>>>> On Mon, Sep 20, 2010 at 1:15 PM, Tianqiang Li <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Hi,
>>>>> I have quick questioin, are there any existing tools that are able to
>>>>> export hive table rows into table database like mysql ? I checked sqoop, but
>>>>> it seems it's for importing data to hadoop/hive, but not the other
>>>>> way(export). Any of light you'd like to shed are welcomed, thanks!
>>>>>
>>>>> Regards,
>>>>> Peter
>>>>>
>>>>
>>>>
>>>
>>
>

 Sqoop 1.0.0+3 is supported against CDH3 B3 vesion of Hadoop
(0.20.2+320). Please upgrade your Hadoop installation and try again.

Arvind

On 10/01/2010 04:09 PM, Tianqiang Li wrote:
> Hi, Carl,
> I got sometime today to install sqoop-1.0.0+3 pointed out by you,
> installation is successful, thanks, however, when I tried to use it
> and type:
>
> sqoop import --connect jdbc::mysql://mysql_host/my_db --table mytable
> --username xxx --password yyy
>
> *10/10/01 16:23:32 ERROR sqoop.Sqoop: Got exception running Sqoop:
> java.lang.RuntimeException: Could not find appropriate Hadoop shim for
> 0.20.1+169.113*
> java.lang.RuntimeException: Could not find appropriate Hadoop shim for
> 0.20.1+169.113
>         at
> com.cloudera.sqoop.shims.ShimLoader.loadShim(ShimLoader.java:190)
>         at
> com.cloudera.sqoop.shims.ShimLoader.getHadoopShim(ShimLoader.java:109)
>         at
> com.cloudera.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:141)
>         at com.cloudera.sqoop.tool.ImportTool.init(ImportTool.java:66)
>         at com.cloudera.sqoop.tool.ImportTool.run(ImportTool.java:142)
>         at com.cloudera.sqoop.Sqoop.run(Sqoop.java:134)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>         at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:170)
>         at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:196)
>         at com.cloudera.sqoop.Sqoop.main(Sqoop.java:205)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> I looked error msg up in internet, get some very related hit.
> https://issues.cloudera.org/browse/SQOOP-82
>
> Our environment is cdh2 hadoop-0.20.1, my question is that is it true
> there is no way to let sqoop-1.0.0 work on my current hadoop version,
> other than upgrading to chd3 beta-2 ? Currently I am not very tending
> to upgrade our system since some important jobs are running on top of
> it on a daily basis. Thanks.
>
> -Peter
>
> On Mon, Sep 20, 2010 at 6:03 PM, Carl Steinbach <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Hi Peter,
>
>     Please try the version of Sqoop that ships with CDH3. This version
>     does not have dependencies on ZK or HBase. You can get copy of the
>     tarball here: http://archive.cloudera.com/cdh/3/sqoop-1.0.0+3.tar.gz
>
>     Thanks.
>
>     Carl
>
>
>     On Mon, Sep 20, 2010 at 5:47 PM, Tianqiang Li <[EMAIL PROTECTED]
>     <mailto:[EMAIL PROTECTED]>> wrote:
>
>         Hi, Carl,
>         I tried to compile the package you mentioned, but it said
>         Hbase and zookeeper are required dependencies, our environment
>         doesn't have these two, and our hadoop version is from CDH2,
>         which we probably are not convenient to upgrade to CDH3 in
>         short term, any chance to let the sqoop v1.1.0 run with CDH2
>         without hbase and zookeeper installed ?
>
>         -Peter
>
>
>         On Mon, Sep 20, 2010 at 2:38 PM, Tianqiang Li
>         <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
>
>             Thanks Carl for you hints, let me play with this version
>             of sqoop.
>
>             -Peter
>
>
>             On Mon, Sep 20, 2010 at 2:03 PM, Carl Steinbach
>             <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> wrote:
>
>                 Hi Peter,
>
>                 Sqoop supports both import (RDBMS --> HDFS) and export
>                 (HDFS --> RDBMS) modes of operation. Please consult
>                 the user guide in the latest version of Sqoop for more
>                 information:
>                 http://github.com/downloads/cloudera/sqoop/sqoop-1.1.0.tar.gz