Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Can sqoop import data from mysql to cassandra


Copy link to this message
-
Re: Can sqoop import data from mysql to cassandra
But if Cassandra supports JDBC interface, it should work and may be
suboptimal.

On Wed, Aug 8, 2012 at 3:47 AM, [EMAIL PROTECTED] <
[EMAIL PROTECTED]> wrote:

> Hi Jarcec,
>
> Thanks a lot.
>
> Regards
> Rajesh Kumar
>
> On Wednesday 08 August 2012 04:05 PM, Jarek Jarcec Cecho wrote:
>
>> Hi Rajesh,
>> as far as I know Apache sqoop do not have build-in support for Cassandra
>> at the moment.
>>
>> Jarcec
>>
>> On Wed, Aug 08, 2012 at 03:19:07PM +0530, [EMAIL PROTECTED] wrote:
>>
>>> Hi Everyone,
>>> Can sqoop version 1.4.1 or any other version, import data from mysql
>>> database to cassandra (into cassandra file system rather than hive
>>> or HDFS).
>>> If so whats the command for it.
>>>
>>> Thanks
>>> Rajesh Kumar
>>> On Wednesday 08 August 2012 01:13 AM, Jarek Jarcec Cecho wrote:
>>>
>>>> Hi Rajesh,
>>>> this seems as a custom build provided by third party. I'm afraid that
>>>> we can't help you as the exception itself is not generated by Sqoop, but
>>>> some additional code that we're simply not familiar with. I would advise
>>>> contacting original source of your build for more help.
>>>>
>>>> Jarcec
>>>>
>>>> On Tue, Aug 07, 2012 at 12:19:15PM +0530, [EMAIL PROTECTED]rote:
>>>>
>>>>> Hi Jarcec,
>>>>>
>>>>> I am using sqoop utility provided by Datastax in their Enterprise
>>>>> version: Sqoop 1.4.1-dse-SNAPSHOT
>>>>>
>>>>> I am using the following command to import :
>>>>>
>>>>> ./dse sqoop import --connect jdbc:mysql://192.168.1.102/**npa_nxx_demo<http://192.168.1.102/npa_nxx_demo>
>>>>> --username root --password *** --table npa_nxx --cassandra-keyspace
>>>>> pdc_crawler --cassandra-column-family npa_nxx_cf --cassandra-row-key
>>>>> npa_nxx_key --cassandra-thrift-host 192.168.0.203,192.168.0.224
>>>>> --cassandra-create-schema --verbose
>>>>>
>>>>> This is the verbose output :
>>>>> ./dse sqoop import --connect jdbc:mysql://192.168.0.102/**npa_nxx_demo<http://192.168.0.102/npa_nxx_demo>
>>>>> --username root --password orka1 --table npa_nxx
>>>>> --cassandra-keyspace pdc_crawler --cassandra-column-family
>>>>> npa_nxx_cf --cassandra-row-key npa_nxx_key --cassandra-thrift-host
>>>>> 192.168.0.201,192.168.0.202 --cassandra-create-schema --verbose
>>>>> 12/08/07 12:12:26 WARN tool.BaseSqoopTool: Setting your password on
>>>>> the command-line is insecure. Consider using -P instead.
>>>>> 12/08/07 12:12:26 INFO manager.MySQLManager: Preparing to use a
>>>>> MySQL streaming resultset.
>>>>> 12/08/07 12:12:26 INFO tool.CodeGenTool: Beginning code generation
>>>>> 12/08/07 12:12:27 INFO manager.SqlManager: Executing SQL statement:
>>>>> SELECT t.* FROM `npa_nxx` AS t LIMIT 1
>>>>> 12/08/07 12:12:27 INFO orm.CompilationManager: HADOOP_HOME is
>>>>> /home/Downloads/dse-2.1/**resources/hadoop/bin/..
>>>>> Note:
>>>>> /tmp/sqoop-root/compile/**474195de4276c8abd8d19c14d4daed**
>>>>> a5/npa_nxx.java
>>>>> uses or overrides a deprecated API.
>>>>> Note: Recompile with -Xlint:deprecation for details.
>>>>> 12/08/07 12:12:28 INFO orm.CompilationManager: Writing jar file:
>>>>> /tmp/sqoop-root/compile/**474195de4276c8abd8d19c14d4daed**
>>>>> a5/npa_nxx.jar
>>>>> 12/08/07 12:12:28 WARN manager.MySQLManager: It looks like you are
>>>>> importing from mysql.
>>>>> 12/08/07 12:12:28 WARN manager.MySQLManager: This transfer can be
>>>>> faster! Use the --direct
>>>>> 12/08/07 12:12:28 WARN manager.MySQLManager: option to exercise a
>>>>> MySQL-specific fast path.
>>>>> 12/08/07 12:12:28 INFO manager.MySQLManager: Setting zero DATETIME
>>>>> behavior to convertToNull (mysql)
>>>>> 12/08/07 12:12:28 INFO mapreduce.ImportJobBase: Beginning import of
>>>>> npa_nxx
>>>>> 12/08/07 12:12:31 ERROR sqoop.Sqoop: Got exception running Sqoop:
>>>>> org.apache.cassandra.db.**marshal.MarshalException: 97 is not
>>>>> recognized as a valid type
>>>>> org.apache.cassandra.db.**marshal.MarshalException: 97 is not
>>>>> recognized as a valid type
>>>>>      at com.datastax.bdp.util.**CompositeUtil.deserialize(**
>>>>> CompositeUtil.java:93)
Regards,
Venkatesh

Phone: (408) 658-8368
EMail: [EMAIL PROTECTED]

http://in.linkedin.com/in/seetharamvenkatesh
http://about.me/SeetharamVenkatesh

“Perfection (in design) is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.”
- Antoine de Saint-Exupéry
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB