Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> mysql to hbase, how to custom row-family?


Copy link to this message
-
Re: mysql to hbase, how to custom row-family?
Hi Robber,
Sqoop won't import rows where all columns that should be inserted into cell contains NULL. Would you mind taking a look if in your case the name and age are NULL? Another possibility to set the property sqoop.hbase.add.row.key. You might find more information about this property in SQOOP-870 [1] or in our user guide [1].

Jarcec

Links:
1: https://issues.apache.org/jira/browse/SQOOP-870
2: http://sqoop.apache.org/docs/1.4.3/SqoopUserGuide.html#_additional_import_configuration_properties

On Mon, May 20, 2013 at 11:02:56PM +0800, Robber Phex wrote:
> I think I can do it:
>
> HBase                       MySQL Columns
>
> row key    <-----------     id
> baseinfo   <-----------     name
>                  `-<---     age
>
> But, I use following command import zero record.
>
> sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password
> password --table testtable --hbase-table hbasetable --column-family
> baseinfo --columns "name,age" --hbase-row-key id
>
> Or,you means:
>
> sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password
> password --table testtable --hbase-table hbasetable --column-family name
> --hbase-row-key id
>
> sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password
> password --table testtable --hbase-table hbasetable --column-family age
> --hbase-row-key id
>
> Thanks.
>
> On Mon, May 20, 2013 at 10:10 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Robber,
> > Sqoop currently supports importing only into single column family. You can
> > import data twice using two different column families in each case and
> > parameter --columns to filter only relevant columns for given column family.
> >
> > Jarcec
> >
> > On Mon, May 20, 2013 at 09:42:14PM +0800, Robber Phex wrote:
> > > I have a table in mysql like this:
> > > +-------+-------------+------+-----+---------+-------+
> > > | Field | Type        | Null | Key | Default | Extra |
> > > +-------+-------------+------+-----+---------+-------+
> > > | id    | int(11)     | NO   | PRI | NULL    |       |
> > > | name  | varchar(45) | YES  |     | NULL    |       |
> > > | age   | int(11)     | YES  |     | NULL    |       |
> > > | email | varchar(20) | YES  |     | NULL    |       |
> > > | title | varchar(10) | YES  |     | NULL    |       |
> > > +-------+-------------+------+-----+---------+-------+
> > > I want to export it to hbase by sqoop.
> > > I execute this:
> > > sqoop import --connect jdbc:mysql://h2/test --username testu --password
> > > testp \
> > >   --table testtable --hbase-create-table --hbase-table hbasetable \
> > >   --column-family name --column-family age --column-family email
> > > --column-family title \
> > >   --hbase-row-key id
> > >
> > > I get full data in hbase, but I want custom row-family. Exactly I need
> > > transformation like this:
> > >
> > > HBase                       MySQL Columns
> > >
> > > row key    <-----------     id
> > > baseinfo   <-----------     name
> > >                  `-<---     age
> > > workinfo   <-----------     email
> > >                  `-<---     title
> > >
> > > Anyone have any idea?
> > >
> > > --
> > > Regards,
> > > RobberPhex
> > >
> > > About me: http://about.me/RobberPhex
> >
>
>
>
> --
> Regards,
> RobberPhex
>
> About me: http://about.me/RobberPhex
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB