Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> mysql to hbase, how to custom row-family?


Copy link to this message
-
Re: mysql to hbase, how to custom row-family?
Hi Robber,
Sqoop won't import rows where all columns that should be inserted into cell contains NULL. Would you mind taking a look if in your case the name and age are NULL? Another possibility to set the property sqoop.hbase.add.row.key. You might find more information about this property in SQOOP-870 [1] or in our user guide [1].

Jarcec

Links:
1: https://issues.apache.org/jira/browse/SQOOP-870
2: http://sqoop.apache.org/docs/1.4.3/SqoopUserGuide.html#_additional_import_configuration_properties

On Mon, May 20, 2013 at 11:02:56PM +0800, Robber Phex wrote:
> I think I can do it:
>
> HBase                       MySQL Columns
>
> row key    <-----------     id
> baseinfo   <-----------     name
>                  `-<---     age
>
> But, I use following command import zero record.
>
> sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password
> password --table testtable --hbase-table hbasetable --column-family
> baseinfo --columns "name,age" --hbase-row-key id
>
> Or,you means:
>
> sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password
> password --table testtable --hbase-table hbasetable --column-family name
> --hbase-row-key id
>
> sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password
> password --table testtable --hbase-table hbasetable --column-family age
> --hbase-row-key id
>
> Thanks.
>
> On Mon, May 20, 2013 at 10:10 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Robber,
> > Sqoop currently supports importing only into single column family. You can
> > import data twice using two different column families in each case and
> > parameter --columns to filter only relevant columns for given column family.
> >
> > Jarcec
> >
> > On Mon, May 20, 2013 at 09:42:14PM +0800, Robber Phex wrote:
> > > I have a table in mysql like this:
> > > +-------+-------------+------+-----+---------+-------+
> > > | Field | Type        | Null | Key | Default | Extra |
> > > +-------+-------------+------+-----+---------+-------+
> > > | id    | int(11)     | NO   | PRI | NULL    |       |
> > > | name  | varchar(45) | YES  |     | NULL    |       |
> > > | age   | int(11)     | YES  |     | NULL    |       |
> > > | email | varchar(20) | YES  |     | NULL    |       |
> > > | title | varchar(10) | YES  |     | NULL    |       |
> > > +-------+-------------+------+-----+---------+-------+
> > > I want to export it to hbase by sqoop.
> > > I execute this:
> > > sqoop import --connect jdbc:mysql://h2/test --username testu --password
> > > testp \
> > >   --table testtable --hbase-create-table --hbase-table hbasetable \
> > >   --column-family name --column-family age --column-family email
> > > --column-family title \
> > >   --hbase-row-key id
> > >
> > > I get full data in hbase, but I want custom row-family. Exactly I need
> > > transformation like this:
> > >
> > > HBase                       MySQL Columns
> > >
> > > row key    <-----------     id
> > > baseinfo   <-----------     name
> > >                  `-<---     age
> > > workinfo   <-----------     email
> > >                  `-<---     title
> > >
> > > Anyone have any idea?
> > >
> > > --
> > > Regards,
> > > RobberPhex
> > >
> > > About me: http://about.me/RobberPhex
> >
>
>
>
> --
> Regards,
> RobberPhex
>
> About me: http://about.me/RobberPhex