Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Sqoop hive import- specify hive database


Copy link to this message
-
Re: Sqoop hive import- specify hive database
Jarek Jarcec Cecho 2013-01-31, 16:23
Hi Nitin,
Sqoop will by default use string "null" as substitution for NULL values. You can override this default behaviour by using following two arguments:

  --null-string '\\N' --null-non-string '\\N'

Please note that Sqoop is using those parameters in generated source code and thus they need to be properly escaped. The example with \\N will actually results in \N being used (that is compatible with Hive). You can find more information about null substitution handling in Sqoop user guide [1].

Jarcec

Links:
1: http://s.apache.org/nq

On Wed, Jan 30, 2013 at 06:33:31PM -0500, Nitin kak wrote:
> I am using sqoop 1.4 and Hive 0.9.
>
> "Also please note that hive import is not supported when is Sqoop used from
> Oozie.". Thats strange...because I am using it.
>
> I was doing it the way you said before but there is a "null" value issue
> which comes in between. The thing is while importing to a hive table from a
> HDFS directory, the NULL values are taken as null strings(well at least for
> the non-numeric columns). This creates an inconsistency as for numeric
> columns Hive takes \N as null while for string columns, there is "null"
> string. This creates issue while exporting data back from HDFS to RDBMS.
>
> On Wed, Jan 30, 2013 at 6:22 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Nitin,
> > would you mind sharing what Sqoop and Hive version are you using? The
> > --hive-table command in form <database>.<table> is the correct and
> > supported way. However I barely remember that I faced similar issue in the
> > past, updating Hive has solved that for me.
> >
> > Also please note that hive import is not supported when is Sqoop used from
> > Oozie. Recommended workaround is to split the task into two actions - in
> > first sqoop action import data into HDFS, in second hive action load them
> > into Hive.
> >
> > Jarcec
> >
> > On Wed, Jan 30, 2013 at 06:00:01PM -0500, Nitin kak wrote:
> > > Hi All,
> > >
> > >      Is there a way to specify the Hive database in the "sqoop-import"
> > > command into hive(using --hive-import). Tried --hive-table
> > > "<database_name>.<table_name>" does not work. I ran the command using
> > Oozie
> > > but think wont work otherwise too.
> > >
> > > Thanks,
> > > Nitin
> >