Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Issue with Sqoop column aliases


Copy link to this message
-
Re: Issue with Sqoop column aliases
Thank you!

Jarcec

On Wed, Sep 18, 2013 at 04:08:31PM +0000, Babu, Prashanth wrote:
> Thanks Jarcec.
> I have raised a bug for this.
> https://issues.apache.org/jira/browse/SQOOP-1204
>
> Regards,
> Prashanth.
>
> -----Original Message-----
> From: Jarek Jarcec Cecho [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, September 18, 2013 8:23 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Issue with Sqoop column aliases
>
> This seems like a real bug to me, would you mind opening a JIRA on [1]?
>
> Jarcec
>
> Links:
> 1: https://issues.apache.org/jira/browse/SQOOP
>
> On Wed, Sep 18, 2013 at 02:23:32AM +0000, Babu, Prashanth wrote:
> > Simon,
> >
> > I also faced this issue while importing to HBase using Sqoop.
> > I came up with the exact same workaround for my case as well.
> >
> > Regards,
> > Prashanth.
> >
> > -----Original Message-----
> > From: Simon Lee [mailto:[EMAIL PROTECTED]]
> > Sent: Tuesday, September 17, 2013 4:16 AM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Issue with Sqoop column aliases
> >
> > The same problem also occurred when importing into HBase. Sqoop does not use the alias name in the following example,and throws a "Duplicate Column" error.  
> >
> > $ sqoop import --query "select id as p_id, p.name as p_name, o.name as o_name from person as p, org as o WHERE p.org_id = o.id and $CONDITIONS "
> > --hbase-table person_and_org --column-family demographics --split-by person.id --boundary-query "select min(p.id), max(p.id) from person as p"
> >
> > 13/09/10 19:00:08 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> > 13/09/10 19:00:08 INFO tool.CodeGenTool: Beginning code generation
> > 13/09/10 19:00:08 ERROR tool.ImportTool: Imported Failed: Duplicate Column identifier specified: 'name'
> >
> > The workaround is to use a function in the free format SQL.
> >
> > select id as p_id, concat(p.name, "") as p_name, concat(o.name, "") as
> > o_name from person as p, org as o WHERE p.org_id = o.id and
> > $CONDITIONS
> >
> >
> >
> >
> >
> > On 9/16/13 3:10 AM, "Babu, Prashanth" <[EMAIL PROTECTED]> wrote:
> >
> > >Thanks Jarcec.
> > >It's the same issue even with "import" command as well.
> > >
> > >Regards,
> > >Prashanth.
> > >
> > >
> > >-----Original Message-----
> > >From: Jarek Jarcec Cecho [mailto:[EMAIL PROTECTED]]
> > >Sent: Monday, September 16, 2013 1:39 PM
> > >To: [EMAIL PROTECTED]
> > >Subject: Re: Issue with Sqoop column aliases
> > >
> > >Hi Prashanth,
> > >I would like to mention that the "eval" tool is provided only for
> > >evaluation purpose and should not be used in production.
> > >
> > >Jarcec
> > >
> > >On Fri, Sep 13, 2013 at 02:56:22AM +0000, Babu, Prashanth wrote:
> > >> Hi,
> > >>
> > >>
> > >>
> > >> I came across an issue in Sqoop v1.4.4 import for column aliases. I
> > >>remember it was present even in earlier versions [I first saw this
> > >>issue in v1.2.0 dating back to mid-2011].
> > >>
> > >>
> > >>
> > >> Consider a Sqoop eval:
> > >>
> > >> sqoop eval --connect jdbc:mysql://my_ip/my_db --username root
> > >>--password **** --query "SELECT c.id AS cid, concat(c.firstname,
> > >>c.lastname) AS name FROM customer c LIMIT 10"
> > >>
> > >>
> > >>
> > >> The above query gets me the data as required and but one of the
> > >>column aliases is incorrect [instead of cid, it shows the column alias as id].
> > >>
> > >> id            name
> > >>
> > >>
> > >>
> > >> I have found a workaround [though it is not elegant per-se]
> > >>
> > >> sqoop eval --connect jdbc:mysql://my_ip/my_db --username root
> > >>--password **** --query "SELECT concat(c.id, '') AS cid,
> > >>concat(c.firstname, c.lastname) AS name FROM customer c LIMIT 10"
> > >>
> > >>
> > >>
> > >> The above query gets me the data as required and with the exact
> > >>column aliases I have mentioned.
> > >>
> > >> cid          name
> > >>
> > >>
> > >>
> > >> I am not sure if I am missing something here.
> > >>
> > >> Is it like if I am getting just the column, Sqoop does not consider