Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Issue with Sqoop column aliases


Copy link to this message
-
Re: Issue with Sqoop column aliases
Thank you!

Jarcec

On Wed, Sep 18, 2013 at 04:08:31PM +0000, Babu, Prashanth wrote:
> Thanks Jarcec.
> I have raised a bug for this.
> https://issues.apache.org/jira/browse/SQOOP-1204
>
> Regards,
> Prashanth.
>
> -----Original Message-----
> From: Jarek Jarcec Cecho [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, September 18, 2013 8:23 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Issue with Sqoop column aliases
>
> This seems like a real bug to me, would you mind opening a JIRA on [1]?
>
> Jarcec
>
> Links:
> 1: https://issues.apache.org/jira/browse/SQOOP
>
> On Wed, Sep 18, 2013 at 02:23:32AM +0000, Babu, Prashanth wrote:
> > Simon,
> >
> > I also faced this issue while importing to HBase using Sqoop.
> > I came up with the exact same workaround for my case as well.
> >
> > Regards,
> > Prashanth.
> >
> > -----Original Message-----
> > From: Simon Lee [mailto:[EMAIL PROTECTED]]
> > Sent: Tuesday, September 17, 2013 4:16 AM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Issue with Sqoop column aliases
> >
> > The same problem also occurred when importing into HBase. Sqoop does not use the alias name in the following example,and throws a "Duplicate Column" error.  
> >
> > $ sqoop import --query "select id as p_id, p.name as p_name, o.name as o_name from person as p, org as o WHERE p.org_id = o.id and $CONDITIONS "
> > --hbase-table person_and_org --column-family demographics --split-by person.id --boundary-query "select min(p.id), max(p.id) from person as p"
> >
> > 13/09/10 19:00:08 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
> > 13/09/10 19:00:08 INFO tool.CodeGenTool: Beginning code generation
> > 13/09/10 19:00:08 ERROR tool.ImportTool: Imported Failed: Duplicate Column identifier specified: 'name'
> >
> > The workaround is to use a function in the free format SQL.
> >
> > select id as p_id, concat(p.name, "") as p_name, concat(o.name, "") as
> > o_name from person as p, org as o WHERE p.org_id = o.id and
> > $CONDITIONS
> >
> >
> >
> >
> >
> > On 9/16/13 3:10 AM, "Babu, Prashanth" <[EMAIL PROTECTED]> wrote:
> >
> > >Thanks Jarcec.
> > >It's the same issue even with "import" command as well.
> > >
> > >Regards,
> > >Prashanth.
> > >
> > >
> > >-----Original Message-----
> > >From: Jarek Jarcec Cecho [mailto:[EMAIL PROTECTED]]
> > >Sent: Monday, September 16, 2013 1:39 PM
> > >To: [EMAIL PROTECTED]
> > >Subject: Re: Issue with Sqoop column aliases
> > >
> > >Hi Prashanth,
> > >I would like to mention that the "eval" tool is provided only for
> > >evaluation purpose and should not be used in production.
> > >
> > >Jarcec
> > >
> > >On Fri, Sep 13, 2013 at 02:56:22AM +0000, Babu, Prashanth wrote:
> > >> Hi,
> > >>
> > >>
> > >>
> > >> I came across an issue in Sqoop v1.4.4 import for column aliases. I
> > >>remember it was present even in earlier versions [I first saw this
> > >>issue in v1.2.0 dating back to mid-2011].
> > >>
> > >>
> > >>
> > >> Consider a Sqoop eval:
> > >>
> > >> sqoop eval --connect jdbc:mysql://my_ip/my_db --username root
> > >>--password **** --query "SELECT c.id AS cid, concat(c.firstname,
> > >>c.lastname) AS name FROM customer c LIMIT 10"
> > >>
> > >>
> > >>
> > >> The above query gets me the data as required and but one of the
> > >>column aliases is incorrect [instead of cid, it shows the column alias as id].
> > >>
> > >> id            name
> > >>
> > >>
> > >>
> > >> I have found a workaround [though it is not elegant per-se]
> > >>
> > >> sqoop eval --connect jdbc:mysql://my_ip/my_db --username root
> > >>--password **** --query "SELECT concat(c.id, '') AS cid,
> > >>concat(c.firstname, c.lastname) AS name FROM customer c LIMIT 10"
> > >>
> > >>
> > >>
> > >> The above query gets me the data as required and with the exact
> > >>column aliases I have mentioned.
> > >>
> > >> cid          name
> > >>
> > >>
> > >>
> > >> I am not sure if I am missing something here.
> > >>
> > >> Is it like if I am getting just the column, Sqoop does not consider
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB