Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Support of arrays in fields


+
Ruslan Al-Fakikh 2013-04-09, 16:40
+
Jarek Jarcec Cecho 2013-04-15, 04:58
+
Ruslan Al-Fakikh 2013-04-19, 21:08
+
Jarek Jarcec Cecho 2013-04-19, 21:55
Copy link to this message
-
Re: Support of arrays in fields
Ok, I see.
I appreciate the clarification. Here is the newly created issue:
https://issues.apache.org/jira/browse/SQOOP-1000

Ruslan
On Sat, Apr 20, 2013 at 1:55 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:

> Hi Ruslan,
> we tried to do bulk import from Cloudera JIRA instance into ASF instance
> when we were incubating Sqoop. Unfortunately we've run into issues, so we
> decided to keep the Cloudera instance read only and create corresponding
> ASF JIRAs on per needed basis. Please check out INFRA-3763 [1] for more
> background.
>
> Jarcec
>
> Links:
> 1: https://issues.apache.org/jira/browse/INFRA-3763
>
> On Sat, Apr 20, 2013 at 01:08:36AM +0400, Ruslan Al-Fakikh wrote:
> > Jarek,
> >
> > This is a second time :-) you are asking me to open a jira on Apache that
> > is already present on Cloudera. If you are saying that Cloudera site is
> no
> > longer used:
> >
> https://issues.apache.org/jira/browse/SQOOP-390?focusedCommentId=13631487&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13631487
> > then why haven't all its issues been migrated in batch?
> >
> > Thanks
> >
> >
> > On Mon, Apr 15, 2013 at 8:58 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]
> >wrote:
> >
> > > Hi Ruslan,
> > > I'm afraid that Sqoop currently do not supports arrays natively. The
> > > import case can be workaround by using array_to_string function, but
> I'm
> > > not sure how to easily workaround export. Would you mind opening a new
> JIRA
> > > on Apache JIRA [1] for that?
> > >
> > > Jarcec
> > >
> > > Links:
> > > 1: https://issues.apache.org/jira/browse/SQOOP
> > >
> > > On Tue, Apr 09, 2013 at 08:40:54PM +0400, Ruslan Al-Fakikh wrote:
> > > > Hey guys,
> > > >
> > > > Sorry for raising this old question, but is there a workaround for
> > > > uploading data with arrays in fields?
> > > > I have a table like this:
> > > > CREATE TABLE tablename
> > > > (
> > > > image_urls character varying(300)[]
> > > > );
> > > > and I am uplooading from a file on HDFS. Basically I can change a
> format
> > > of
> > > > the file, but in what form should I do it in order to make Sqoop
> upload
> > > it
> > > > to this unsupported data type? Maybe there is a workaround.
> > > >
> > > > Also I saw this issue, but it is still unresolved:
> > > > https://issues.cloudera.org/browse/SQOOP-160
> > > >
> > > > Any help would be appreciated
> > > >
> > > >
> > > > On Fri, Aug 24, 2012 at 10:07 AM, Jarek Jarcec Cecho <
> [EMAIL PROTECTED]
> > > >wrote:
> > > >
> > > > > You might consider utilizing Postgresql's function array_to_string
> to
> > > > > "join" the array into one string. You would have to change the
> import
> > > from
> > > > > --table to --query then thought.
> > > > >
> > > > > Jarcec
> > > > >
> > > > > On Fri, Aug 24, 2012 at 10:40:46AM +0530, Adarsh Sharma wrote:
> > > > > > Thanks Jarcec for the update, So Sqoop is not suitable for
> shifting
> > > data
> > > > > > from DB to HDFS , if some columns have integer[] or bigint[]
> > > datatypes.
> > > > > >
> > > > > > Is there any way i can sh*ift  date having bigint[] datatypes
> from*
> > > > > postgresql
> > > > > > DB to HDFS using Sqoop or I need to test another tool like Talend
> > > etc.
> > > > > >
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > >
> > > > > > On Thu, Aug 23, 2012 at 11:45 PM, Jarek Jarcec Cecho <
> > > [EMAIL PROTECTED]
> > > > > >wrote:
> > > > > >
> > > > > > > Hi Adarsh,
> > > > > > > as far as I know then Sqoop should not have any issues with
> bigint
> > > data
> > > > > > > type.
> > > > > > >
> > > > > > > Based on provided log fragment, It seems that you're having
> issues
> > > with
> > > > > > > SQL type 2003 that should be ARRAY (see 1). I'm afraid that
> array
> > > is
> > > > > really
> > > > > > > not supported in Sqoop at the moment.
> > > > > > >
> > > > > > > Jarcec
> > > > > > >
> > > > > > > 1:
> > > > > > >
> > > > >
> > >
> http://docs.oracle.com/javase/1.5.0/docs/api/constant-values.html#java.sql.Types.ARRAY
> > > > > >