Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - sqoop-1.4.2 --export-dir


Copy link to this message
-
Re: sqoop-1.4.2 --export-dir
Jarek Jarcec Cecho 2013-03-15, 18:49
Please do not hesitate and file a JIRA [1] for this feature request!

Jarcec

Links:
1: https://issues.apache.org/jira/browse/SQOOP

On Fri, Mar 15, 2013 at 02:38:48PM -0400, Matthieu Labour wrote:
> Jarcec,
> Thank you for your answer.
> Is it something you plan to support in the future?
> Cheers
> -matt
>
> On Fri, Mar 15, 2013 at 2:05 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Matthieu,
> > You're right, parameter --export-dir is not doing any recursive lookup for
> > files. It expects directory with files that you want export.
> >
> > Jarcec
> >
> > On Fri, Mar 15, 2013 at 01:36:50PM -0400, Matthieu Labour wrote:
> > > Hi
> > >
> > > I am using sqoop-1.4.2 to export to Sql. Thank you for creating this too.
> > >
> > > My question concerns the use of --export-dir. It seems that --export-dir
> > > does not work when the dir being passed is the root of subdirectories.
> > > Please can you advise? Thank you.
> > >
> > > Example:
> > >
> > > The following command exports the data to Sql
> > >
> > > ~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
> > > jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:
> > XXXX/xxxxxxxxxxxxx
> > > --username xxxxxxxxxx --password xxxxxxxxxx --table
> > > ml_ys_log_gmt_daily_experiment_2 --export-dir
> > > =hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
> > > --input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
> > > --batch
> > >
> > >
> > > hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
> > > hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
> > > Found 1 items
> > > -rw-r--r--   1 hadoop supergroup   15931406 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01/part-r-00001
> > >
> > > The following command does not export the data to Sql
> > >
> > > ~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
> > > jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:
> > XXXX/xxxxxxxxxxxxx
> > > --username xxxxxxxxxx --password xxxxxxxxxx --table
> > > ml_ys_log_gmt_daily_experiment_2 --export-dir
> > > =hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized
> > > --input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
> > > --batch
> > >
> > > hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
> > > hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/
> > > Found 44 items
> > > -rw-r--r--   1 hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/_SUCCESS
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-02
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-03
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-04
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-05
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-06
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-07
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-08
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-09
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-10
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-11