Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> sqoop-1.4.2 --export-dir


Copy link to this message
-
Re: sqoop-1.4.2 --export-dir
Please do not hesitate and file a JIRA [1] for this feature request!

Jarcec

Links:
1: https://issues.apache.org/jira/browse/SQOOP

On Fri, Mar 15, 2013 at 02:38:48PM -0400, Matthieu Labour wrote:
> Jarcec,
> Thank you for your answer.
> Is it something you plan to support in the future?
> Cheers
> -matt
>
> On Fri, Mar 15, 2013 at 2:05 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>
> > Hi Matthieu,
> > You're right, parameter --export-dir is not doing any recursive lookup for
> > files. It expects directory with files that you want export.
> >
> > Jarcec
> >
> > On Fri, Mar 15, 2013 at 01:36:50PM -0400, Matthieu Labour wrote:
> > > Hi
> > >
> > > I am using sqoop-1.4.2 to export to Sql. Thank you for creating this too.
> > >
> > > My question concerns the use of --export-dir. It seems that --export-dir
> > > does not work when the dir being passed is the root of subdirectories.
> > > Please can you advise? Thank you.
> > >
> > > Example:
> > >
> > > The following command exports the data to Sql
> > >
> > > ~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
> > > jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:
> > XXXX/xxxxxxxxxxxxx
> > > --username xxxxxxxxxx --password xxxxxxxxxx --table
> > > ml_ys_log_gmt_daily_experiment_2 --export-dir
> > > =hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
> > > --input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
> > > --batch
> > >
> > >
> > > hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
> > > hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
> > > Found 1 items
> > > -rw-r--r--   1 hadoop supergroup   15931406 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01/part-r-00001
> > >
> > > The following command does not export the data to Sql
> > >
> > > ~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
> > > jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:
> > XXXX/xxxxxxxxxxxxx
> > > --username xxxxxxxxxx --password xxxxxxxxxx --table
> > > ml_ys_log_gmt_daily_experiment_2 --export-dir
> > > =hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized
> > > --input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
> > > --batch
> > >
> > > hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
> > > hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/
> > > Found 44 items
> > > -rw-r--r--   1 hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/_SUCCESS
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-02
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-03
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-04
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-05
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-06
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-07
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-08
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-09
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-10
> > > drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
> > > /mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB