Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - sqoop-1.4.2 --export-dir


Copy link to this message
-
sqoop-1.4.2 --export-dir
Matthieu Labour 2013-03-15, 17:36
Hi

I am using sqoop-1.4.2 to export to Sql. Thank you for creating this too.

My question concerns the use of --export-dir. It seems that --export-dir
does not work when the dir being passed is the root of subdirectories.
Please can you advise? Thank you.

Example:

The following command exports the data to Sql

~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:XXXX/xxxxxxxxxxxxx
--username xxxxxxxxxx --password xxxxxxxxxx --table
ml_ys_log_gmt_daily_experiment_2 --export-dir
=hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
--input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
--batch
hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
Found 1 items
-rw-r--r--   1 hadoop supergroup   15931406 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01/part-r-00001

The following command does not export the data to Sql

~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:XXXX/xxxxxxxxxxxxx
--username xxxxxxxxxx --password xxxxxxxxxx --table
ml_ys_log_gmt_daily_experiment_2 --export-dir
=hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized
--input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
--batch

hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/
Found 44 items
-rw-r--r--   1 hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/_SUCCESS
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-02
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-03
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-04
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-05
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-06
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-07
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-08
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-09
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-10
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-11
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-12
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-13
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-14
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-15
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-16
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-17
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-18
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-19
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-20
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-21
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-22
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-23
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-24
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-25
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-26
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-27
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-28
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-01
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-02
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-03
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-04
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-05
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-06
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-07
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-08
drwxr-xr-x