Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> sqoop-1.4.2 --export-dir


Copy link to this message
-
sqoop-1.4.2 --export-dir
Hi

I am using sqoop-1.4.2 to export to Sql. Thank you for creating this too.

My question concerns the use of --export-dir. It seems that --export-dir
does not work when the dir being passed is the root of subdirectories.
Please can you advise? Thank you.

Example:

The following command exports the data to Sql

~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:XXXX/xxxxxxxxxxxxx
--username xxxxxxxxxx --password xxxxxxxxxx --table
ml_ys_log_gmt_daily_experiment_2 --export-dir
=hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
--input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
--batch
hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
Found 1 items
-rw-r--r--   1 hadoop supergroup   15931406 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01/part-r-00001

The following command does not export the data to Sql

~/sqoop-1.4.2.bin__hadoop-1.0.0/bin/sqoop export --connect
jdbc:postgresql://ec2-XX-XXX-XXX-XXX.compute-1.amazonaws.com:XXXX/xxxxxxxxxxxxx
--username xxxxxxxxxx --password xxxxxxxxxx --table
ml_ys_log_gmt_daily_experiment_2 --export-dir
=hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized
--input-fields-terminated-by='\t' --lines-terminated-by='\n' --verbose
--batch

hadoop@domU-XX-XX-XX-XX-XX-XX:/mnt/var/lib/hadoop/steps/2$ hadoop fs -ls
hdfs:///mnt/var/lib/hadoop/dfs/logs_daily_sanitized/
Found 44 items
-rw-r--r--   1 hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/_SUCCESS
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-01
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-02
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-03
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-04
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-05
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-06
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-07
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-08
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-09
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-10
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-11
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-12
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-13
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-14
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-15
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-16
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-17
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-18
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-19
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-20
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-21
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:04
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-22
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-23
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-24
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-25
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-26
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-27
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-02-28
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-01
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-02
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-03
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-04
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-05
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-06
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-07
drwxr-xr-x   - hadoop supergroup          0 2013-03-15 17:03
/mnt/var/lib/hadoop/dfs/logs_daily_sanitized/dt=2013-03-08
drwxr-xr-x  
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB