Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop import Format Issue


+
Vineet Mishra 2013-01-18, 11:20
+
Vineet Mishra 2013-01-21, 11:20
+
Jarek Jarcec Cecho 2013-01-22, 00:49
Copy link to this message
-
Re: Sqoop import Format Issue
Hallo Everyone,

I am working with the *Sqoop Export* command and while I was trying to
export the data from HDFS to PostgreSQL Database using the command,
*
sudo -u hdfs sqoop export --connect jdbc:postgresql://localhost/mydatabase
--username vineet -P -m 1 --table testsqoop --export-dir
/user/hdfs/Sqoop/Sqooptest --input-fields-terminated-by '\0001'*

I was getting following error,

13/01/21 08:18:23 INFO manager.SqlManager: Using default fetchSize of 1000
13/01/21 08:18:23 INFO tool.CodeGenTool: Beginning code generation
13/01/21 08:18:23 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM "testsqoop" AS t LIMIT 1
13/01/21 08:18:23 INFO orm.CompilationManager: HADOOP_HOME is
/usr/lib/hadoop/libexec/..
13/01/21 08:18:23 INFO orm.CompilationManager: Found hadoop core jar at:
/usr/lib/hadoop/libexec/../hadoop-core.jar
Note:
/tmp/sqoop-hdfs/compile/32830bc954c996cde42e46f6a8599883/sqooptest.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/01/21 08:18:23 ERROR orm.CompilationManager: Could not make directory:
/home/serendio/.
13/01/21 08:18:23 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hdfs/compile/32830bc954c996cde42e46f6a8599883/sqooptest.jar
13/01/21 08:18:23 INFO mapreduce.ExportJobBase: Beginning export of
testsqoop
13/01/21 08:18:25 INFO input.FileInputFormat: Total input paths to process
: 1
13/01/21 08:18:25 INFO input.FileInputFormat: Total input paths to process
: 1
13/01/21 08:18:25 INFO mapred.JobClient: Running job: job_201301150939_0042
13/01/21 08:18:26 INFO mapred.JobClient:  map 0% reduce 0%
13/01/21 08:18:36 INFO mapred.JobClient: Task Id :
attempt_201301150939_0042_m_000000_0, Status : FAILED
java.lang.NumberFormatException
    at java.math.BigDecimal.<init>(BigDecimal.java:459)
    at java.math.BigDecimal.<init>(BigDecimal.java:728)
    at sqooptest.__loadFromFields(sqooptest.java:191)
    at sqooptest.parse(sqooptest.java:143)
    at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:77)
    at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1122)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
I even googled but could not find any relevant solution, has anybody worked
on this issue?

Thanks in advance!

On Fri, Jan 18, 2013 at 4:50 PM, Vineet Mishra
<[EMAIL PROTECTED]>wrote:

> Hallo All,
>
> I am working with Sqoop to query from the database to for my Map/Reduce
> job, the issue I am struggling with is
>
> 1) Is there a way to directly pass the Sqoop Import Result to the Map for
> processing rather than storing it to file system and then processing.
>
> If above can't be done then let me know whether,
>
> 2) I want the format of the file which is coming from the sqoop import
> command  to be comma(,) separated single line file, but my output is coming
> as a multi row file.
>
>
> I am querying from the Database fetching a single column of some random
> values, which is coming in the format,
>
> 1234
> 1235
> 1564
> 1674
> 1546
> 2546
> 5653
> 6434
>
> the output which I was expecting was,
>
> 1234,1235,1564,1674,1546,2546,5653,6434
>
> and so on. . .
>
> Urgent!!!
> Sqoop Developers/Users please reply!
>
>
> --
> Thanks and Regards
> Vineet Mishra
>

--
Thanks and Regards
Vineet Mishra
+
abhijeet gaikwad 2013-01-21, 08:38
+
Vineet Mishra 2013-01-21, 08:43
+
abhijeet gaikwad 2013-01-21, 11:53
+
Vineet Mishra 2013-01-21, 12:01
+
Vineet Mishra 2013-01-18, 12:31
+
abhijeet gaikwad 2013-01-18, 12:41
+
abhijeet gaikwad 2013-01-18, 11:50
+
Vineet Mishra 2013-01-18, 12:06
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB