Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Sqoop import Format Issue


+
Vineet Mishra 2013-01-18, 11:20
+
Vineet Mishra 2013-01-21, 11:20
+
Jarek Jarcec Cecho 2013-01-22, 00:49
Copy link to this message
-
Re: Sqoop import Format Issue
Hallo Everyone,

I am working with the *Sqoop Export* command and while I was trying to
export the data from HDFS to PostgreSQL Database using the command,
*
sudo -u hdfs sqoop export --connect jdbc:postgresql://localhost/mydatabase
--username vineet -P -m 1 --table testsqoop --export-dir
/user/hdfs/Sqoop/Sqooptest --input-fields-terminated-by '\0001'*

I was getting following error,

13/01/21 08:18:23 INFO manager.SqlManager: Using default fetchSize of 1000
13/01/21 08:18:23 INFO tool.CodeGenTool: Beginning code generation
13/01/21 08:18:23 INFO manager.SqlManager: Executing SQL statement: SELECT
t.* FROM "testsqoop" AS t LIMIT 1
13/01/21 08:18:23 INFO orm.CompilationManager: HADOOP_HOME is
/usr/lib/hadoop/libexec/..
13/01/21 08:18:23 INFO orm.CompilationManager: Found hadoop core jar at:
/usr/lib/hadoop/libexec/../hadoop-core.jar
Note:
/tmp/sqoop-hdfs/compile/32830bc954c996cde42e46f6a8599883/sqooptest.java
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
13/01/21 08:18:23 ERROR orm.CompilationManager: Could not make directory:
/home/serendio/.
13/01/21 08:18:23 INFO orm.CompilationManager: Writing jar file:
/tmp/sqoop-hdfs/compile/32830bc954c996cde42e46f6a8599883/sqooptest.jar
13/01/21 08:18:23 INFO mapreduce.ExportJobBase: Beginning export of
testsqoop
13/01/21 08:18:25 INFO input.FileInputFormat: Total input paths to process
: 1
13/01/21 08:18:25 INFO input.FileInputFormat: Total input paths to process
: 1
13/01/21 08:18:25 INFO mapred.JobClient: Running job: job_201301150939_0042
13/01/21 08:18:26 INFO mapred.JobClient:  map 0% reduce 0%
13/01/21 08:18:36 INFO mapred.JobClient: Task Id :
attempt_201301150939_0042_m_000000_0, Status : FAILED
java.lang.NumberFormatException
    at java.math.BigDecimal.<init>(BigDecimal.java:459)
    at java.math.BigDecimal.<init>(BigDecimal.java:728)
    at sqooptest.__loadFromFields(sqooptest.java:191)
    at sqooptest.parse(sqooptest.java:143)
    at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:77)
    at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:36)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
    at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:182)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1122)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)
I even googled but could not find any relevant solution, has anybody worked
on this issue?

Thanks in advance!

On Fri, Jan 18, 2013 at 4:50 PM, Vineet Mishra
<[EMAIL PROTECTED]>wrote:

> Hallo All,
>
> I am working with Sqoop to query from the database to for my Map/Reduce
> job, the issue I am struggling with is
>
> 1) Is there a way to directly pass the Sqoop Import Result to the Map for
> processing rather than storing it to file system and then processing.
>
> If above can't be done then let me know whether,
>
> 2) I want the format of the file which is coming from the sqoop import
> command  to be comma(,) separated single line file, but my output is coming
> as a multi row file.
>
>
> I am querying from the Database fetching a single column of some random
> values, which is coming in the format,
>
> 1234
> 1235
> 1564
> 1674
> 1546
> 2546
> 5653
> 6434
>
> the output which I was expecting was,
>
> 1234,1235,1564,1674,1546,2546,5653,6434
>
> and so on. . .
>
> Urgent!!!
> Sqoop Developers/Users please reply!
>
>
> --
> Thanks and Regards
> Vineet Mishra
>

--
Thanks and Regards
Vineet Mishra
+
abhijeet gaikwad 2013-01-21, 08:38
+
Vineet Mishra 2013-01-21, 08:43
+
abhijeet gaikwad 2013-01-21, 11:53
+
Vineet Mishra 2013-01-21, 12:01
+
Vineet Mishra 2013-01-18, 12:31
+
abhijeet gaikwad 2013-01-18, 12:41
+
abhijeet gaikwad 2013-01-18, 11:50
+
Vineet Mishra 2013-01-18, 12:06