Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Relevance of Java Batches project

Copy link to this message
RE: Relevance of Java Batches project
Venkatesan Ranganathan 2013-01-02, 18:20
Thanks for sharing the links.    Can you please elaborate on the capabilities that you are considering to add?

IMHO, based on my understanding of SQOOP, this looks like something orthogonal to SQOOP's usage scenarios.  Basically SQOOP uses simple insert, select and update/merge queries (including bulk, external dump/upload, batch insert/rowset fectch etc) given a table to load/extract.  There is support for free form queries in SQL in the target DB already provided by the user.  I don't think this fundamental usage model is changing in SQOOP 2 also (based on my current understanding of what it is and how it is implemented) where the intention is to  extend SQOOP  to other non-RDBMS databases.  

Looking forward to others' comments also.

BTW, my best wishes for a  happy new year to all.



Subject: Relevance of Java Batches project
Date: Wed, 2 Jan 2013 11:27:27 -0500

Hi,I've been looking over the literature on database supported program execution [1] and [2]. These look like promising ideas for incorporation into Sqoop, especially with respect to expanding the capabilities of free form queries. Has any of this literature influence Sqoop design?

[1] Haskell Boards the Ferry: Database-Supported Program Execution for Haskell George Giorgidze, Torsten Grust, Tom Schreiber, and Jeroen Weijers 22nd International Symposium on Implementation and Application of Functional Languages (IFL 2010), Alphen aan den Rijn, Netherlands, volume 6647 of Lecture Notes in Computer Science. Springer, 2011. http://db.inf.uni-tuebingen.de/files/publications/ferryhaskell.pdf[2] Remote Batch Invocation for SQL Databases, Ben Wiedermann and William R. Cook The 13th International Symposium on Database Programming Languages (DBPL), 2011. http://www.cs.utexas.edu/~wcook/Drafts/2011/batchdb.pdf