Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Java Client for Sqoop Import


+
Vineet Mishra 2014-12-12, 11:36
+
Vineet Mishra 2014-12-14, 22:00
+
Abraham Elmahrek 2014-12-15, 01:39
+
Vineet Mishra 2014-12-15, 11:54
+
Sethuramaswamy, Suresh 2014-12-15, 14:41
Copy link to this message
-
RE: Java Client for Sqoop Import
Another option if you don’t want to write the output to a file you can add your own appender in log4j. You can then collect the log entries with your custom appender into memory and scan it there.

From: Sethuramaswamy, Suresh [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, 16 December 2014 1:39 AM
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Subject: RE: Java Client for Sqoop Import

Hi Vineet,

Writing the output log to a file and grep for the count  may be an option to try out.

$(grep -oP 'Retrieved \K\d*' <logfileName>)

Suresh Sethuramaswamy

From: Vineet Mishra [mailto:[EMAIL PROTECTED]]
Sent: Monday, December 15, 2014 6:54 AM
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Subject: Re: Java Client for Sqoop Import

Hi Abe,

Thanks for your mail, well I tried this link and able to make Sqoop invocation possible from a java code snippet.

http://www.acnenomor.com/451103p2/netbeanshadoop-errors-sqoophome-is-unset-may-not-be-able-to-find-all-job-dependencieshadoophome-is-not-set

I am much looking for some way of getting the number of records processed through sqoop import

INFO mapreduce.ImportJobBase: Retrieved 5 records.

so as to avoid another level of processing getting the import counts, I can get the records in the console but unfortunately not in the Sqoop.runTool since its not having access to any possible methods for getting processed result counts.

Looking forward for any workaround to get this hack.

Regards!

On Mon, Dec 15, 2014 at 7:08 AM, Abraham Elmahrek <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hey man,

Have you tried Sqoop2's Java client? It would currently only be useful when transfering data FROM/TO HDFS. This is changing fast though.

-Abe

On Sun, Dec 14, 2014 at 1:59 PM, Vineet Mishra <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Has Anyone picked this up!

On Fri, Dec 12, 2014 at 5:06 PM, Vineet Mishra <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi,

I am looking for some client for running sqoop jobs in Java. Although I have easily imported data through sqoop shell command but looking out for the other alternative through implementing the same in Java.

On urgent call!
==============================================================================
Please access the attached hyperlink for an important electronic communications disclaimer:
http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
==============================================================================

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB