Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> how to get job counters with sqoop 1.4.4 java api?


+
Matt_Digan@... 2013-08-07, 14:02
Copy link to this message
-
Re: how to get job counters with sqoop 1.4.4 java api?
Hi Matt,
Sqoop 1 do not have any officially supported Java API and thus it's not exposing all the useful information one might expect.

You might consider taking a look into Sqoop 2 as it contains fully supported Java Client API:

http://sqoop.apache.org/docs/1.99.2/ClientAPI.html

Jarcec

On Wed, Aug 07, 2013 at 02:02:29PM +0000, [EMAIL PROTECTED] wrote:
> Hi,
>
> I'm using Sqoop 1.4.4 and its java api to run an import job and I'm having
> trouble figuring out how to access the job counters once the import has
> completed. I see suitable methods in the ConfigurationHelper class, like
> getNumMapOutputRecords, but I'm not sure how to pass the job to them.
>
> Is there a way to get at the job from the SqoopTool or Sqoop objects?
>
> My code looks something like this:
>
> SqoopTool sqoopTool = new ImportTool();
>
> SqoopOptions options = new SqoopOptions();
> options.setConnectString(connectString);
> options.setUsername(username);
> options.setPassword(password);
> options.setTableName(table);
> options.setColumns(columns);
> options.setWhereClause(whereClause);
> options.setTargetDir(targerDir);
> options.setNumMappers(1);
> options.setFileLayout(FileLayout.TextFile);
> options.setFieldsTerminatedBy(delimiter);
>
> Configuration config = new Configuration();
> config.set("oracle.sessionTimeZone", timezone.getID());
>
> System.setProperty(Sqoop.SQOOP_RETHROW_PROPERTY, "1");
>
> Sqoop sqoop = new Sqoop(sqoopTool, config, options);
>
> String[] nullArgs = new String[0];
>
> Sqoop.runSqoop(sqoop, nullArgs);
>
> Thanks,
> Matt Digan
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB