Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop, mail # user - how to get job counters with sqoop 1.4.4 java api?


+
Matt_Digan@... 2013-08-07, 14:02
Copy link to this message
-
Re: how to get job counters with sqoop 1.4.4 java api?
Jarek Jarcec Cecho 2013-08-12, 00:37
Hi Matt,
Sqoop 1 do not have any officially supported Java API and thus it's not exposing all the useful information one might expect.

You might consider taking a look into Sqoop 2 as it contains fully supported Java Client API:

http://sqoop.apache.org/docs/1.99.2/ClientAPI.html

Jarcec

On Wed, Aug 07, 2013 at 02:02:29PM +0000, [EMAIL PROTECTED] wrote:
> Hi,
>
> I'm using Sqoop 1.4.4 and its java api to run an import job and I'm having
> trouble figuring out how to access the job counters once the import has
> completed. I see suitable methods in the ConfigurationHelper class, like
> getNumMapOutputRecords, but I'm not sure how to pass the job to them.
>
> Is there a way to get at the job from the SqoopTool or Sqoop objects?
>
> My code looks something like this:
>
> SqoopTool sqoopTool = new ImportTool();
>
> SqoopOptions options = new SqoopOptions();
> options.setConnectString(connectString);
> options.setUsername(username);
> options.setPassword(password);
> options.setTableName(table);
> options.setColumns(columns);
> options.setWhereClause(whereClause);
> options.setTargetDir(targerDir);
> options.setNumMappers(1);
> options.setFileLayout(FileLayout.TextFile);
> options.setFieldsTerminatedBy(delimiter);
>
> Configuration config = new Configuration();
> config.set("oracle.sessionTimeZone", timezone.getID());
>
> System.setProperty(Sqoop.SQOOP_RETHROW_PROPERTY, "1");
>
> Sqoop sqoop = new Sqoop(sqoopTool, config, options);
>
> String[] nullArgs = new String[0];
>
> Sqoop.runSqoop(sqoop, nullArgs);
>
> Thanks,
> Matt Digan
>
>
>