Sqoop, mail # dev - Re: Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Hari Shreedharan 2012-11-08, 22:43

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7978/
-----------------------------------------------------------

Review request for Sqoop.
Description
-------

Added MR counters support for bytes written and for rows read.
This addresses bug SQOOP-671.
   https://issues.apache.org/jira/browse/SQOOP-671
Diffs
-----

  common/src/main/java/org/apache/sqoop/submission/counter/SqoopCounters.java PRE-CREATION
  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1b3fcff
  execution/mapreduce/src/main/java/org/apache/sqoop/job/io/Data.java 4ddd132
  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 6892b4b
  spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java ba04be9

Diff: https://reviews.apache.org/r/7978/diff/
Testing
-------
Thanks,

Hari Shreedharan

Re: Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Hari Shreedharan 2012-11-08, 23:01

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7978/
-----------------------------------------------------------

(Updated Nov. 8, 2012, 11:01 p.m.)
Review request for Sqoop.
Changes
-------

Fixing compile errors in unit tests.
Description
-------

Added MR counters support for bytes written and for rows read.
This addresses bug SQOOP-671.
   https://issues.apache.org/jira/browse/SQOOP-671
Diffs (updated)
-----

  common/src/main/java/org/apache/sqoop/submission/counter/SqoopCounters.java PRE-CREATION
  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1b3fcff
  execution/mapreduce/src/main/java/org/apache/sqoop/job/io/Data.java 4ddd132
  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 6892b4b
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 812dd8e
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java e269899
  spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java ba04be9

Diff: https://reviews.apache.org/r/7978/diff/
Testing
-------
Thanks,

Hari Shreedharan

Re: Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Hari Shreedharan 2012-11-09, 03:26

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7978/
-----------------------------------------------------------

(Updated Nov. 9, 2012, 3:26 a.m.)
Review request for Sqoop.
Changes
-------

Use Sqoop counters for writing out bytes too.
Description
-------

Added MR counters support for bytes written and for rows read.
This addresses bug SQOOP-671.
   https://issues.apache.org/jira/browse/SQOOP-671
Diffs (updated)
-----

  common/src/main/java/org/apache/sqoop/submission/counter/SqoopCounters.java PRE-CREATION
  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1b3fcff
  execution/mapreduce/src/main/java/org/apache/sqoop/job/io/Data.java 4ddd132
  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 6892b4b
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 812dd8e
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java e269899
  spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java ba04be9

Diff: https://reviews.apache.org/r/7978/diff/
Testing
-------
Thanks,

Hari Shreedharan

Re: Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Hari Shreedharan 2012-11-09, 20:47

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7978/
-----------------------------------------------------------

(Updated Nov. 9, 2012, 8:47 p.m.)
Review request for Sqoop.
Changes
-------

add support for csv counting.
Description
-------

Added MR counters support for bytes written and for rows read.
This addresses bug SQOOP-671.
   https://issues.apache.org/jira/browse/SQOOP-671
Diffs (updated)
-----

  common/src/main/java/org/apache/sqoop/submission/counter/SqoopCounters.java PRE-CREATION
  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1b3fcff
  execution/mapreduce/src/main/java/org/apache/sqoop/job/io/Data.java 4ddd132
  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 6892b4b
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 812dd8e
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java e269899
  spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java ba04be9

Diff: https://reviews.apache.org/r/7978/diff/
Testing
-------
Thanks,

Hari Shreedharan

Re: Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Hari Shreedharan 2012-11-10, 22:22

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7978/
-----------------------------------------------------------

(Updated Nov. 10, 2012, 10:22 p.m.)
Review request for Sqoop.
Changes
-------

I added some comments on the jira - since jira conversations are usually better.
Description
-------

Added MR counters support for bytes written and for rows read.
This addresses bug SQOOP-671.
   https://issues.apache.org/jira/browse/SQOOP-671
Diffs (updated)
-----

  common/src/main/java/org/apache/sqoop/submission/counter/SqoopCounters.java PRE-CREATION
  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1b3fcff
  execution/mapreduce/src/main/java/org/apache/sqoop/job/io/Data.java 4ddd132
  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 6892b4b
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 812dd8e
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java e269899
  spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java ba04be9

Diff: https://reviews.apache.org/r/7978/diff/
Testing
-------
Thanks,

Hari Shreedharan

Re: Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Hari Shreedharan 2012-11-14, 08:26

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7978/
-----------------------------------------------------------

(Updated Nov. 14, 2012, 8:26 a.m.)
Review request for Sqoop.
Changes
-------

Removed bytes read from the patch. Details on the jira.
Description
-------

Added MR counters support for bytes written and for rows read.
This addresses bug SQOOP-671.
   https://issues.apache.org/jira/browse/SQOOP-671
Diffs (updated)
-----

  common/src/main/java/org/apache/sqoop/submission/counter/SqoopCounters.java PRE-CREATION
  connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1b3fcff
  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 1c0f3aa
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 875a123
  execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java 6e49cc2
  spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java ba04be9

Diff: https://reviews.apache.org/r/7978/diff/
Testing
-------
Thanks,

Hari Shreedharan

Re: Review Request: SQOOP-671. Mapreduce counters are not used in generated mapreduce jobs

Jarek Cecho 2012-11-14, 17:42

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7978/#review13429
-----------------------------------------------------------

Ship it!
Thank you Hari for your effort to make this happen. I greatly appreciate that!

Jarcec

- Jarek Cecho
On Nov. 14, 2012, 8:26 a.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7978/
> -----------------------------------------------------------
>
> (Updated Nov. 14, 2012, 8:26 a.m.)
>
>
> Review request for Sqoop.
>
>
> Description
> -------
>
> Added MR counters support for bytes written and for rows read.
>
>
> This addresses bug SQOOP-671.
> https://issues.apache.org/jira/browse/SQOOP-671
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/submission/counter/SqoopCounters.java PRE-CREATION
> connector/connector-generic-jdbc/src/main/java/org/apache/sqoop/connector/jdbc/GenericJdbcImportExtractor.java 1b3fcff
> execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopMapper.java 1c0f3aa
> execution/mapreduce/src/test/java/org/apache/sqoop/job/TestHdfsLoad.java 875a123
> execution/mapreduce/src/test/java/org/apache/sqoop/job/TestMapReduce.java 6e49cc2
> spi/src/main/java/org/apache/sqoop/job/etl/Extractor.java ba04be9
>
> Diff: https://reviews.apache.org/r/7978/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Hari Shreedharan
>
>