Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive Tez Counters


Copy link to this message
-
Hive Tez Counters
Hi,

Needed info on where I can get detailed job counters for Hive on Tez. Am
running this on a HDP cluster with Hive 0.13 and see only the following job
counters through Hive Tez in Yarn logs . Cannot see any ReduceOperator
counters and the CPU_MILLISECONDS in some cases in -ve. Is this accurate?

[[File System Counters
FILE: BYTES_READ=512,
FILE: BYTES_WRITTEN=3079881,
FILE: READ_OPS=0, FILE: LARGE_READ_OPS=0, FILE: WRITE_OPS=0, HDFS:
BYTES_READ=8215153, HDFS: BYTES_WRITTEN=0, HDFS: READ_OPS=3, HDFS:
LARGE_READ_OPS=0, HDFS: WRITE_OPS=0]

[org.apache.tez.common.counters.TaskCounter SPILLED_RECORDS=222543,
GC_TIME_MILLIS=172, CPU_MILLISECONDS=5830, PHYSICAL_MEMORY_BYTES=667566080,
VIRTUAL_MEMORY_BYTES=1887797248, COMMITTED_HEAP_BYTES=1011023872,
INPUT_RECORDS_PROCESSED=222543,
OUTPUT_RECORDS=222543,
OUTPUT_BYTES=23543896,
OUTPUT_BYTES_WITH_OVERHEAD=23989024, OUTPUT_BYTES_PHYSICAL=3079369,
ADDITIONAL_SPILLS_BYTES_WRITTEN=0, ADDITIONAL_SPILLS_BYTES_READ=0,
ADDITIONAL_SPILL_COUNT=0]
[org.apache.hadoop.hive.ql.exec.MapOperator$Counter DESERIALIZE_ERRORS=0]]

Thanks
Suma