Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Creating compressed data with scooop


Copy link to this message
-
Re: Creating compressed data with scooop
Hi Jarcec,

Are you referring to task logs ? in below screen. When I click on "All" or
"Last 4KB" or "Last 8KB', the page times out :-( Not sure why

task_201301072354_0114_m_000000 attempts for
0114_1357651667175_saachhra<http://sl73caeq02.visa.com:50030/jobdetailshistory.jsp?logFile=file:/hadoop1/logs/hadoop/history/done/sl73caeq02.visa.com_1357602885590_/2013/00/08/000000/job_201301072354_0114_1357651667175_saachhra_optqc5d.TQC_ELIG_TRAN_HIST.jar>
Task
IdStart TimeFinish Time HostErrorTask Logs Counters
attempt_201301072354_0114_m_000000_0 8/01 13:27:538/01 13:27:58 (5sec)
/default/host Last
4KB<http://sl73caeq03.visa.com:50060/tasklog?attemptid=attempt_201301072354_0114_m_000000_0&start=-4097>
Last 8KB<http://sl73caeq03.visa.com:50060/tasklog?attemptid=attempt_201301072354_0114_m_000000_0&start=-8193>
All<http://sl73caeq03.visa.com:50060/tasklog?attemptid=attempt_201301072354_0114_m_000000_0&all=true>
18<http://sl73caeq02.visa.com:50030/taskstatshistory.jsp?attemptid=attempt_201301072354_0114_m_000000_0&logFile=file:/hadoop1/logs/hadoop/history/done/sl73caeq02.visa.com_1357602885590_/2013/00/08/000000/job_201301072354_0114_1357651667175_saachhra_optqc5d.TQC_ELIG_TRAN_HIST.jar>
However I got the location on linux here it was trying to look.  I am
afraid I wont be able to attach file or put full contents in this email :-(
Is this what you were looking for (See below) ?
- <property>
  <name>mapred.compress.map.output</name>
  <value>true</value>
  <source>mapred-site.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>

- <property>
  <name>io.seqfile.lazydecompress</name>
  <value>true</value>
  <source>core-default.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>

 - <property>
  <name>io.compression.codecs</name>

<value>org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.DeflateCodec,org.apache.hadoop.io.compress.SnappyCodec</value>
  <source>core-site.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>
- <property>
  <name>mapred.output.compression.type</name>
  <value>BLOCK</value>
  <source>mapred-site.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>
- <property>
  <name>mapred.output.compression.codec</name>
  <value>org.apache.hadoop.io.compress.GzipCodec</value>
  <source>programatically</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>

- <property>
  <name>mapred.map.output.compression.codec</name>
  <value>org.apache.hadoop.io.compress.SnappyCodec</value>
  <source>mapred-site.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>
- <property>
  <name>io.seqfile.compress.blocksize</name>
  <value>1000000</value>
  <source>core-default.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>

- <property>
  <name>dfs.image.compress</name>
  <value>false</value>
  <source>hdfs-default.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>

- <property>
  <name>dfs.image.compression.codec</name>
  <value>org.apache.hadoop.io.compress.DefaultCodec</value>
  <source>hdfs-default.xml</source>
  <source>/data/mapred/jt/jobTracker/job_201301072354_0114.xml</source>
  </property>

Good wishes,always !
Santosh
On Thu, Jan 10, 2013 at 5:40 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:

> Hi Santosh,
> almost :-). When you click on this page on the "map" link you should get
> page with map tasks. When you click on one arbitrary task you should get
> table with all corresponding task attempts. On this page you should see
> link to log.
>
> Jarcec
>
> On Wed, Jan 09, 2013 at 07:45:44PM +0800, Santosh Achhra wrote:
> > Hi Jarcec,
> >
> > Hopefully this is log which you had requested.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB