Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> scrach space grows too fast


Copy link to this message
-
RE: RE: scrach space grows too fast
Probably not the end result but the intermediate results are there (which are usually bigger then the end result).
I never used hwi myself.

________________________________
From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 11:00 AM
To: hive-user
Subject: Re: RE: scrach space grows too fast

thank you for your so quick reply. You mean the job result is saved under this directory ? This reminds me another question. When I use hwi, the result is saved under the conf directory by default, do you know how the change the default directory, it seems that I didn't find any parameter related to this in the config file.
2010-08-24
________________________________
shangan
________________________________
发件人: Bennie Schut
发送时间: 2010-08-24  16:44:43
收件人: '[EMAIL PROTECTED]'
抄送:
主题: RE: scrach space grows too fast
I was actually planning on filing a bug for this. I think it is supposed to remove the temp. hadoop results from this dir but it doesn't seem to do that anymore. For now I use a workaround to simply remove anything from there older then x days however that still means you have results from really large jobs laying around there for x days.
________________________________
From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 10:36 AM
To: hive-user
Subject: scrach space grows too fast

what's the parameter "hive.exec.scratchdir" for ? save the job plan ?
the space of this directory is growing too fast and I don't know why files under this directory is not removed after the job is finished. Do they have other functions ?

2010-08-24
________________________________
shangan
__________ Information from ESET NOD32 Antivirus, version of virus signature database 5391 (20100823) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB