Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - scrach space grows too fast


Copy link to this message
-
RE: RE: scrach space grows too fast
Bennie Schut 2010-08-24, 09:20
Probably not the end result but the intermediate results are there (which are usually bigger then the end result).
I never used hwi myself.

________________________________
From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 11:00 AM
To: hive-user
Subject: Re: RE: scrach space grows too fast

thank you for your so quick reply. You mean the job result is saved under this directory ? This reminds me another question. When I use hwi, the result is saved under the conf directory by default, do you know how the change the default directory, it seems that I didn't find any parameter related to this in the config file.
2010-08-24
________________________________
shangan
________________________________
发件人: Bennie Schut
发送时间: 2010-08-24  16:44:43
收件人: '[EMAIL PROTECTED]'
抄送:
主题: RE: scrach space grows too fast
I was actually planning on filing a bug for this. I think it is supposed to remove the temp. hadoop results from this dir but it doesn't seem to do that anymore. For now I use a workaround to simply remove anything from there older then x days however that still means you have results from really large jobs laying around there for x days.
________________________________
From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 10:36 AM
To: hive-user
Subject: scrach space grows too fast

what's the parameter "hive.exec.scratchdir" for ? save the job plan ?
the space of this directory is growing too fast and I don't know why files under this directory is not removed after the job is finished. Do they have other functions ?

2010-08-24
________________________________
shangan
__________ Information from ESET NOD32 Antivirus, version of virus signature database 5391 (20100823) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com