what's the parameter "hive.exec.scratchdir" for ? save the job plan ?
the space of this directory is growing too fast and I don't know why files under this directory is not removed after the job is finished. Do they have other functions ?

2010-08-24

shangan

I was actually planning on filing a bug for this. I think it is supposed to remove the temp. hadoop results from this dir but it doesn't seem to do that anymore. For now I use a workaround to simply remove anything from there older then x days however that still means you have results from really large jobs laying around there for x days.
________________________________
From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 10:36 AM
To: hive-user
Subject: scrach space grows too fast

what's the parameter "hive.exec.scratchdir" for ? save the job plan ?
the space of this directory is growing too fast and I don't know why files under this directory is not removed after the job is finished. Do they have other functions ?

2010-08-24
________________________________
shangan

thank you for your so quick reply. You mean the job result is saved under this directory ? This reminds me another question. When I use hwi, the result is saved under the conf directory by default, do you know how the change the default directory, it seems that I didn't find any parameter related to this in the config file.
2010-08-24

shangan

发件人： Bennie Schut
发送时间： 2010-08-24  16:44:43
收件人： '[EMAIL PROTECTED]'
抄送：
主题： RE: scrach space grows too fast
 
I was actually planning on filing a bug for this. I think it is supposed to remove the temp. hadoop results from this dir but it doesn't seem to do that anymore. For now I use a workaround to simply remove anything from there older then x days however that still means you have results from really large jobs laying around there for x days.

From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 10:36 AM
To: hive-user
Subject: scrach space grows too fast
 
what's the parameter "hive.exec.scratchdir" for ? save the job plan ?
the space of this directory is growing too fast and I don't know why files under this directory is not removed after the job is finished. Do they have other functions ?
 
2010-08-24

shangan
__________ Information from ESET NOD32 Antivirus, version of virus signature database 5391 (20100823) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

Probably not the end result but the intermediate results are there (which are usually bigger then the end result).
I never used hwi myself.

________________________________
From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 11:00 AM
To: hive-user
Subject: Re: RE: scrach space grows too fast

thank you for your so quick reply. You mean the job result is saved under this directory ? This reminds me another question. When I use hwi, the result is saved under the conf directory by default, do you know how the change the default directory, it seems that I didn't find any parameter related to this in the config file.
2010-08-24
________________________________
shangan
________________________________
发件人： Bennie Schut
发送时间： 2010-08-24  16:44:43
收件人： '[EMAIL PROTECTED]'
抄送：
主题： RE: scrach space grows too fast
I was actually planning on filing a bug for this. I think it is supposed to remove the temp. hadoop results from this dir but it doesn't seem to do that anymore. For now I use a workaround to simply remove anything from there older then x days however that still means you have results from really large jobs laying around there for x days.
________________________________
From: shangan [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, August 24, 2010 10:36 AM
To: hive-user
Subject: scrach space grows too fast

what's the parameter "hive.exec.scratchdir" for ? save the job plan ?
the space of this directory is growing too fast and I don't know why files under this directory is not removed after the job is finished. Do they have other functions ?

2010-08-24
________________________________
shangan
__________ Information from ESET NOD32 Antivirus, version of virus signature database 5391 (20100823) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com