Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Log files occupy lot of Disk size


Copy link to this message
-
RE: Log files occupy lot of Disk size
Kartashov, Andy 2012-11-23, 21:44
Every so often I simply wipe out the whole directory content:

/var/log/hadoop-hdfs/*
/var/log/hadoop-0.20-mapreduce/*

I also clean /home/mapred/local/* and /tmp
-----Original Message-----
From: Harsh J [mailto:[EMAIL PROTECTED]]
Sent: Friday, November 23, 2012 2:43 AM
To: <[EMAIL PROTECTED]>
Subject: Re: Log files occupy lot of Disk size

Lower your log levels if you do not need all that verbosity. You can control log retention, max sizes to keep, max number of files to keep, and logging levels, etc. via each components' log4j.properties file.

On Fri, Nov 23, 2012 at 12:42 PM, iwannaplay games <[EMAIL PROTECTED]> wrote:
> If i delete the log file without stopping the cluster  won't it
> terminate the session.
>
>
>
> On 11/23/12, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
>> Hi there,
>>
>>     You can write a small job or some script which periodically
>> checks for the log growth and performs the delete after certain threshold.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Fri, Nov 23, 2012 at 12:28 PM, iwannaplay games <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> Everytime i query hbase or hive ,there is a significant growth in my
>>> log files and it consumes lot of space from my hard disk....(Approx
>>> 40
>>> gb)
>>> So i stop the cluster ,delete all the logs and free the space and
>>> then again start the cluster to start my work.
>>>
>>> Is there any other solution coz i cannot restart the cluster everyday.
>>>
>>

--
Harsh J
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le présent courriel et toute pièce jointe qui l'accompagne sont confidentiels, protégés par le droit d'auteur et peuvent être couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autorisée est interdite. Si vous n'êtes pas le destinataire prévu de ce courriel, supprimez-le et contactez immédiatement l'expéditeur. Veuillez penser à l'environnement avant d'imprimer le présent courriel