Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> pipeout files

Copy link to this message
Re: pipeout files
This is likely some artifact hive leaves behind.

Our filecrush tool has piece called Clean.clean


I use it to delete anything in hdfs /tmp older then N seconds.


On Fri, Sep 7, 2012 at 11:28 AM, Sam Darwin <[EMAIL PROTECTED]> wrote:
> Hi,
> I am seeing like one million of these files on our hadoop cluster.
> 1005717 files like /tmp/hdfs/hdfs_2012082902171088341605155583849.pipeout
> 1005742 files like /tmp/hdfs/hive_job_log_hdfs_201208290217_1000376604.txt
> My questions are:
> 1.   What is a .pipeout file, and can they be deleted at any time?
> What might happen if a pipeout file is removed that shouldn't be
> removed?
> 2.   Is it entirely up the admin to log rotate these?    Why aren't
> they rotated by default when you install the packages?
> Thanks,
> Sam