Sam Darwin 2012-09-07, 15:28
This is likely some artifact hive leaves behind.
Our filecrush tool has piece called Clean.clean
I use it to delete anything in hdfs /tmp older then N seconds.
On Fri, Sep 7, 2012 at 11:28 AM, Sam Darwin <[EMAIL PROTECTED]> wrote:
> I am seeing like one million of these files on our hadoop cluster.
> 1005717 files like /tmp/hdfs/hdfs_2012082902171088341605155583849.pipeout
> 1005742 files like /tmp/hdfs/hive_job_log_hdfs_201208290217_1000376604.txt
> My questions are:
> 1. What is a .pipeout file, and can they be deleted at any time?
> What might happen if a pipeout file is removed that shouldn't be
> 2. Is it entirely up the admin to log rotate these? Why aren't
> they rotated by default when you install the packages?