Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> MapReduce on Local files


Copy link to this message
-
Re: MapReduce on Local files
Not quite sure if I got your question. These tidbits may help though,
from what I can understand:

* LocalFileSystem's listing uses Java's APIs for file/dir listing, and
has no concept of what a hidden file is on its own. It retrieves the
whole list.
* MR's FileInputFormat (and normal derivatives) does filter away "."
and "_" starting path names, from added input paths to the job.

On Wed, Apr 3, 2013 at 3:09 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> ---------- Forwarded message ----------
> From: Mohammad Tariq <[EMAIL PROTECTED]>
> Date: Tue, Apr 2, 2013 at 5:16 PM
> Subject: MapReduce on Local files
> To: [EMAIL PROTECTED]
>
>
> Hello list,
>
>            Is a MR job capable of reading even the hidden temp files present
> inside a directory located on my local FS?I have noticed this thing today
> for the first time because till now I never tried running MR jobs on local
> files.
>
> Thank you so much for your time?
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB