Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Fully distribute TextInputFormat...


Copy link to this message
-
Re: Fully distribute TextInputFormat...
Jeff Zhang 2010-05-10, 12:52
What's the format of this file ? gzip can been split.

On Mon, May 10, 2010 at 5:21 AM, Pierre ANCELOT <[EMAIL PROTECTED]> wrote:
> Hi folks :)
> I have one big file... I read it with FileInputFormat, this generates only
> one task and of course, this doesn't get distributed across the cluster
> nodes.
> Should I use an other Input class or do I have a bug in my implementation?
>
> The desired behavior is one task per line.
>
> Thanks.
>
>
>
> --
> http://www.neko-consulting.com
> Ego sum quis ego servo
> "Je suis ce que je protège"
> "I am what I protect"
>

--
Best Regards

Jeff Zhang