Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Displaying source log file names in pig logs

Copy link to this message
Displaying source log file names in pig logs
Guy Bayes 2010-10-21, 16:57
We have a job that processes several hundred files in a directory

We generally glob the directory in a single load statement

Sometimes the jobs chokes on a bad row in a single file

I could have sworn that pig printed the file name of the chunks it is processing in the task log but cannot see it

Does anyone know under what conditions file names are printed, or how to find the file that is causing the issues?

Romain Rigaux 2010-10-25, 16:02
Guy Bayes 2010-10-25, 16:09