Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Question about Pig BinaryStorage()


Copy link to this message
-
RE: Question about Pig BinaryStorage()
Drop the -x local.

java -cp pig.jar:/home/unwin/hadoop-0.19.1/conf org.apache.pig.Main
myScript5

-----Original Message-----
From: Roger Unwin [mailto:[EMAIL PROTECTED]]
Sent: Thursday, April 23, 2009 2:30 PM
To: Santhosh Srinivasan
Cc: [EMAIL PROTECTED]
Subject: Question about Pig BinaryStorage()

Santhosh,

I am trying to iterate through a group of binary files.  I would like  
the reduce job to get 1 binary file each.  Below is the first part of  
it, trying to read the data in.

I have the following script:

images = load 'images' using BinaryStorage() split by 'file';

dump images;

Here is my invocation:
java -cp pig.jar:/home/unwin/hadoop-0.19.1/conf org.apache.pig.Main -x  
local myScript5
2009-04-23 14:22:38,669 [main] ERROR  
org
.apache
.pig
.backend
.hadoop.executionengine.physicalLayer.relationalOperators.POStore -  
Received error from storer function:  
org.apache.pig.backend.executionengine.ExecException: ERROR 2081:  
Unable to setup the load function.
2009-04-23 14:22:38,673 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - Failed  
jobs!!
2009-04-23 14:22:38,674 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - 1 out  
of 1 failed!
2009-04-23 14:22:38,678 [main] ERROR org.apache.pig.tools.grunt.Grunt  
- ERROR 1066: Unable to open iterator for alias images

Here is where the files are in hadoop:
unwin@hadoop-n:~/pig-0.2.0$ ../hadoop-0.18.3/bin/hadoop dfs -ls 'images'
Found 10 items
-rw-r--r--   2 unwin supergroup     272449 2009-04-22 11:04 /user/
unwin/images/IMG_0010.JPG
-rw-r--r--   2 unwin supergroup     267580 2009-04-22 11:04 /user/
unwin/images/IMG_0011.JPG
-rw-r--r--   2 unwin supergroup     378000 2009-04-22 11:04 /user/
unwin/images/IMG_0012.JPG
-rw-r--r--   2 unwin supergroup     327829 2009-04-22 11:04 /user/
unwin/images/IMG_0013.JPG
-rw-r--r--   2 unwin supergroup     476088 2009-04-22 11:04 /user/
unwin/images/IMG_0014.JPG
-rw-r--r--   2 unwin supergroup     357258 2009-04-22 11:04 /user/
unwin/images/IMG_0015.JPG
-rw-r--r--   2 unwin supergroup     401496 2009-04-22 11:04 /user/
unwin/images/IMG_0016.JPG
-rw-r--r--   2 unwin supergroup     377798 2009-04-22 11:04 /user/
unwin/images/IMG_0017.JPG
-rw-r--r--   2 unwin supergroup     466437 2009-04-22 11:04 /user/
unwin/images/IMG_0018.JPG
-rw-r--r--   2 unwin supergroup     351952 2009-04-22 11:04 /user/
unwin/images/IMG_0019.JPG

Do you see anything obvious?, or a better way of iterating?

Thanks,

Roger