Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Question about Pig BinaryStorage()


Copy link to this message
-
RE: Question about Pig BinaryStorage()
Drop the -x local.

java -cp pig.jar:/home/unwin/hadoop-0.19.1/conf org.apache.pig.Main
myScript5

-----Original Message-----
From: Roger Unwin [mailto:[EMAIL PROTECTED]]
Sent: Thursday, April 23, 2009 2:30 PM
To: Santhosh Srinivasan
Cc: [EMAIL PROTECTED]
Subject: Question about Pig BinaryStorage()

Santhosh,

I am trying to iterate through a group of binary files.  I would like  
the reduce job to get 1 binary file each.  Below is the first part of  
it, trying to read the data in.

I have the following script:

images = load 'images' using BinaryStorage() split by 'file';

dump images;

Here is my invocation:
java -cp pig.jar:/home/unwin/hadoop-0.19.1/conf org.apache.pig.Main -x  
local myScript5
2009-04-23 14:22:38,669 [main] ERROR  
org
.apache
.pig
.backend
.hadoop.executionengine.physicalLayer.relationalOperators.POStore -  
Received error from storer function:  
org.apache.pig.backend.executionengine.ExecException: ERROR 2081:  
Unable to setup the load function.
2009-04-23 14:22:38,673 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - Failed  
jobs!!
2009-04-23 14:22:38,674 [main] INFO  
org.apache.pig.backend.local.executionengine.LocalPigLauncher - 1 out  
of 1 failed!
2009-04-23 14:22:38,678 [main] ERROR org.apache.pig.tools.grunt.Grunt  
- ERROR 1066: Unable to open iterator for alias images

Here is where the files are in hadoop:
unwin@hadoop-n:~/pig-0.2.0$ ../hadoop-0.18.3/bin/hadoop dfs -ls 'images'
Found 10 items
-rw-r--r--   2 unwin supergroup     272449 2009-04-22 11:04 /user/
unwin/images/IMG_0010.JPG
-rw-r--r--   2 unwin supergroup     267580 2009-04-22 11:04 /user/
unwin/images/IMG_0011.JPG
-rw-r--r--   2 unwin supergroup     378000 2009-04-22 11:04 /user/
unwin/images/IMG_0012.JPG
-rw-r--r--   2 unwin supergroup     327829 2009-04-22 11:04 /user/
unwin/images/IMG_0013.JPG
-rw-r--r--   2 unwin supergroup     476088 2009-04-22 11:04 /user/
unwin/images/IMG_0014.JPG
-rw-r--r--   2 unwin supergroup     357258 2009-04-22 11:04 /user/
unwin/images/IMG_0015.JPG
-rw-r--r--   2 unwin supergroup     401496 2009-04-22 11:04 /user/
unwin/images/IMG_0016.JPG
-rw-r--r--   2 unwin supergroup     377798 2009-04-22 11:04 /user/
unwin/images/IMG_0017.JPG
-rw-r--r--   2 unwin supergroup     466437 2009-04-22 11:04 /user/
unwin/images/IMG_0018.JPG
-rw-r--r--   2 unwin supergroup     351952 2009-04-22 11:04 /user/
unwin/images/IMG_0019.JPG

Do you see anything obvious?, or a better way of iterating?

Thanks,

Roger
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB