Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Question about myImageStorageFunc and PigStore


Copy link to this message
-
RE: Question about myImageStorageFunc and PigStore
Santhosh Srinivasan 2009-04-21, 18:28
Sameer,

You need to write your own UDF to read and write image files to the file
system.
Have a look at the following built-in load and store functions supported
by Pig:

PigStorage:
http://svn.apache.org/viewvc/hadoop/pig/trunk/src/org/apache/pig/builtin
/PigStorage.java?view=log
BinStorage:
http://svn.apache.org/viewvc/hadoop/pig/trunk/src/org/apache/pig/builtin
/BinStorage.java?view=log

PigStorage handles read/write of UTF-8 text data.
BinStorage handles read/write of binary data. The binary data format is
internal to Pig.

Thanks,
Santhosh

-----Original Message-----
From: Sameer Tilak [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, April 21, 2009 10:53 AM
To: [EMAIL PROTECTED]
Subject: Question about myImageStorageFunc and PigStore

Hi everyone,

We're working on an image analysis project using Pig. I wrote my UDF:
myImageFilter. However, can someone please point me to info about UDF:
myImageStorageFunc. My images will be in a directory in HDFS. So should
this
function need to inmplement reading from/ writing to image files to
HDFS? Is
there any existing functionality within Pig to do the same or do I need
to
write my own? If second, is there any example code to do this?

imagein = load '/myimages' using myImageStorageFunc();
imageop = foreach imagein generate myImageFilter(*);
store imageop into '/mythumbnails' using myImageStorageFunc();

I've similar question about PigStore function.

Regards,
--ST.