Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - streaming cacheArchive shared libraries


Copy link to this message
-
Re: streaming cacheArchive shared libraries
Keith Wiley 2011-08-05, 17:49
Right, so it was pushed down a level into the "testlink" directory.  That's why my shared libraries were not linking properly to my mapper executable.  I can fix that by using "cmddev" to redirect LD_LIBRARY_PATH.  I think that'll work.

On Aug 5, 2011, at 10:44 , Ramya Sunil wrote:

> Hi Keith,
>
> I have tried the exact use case you have mentioned and it works fine for me.
> Below is the command line for the same:
>
> [ramya]$ jar vxf samplelib.jar
> created: META-INF/
> inflated: META-INF/MANIFEST.MF
> inflated: libhdfs.so
>
> [ramya]$ hadoop dfs -put samplelib.jar samplelib.jar
>
> [ramya]$ hadoop jar hadoop-streaming.jar -input InputDir -mapper "ls
> testlink/libhdfs.so" -reducer NONE -output out -cacheArchive
> hdfs://<namenode>:<port>/user/ramya/samplelib.jar#testlink
>
> [ramya]$ hadoop dfs -cat out/*
> testlink/libhdfs.so
> testlink/libhdfs.so
> testlink/libhdfs.so
>
>
> Hope it helps.
>
> Thanks
> Ramya
________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"I used to be with it, but then they changed what it was.  Now, what I'm with
isn't it, and what's it seems weird and scary to me."
                                           --  Abe (Grandpa) Simpson
________________________________________________________________________________