Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> streaming cacheArchive shared libraries


Copy link to this message
-
Re: streaming cacheArchive shared libraries
Right, so it was pushed down a level into the "testlink" directory.  That's why my shared libraries were not linking properly to my mapper executable.  I can fix that by using "cmddev" to redirect LD_LIBRARY_PATH.  I think that'll work.

On Aug 5, 2011, at 10:44 , Ramya Sunil wrote:

> Hi Keith,
>
> I have tried the exact use case you have mentioned and it works fine for me.
> Below is the command line for the same:
>
> [ramya]$ jar vxf samplelib.jar
> created: META-INF/
> inflated: META-INF/MANIFEST.MF
> inflated: libhdfs.so
>
> [ramya]$ hadoop dfs -put samplelib.jar samplelib.jar
>
> [ramya]$ hadoop jar hadoop-streaming.jar -input InputDir -mapper "ls
> testlink/libhdfs.so" -reducer NONE -output out -cacheArchive
> hdfs://<namenode>:<port>/user/ramya/samplelib.jar#testlink
>
> [ramya]$ hadoop dfs -cat out/*
> testlink/libhdfs.so
> testlink/libhdfs.so
> testlink/libhdfs.so
>
>
> Hope it helps.
>
> Thanks
> Ramya
________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"I used to be with it, but then they changed what it was.  Now, what I'm with
isn't it, and what's it seems weird and scary to me."
                                           --  Abe (Grandpa) Simpson
________________________________________________________________________________
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB