Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> distributed cache in pig


Copy link to this message
-
RE: distributed cache in pig
This is because Hadoop 20 does not support distributed cache in local
mode. My understanding is that it would be part of Hadoop 22.

Olga

-----Original Message-----
From: Gang Luo [mailto:[EMAIL PROTECTED]]
Sent: Monday, June 07, 2010 3:40 PM
To: [EMAIL PROTECTED]
Subject: distributed cache in pig

HI all,
I notice that whether pig use distributed cache depends on the context
(local or mapreduce). When running in mapreduce mode, the distributed
cache is always enable (e.g. replicated join). However, I never find
such method, DistributedCache.getLocalCacheFiles(job), which get the
cached file from the local disk. So, how does pig read these files from
local disk? I am looking at the pig 0.7 source code.

Thanks,
-Gang

      
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB