Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> LZO & Pig (Elephantbird?)


Copy link to this message
-
LZO & Pig (Elephantbird?)
Hello list,

I've installed the LZO codecs (https://github.com/kevinweil/hadoop-lzo) and
now I'm looking into using LZO in Pig. Elephant Bird
(https://github.com/kevinweil/elephant-bird) seems to provide some nice
prefab loaders, but it's requirements do not fit out Hadoop installation
(we're on CDH3b2 with Pig 0.7, EB cannot be used with anything > 0.6). Also
the need for Thrift 0.2 is unclear to me - Thrift is now at 0.5.

Now I did find this project, http://code.google.com/p/hadoop-gpl-packing/,
saying EB can handle even Pig 0.8. This confuses me - can I or can I not use
Elephant Bird with Pig 0.7, or even upgrade to Pig 0.8?

Since EB is probably not an option, does anybody have some pointers on how
to use LZO'ed files with Pig?

Thanks!

Evert Lammerts
+
Tyler Coffin 2011-01-12, 20:23
+
Dmitriy Ryaboy 2011-01-12, 22:44
+
Dmitriy Ryaboy 2011-01-12, 22:51
+
Evert Lammerts 2011-01-13, 15:43
+
Dmitriy Ryaboy 2011-01-13, 19:26
+
Dmitriy Lyubimov 2011-01-20, 20:16
+
Dmitriy Ryaboy 2011-01-20, 20:41
+
Dmitriy Lyubimov 2011-01-20, 20:44
+
Evert Lammerts 2011-01-13, 20:18
+
Gerrit van Vuuren 2011-01-17, 12:04
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB