-Re: how to integrate snappy into hadoop
Prashant Kommireddi 2011-12-08, 03:29
I have not tried it with HBase, and yes 0.20.2 is not compatible with it.
What is the error you receive when you try compiling Snappy? I don't think
compiling Snappy would be dependent on HBase.
2011/12/7 Jinyan Xu <[EMAIL PROTECTED]>
> Hi Prashant Kommireddi,
> Last week, I read build-hadoop-from-source and follow it, but I failed to
> compile hbase with "mvn compile -Dsnappy ". Did you install HBase 0.90.2?
> According build-hadoop-from-source HBase 0.90.2 is incompatible with
> -----Original Message-----
> From: Prashant Kommireddi [mailto:[EMAIL PROTECTED]]
> Sent: 2011年12月8日 10:13
> To: [EMAIL PROTECTED]
> Subject: Re: how to integrate snappy into hadoop
> I had to struggle a bit while building Snappy for Hadoop 0.20.2 on Ubuntu.
> However, I have now been able to install it on a 10 node cluster and it
> works great for map output compression. Please check these notes, may be it
> might help in addition to official Hadoop-Snappy notes.
> Tom White (amongst a few others) is the best person to answer this since he
> is working directly with this integration project.
> *Installing Maven 3*
> *Snappy build*
> *Creating the symbolic link for BUILD to pass:* THIS IS IMPORTANT
> sudo ln -s
> *Additional build notes:*
> Install commands I issued on localhost, these were notes for myself. There
> are dependencies that Snappy build needs, and you might NOT need all of the
> commands that I have issues below. Please refer to official notes and
> install accordingly.
> pkommireddi@pkommireddi-wsl:~$ history | grep install
> 1678 sudo apt-get install python-software-properties
> 1681 sudo apt-get install maven
> 1750 make install
> 1751 ./configure && make && sudo make install
> 1752 sudo apt-get install zlibc zlib1g zlib1g-dev
> 1763 sudo apt-get install subversion
> 1776 ./configure && make && sudo make install
> 1824 sudo apt-get install zlibc zlib1g zlib1g-dev
> 1846 make install
> 1847 sudo make install
> 1858 make installcheck
> 1861 sudo make install
> 1862 sudo make installcheck
> 1904 sudo apt-get install libtool
> 1907 sudo apt-get install automake
> Prashant Kommireddi
> On Wed, Dec 7, 2011 at 5:39 PM, Jinyan Xu <[EMAIL PROTECTED]> wrote:
> > Hi ,
> > Anyone else have the experience integrating snappy into hadoop ? help me
> > with it....
> > I find google doesn't provide the hadoop-snappy now :
> > "Hadoop-snappy is integrated into Hadoop Common(JUN 2011).
> > Hadoop-Snappy can be used as an add-on for recent (released) versions of
> > Hadoop that do not provide Snappy Codec support yet.
> > Hadoop-Snappy is being kept in synch with Hadoop Common. "
> > Thanks!
> > ________________________________
> > The information and any attached documents contained in this message
> > may be confidential and/or legally privileged. The message is
> > intended solely for the addressee(s). If you are not the intended
> > recipient, you are hereby notified that any use, dissemination, or
> > reproduction is strictly prohibited and may be unlawful. If you are
> > not the intended recipient, please contact the sender immediately by
> > return e-mail and destroy all copies of the original message.
> The information and any attached documents contained in this message
> may be confidential and/or legally privileged. The message is
> intended solely for the addressee(s). If you are not the intended
> recipient, you are hereby notified that any use, dissemination, or
> reproduction is strictly prohibited and may be unlawful. If you are
> not the intended recipient, please contact the sender immediately by
> return e-mail and destroy all copies of the original message.