Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Run Shark on Windows?

Copy link to this message
Re: Run Shark on Windows?
Good point. When I used Windows and Cygwin years ago, I sometimes had
problems with unix-like paths confusing the windows JVM. If that
happens, hopefully the fix is to use windows paths in the Hive
property files for the metastore especially if you're using Derby.
Otherwise, it might work to start shark with the windows terminal,
bypassing cygwin.


On Wed, Nov 14, 2012 at 5:47 PM, Reynold Xin <[EMAIL PROTECTED]> wrote:
> Hi,
> Reynold from the Shark team at Berkeley here.
> I haven't tried this on Windows myself, but for single node testing, if you
> have a bash shell (e.g. Cygwin), and you are able to launch Shark using the
> bash script, then it should work.
> The biggest unknown factor is whether the Hive metastore would work on
> Windows. We do rely on the Hive code path to read from the metastore.
> --
> Reynold Xin
> AMPLab, UC Berkeley
> http://www.cs.berkeley.edu/~rxin/
> On Wed, Nov 14, 2012 at 3:20 PM, Mark Grover <[EMAIL PROTECTED]>
> wrote:
>> All yours!
>> ---------- Forwarded message ----------
>> From: Dean Wampler <[EMAIL PROTECTED]>
>> Date: Wed, Nov 14, 2012 at 3:17 PM
>> Subject: Run Shark on Windows?
>> To: user <[EMAIL PROTECTED]>
>> Since we had two requests today for help on using Hive in
>> Cygwin/Windows, which doesn't work, I wonder if anyone has tried Shark
>> on Windows? Shark is the port of Hive to Spark, which supports a local
>> mode that just might work fine for testing and debugging on windows.
>> I've only tried Shark on the Mac and I used it with HDFS and my
>> pseudo-distributed mode Hadoop installation, so I have no idea if this
>> will work on Windows and work well. However, I would try it if I were
>> a Windows user ;)
>> Note that Shark is not feature complete with Hive at this time. Also,
>> it might just be easier to bite the bullet and install a virtual
>> machine image of Hadoop on Linux running in VMWare or VirtualBox,
>> anyway.
>> dean
>> --
>> Dean Wampler, Ph.D.
>> thinkbiganalytics.com
>> +1-312-339-1330

Dean Wampler, Ph.D.