MapReduce, mail # user - Any good gui tools for working with hdfs

Re: Any good gui tools for working with hdfs

Joey Echeverria 2011-09-20, 23:12

Have you looked at hue (https://github.com/cloudera/hue)? It has a
web-based GUI file manager.

-Joey

On Tue, Sep 20, 2011 at 6:50 PM, Steve Lewis <[EMAIL PROTECTED]> wrote:
> My dfs is a real mess and I am looking for a good gui fiile manager to allow
> me to clean it up
> deleting a lot of directories
> Anyone written one???
> --
> Steven M. Lewis PhD
> 4221 105th Ave NE
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Skype lordjoe_com
>
>
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434

Re: Any good gui tools for working with hdfs

Harsh J 2011-09-21, 03:25

I personally prefer Hue as well, but here are more alternatives:

- You can use the Eclipse Plugin integrations (what comes bundled, and
also Karmasphere). I believe these use the tree-browser view mode.
- Unfortunately I do not recall the name but there was this one FTP or
remote-file-browsing client software, written in Java entirely, which
catered to HDFS as one of its protocols (seems to be the only one!).
This allowed regular filezilla-style browsing.
- Mount your HDFS with fuse-dfs and you get to use your native file
browser to browse it and do actions like you would on a regular
filesystem.

(I believe there are lot of other options, these are off the top of my head.)

On Wed, Sep 21, 2011 at 4:42 AM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
> Have you looked at hue (https://github.com/cloudera/hue)? It has a
> web-based GUI file manager.
>
> -Joey
>
> On Tue, Sep 20, 2011 at 6:50 PM, Steve Lewis <[EMAIL PROTECTED]> wrote:
>> My dfs is a real mess and I am looking for a good gui fiile manager to allow
>> me to clean it up
>> deleting a lot of directories
>> Anyone written one???
>> --
>> Steven M. Lewis PhD
>> 4221 105th Ave NE
>> Kirkland, WA 98033
>> 206-384-1340 (cell)
>> Skype lordjoe_com
>>
>>
>>
>
>
>
> --
> Joseph Echeverria
> Cloudera, Inc.
> 443.305.9434
>

--
Harsh J

Re: Any good gui tools for working with hdfs

Harsh J 2011-12-21, 11:55

I finally found it, thanks to a friend. The FTP software was:
http://www.mucommander.com/ and it allows S3/HDFS file browsing like
its a piece of cake, among a lot of others. Its not bad a tool to use!
:)

On Wed, Sep 21, 2011 at 8:55 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> I personally prefer Hue as well, but here are more alternatives:
>
> - You can use the Eclipse Plugin integrations (what comes bundled, and
> also Karmasphere). I believe these use the tree-browser view mode.
> - Unfortunately I do not recall the name but there was this one FTP or
> remote-file-browsing client software, written in Java entirely, which
> catered to HDFS as one of its protocols (seems to be the only one!).
> This allowed regular filezilla-style browsing.
> - Mount your HDFS with fuse-dfs and you get to use your native file
> browser to browse it and do actions like you would on a regular
> filesystem.
>
> (I believe there are lot of other options, these are off the top of my head.)
>
> On Wed, Sep 21, 2011 at 4:42 AM, Joey Echeverria <[EMAIL PROTECTED]> wrote:
>> Have you looked at hue (https://github.com/cloudera/hue)? It has a
>> web-based GUI file manager.
>>
>> -Joey
>>
>> On Tue, Sep 20, 2011 at 6:50 PM, Steve Lewis <[EMAIL PROTECTED]> wrote:
>>> My dfs is a real mess and I am looking for a good gui fiile manager to allow
>>> me to clean it up
>>> deleting a lot of directories
>>> Anyone written one???
>>> --
>>> Steven M. Lewis PhD
>>> 4221 105th Ave NE
>>> Kirkland, WA 98033
>>> 206-384-1340 (cell)
>>> Skype lordjoe_com
>>>
>>>
>>>
>>
>>
>>
>> --
>> Joseph Echeverria
>> Cloudera, Inc.
>> 443.305.9434
>>
>
>
>
> --
> Harsh J

--
Harsh J

Re: Any good gui tools for working with hdfs

Nitin Khandelwal 2011-12-21, 12:25

Hi Harsh,

I am not able yo use MuCommander . Will you please tell me what did u enter
in port (corresponding to which property in configuration).

Thanks,
Nitin

On 21 December 2011 17:25, Harsh J <[EMAIL PROTECTED]> wrote:

> I finally found it, thanks to a friend. The FTP software was:
> http://www.mucommander.com/ and it allows S3/HDFS file browsing like
> its a piece of cake, among a lot of others. Its not bad a tool to use!
> :)
>
> On Wed, Sep 21, 2011 at 8:55 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> > I personally prefer Hue as well, but here are more alternatives:
> >
> > - You can use the Eclipse Plugin integrations (what comes bundled, and
> > also Karmasphere). I believe these use the tree-browser view mode.
> > - Unfortunately I do not recall the name but there was this one FTP or
> > remote-file-browsing client software, written in Java entirely, which
> > catered to HDFS as one of its protocols (seems to be the only one!).
> > This allowed regular filezilla-style browsing.
> > - Mount your HDFS with fuse-dfs and you get to use your native file
> > browser to browse it and do actions like you would on a regular
> > filesystem.
> >
> > (I believe there are lot of other options, these are off the top of my
> head.)
> >
> > On Wed, Sep 21, 2011 at 4:42 AM, Joey Echeverria <[EMAIL PROTECTED]>
> wrote:
> >> Have you looked at hue (https://github.com/cloudera/hue)? It has a
> >> web-based GUI file manager.
> >>
> >> -Joey
> >>
> >> On Tue, Sep 20, 2011 at 6:50 PM, Steve Lewis <[EMAIL PROTECTED]>
> wrote:
> >>> My dfs is a real mess and I am looking for a good gui fiile manager to
> allow
> >>> me to clean it up
> >>> deleting a lot of directories
> >>> Anyone written one???
> >>> --
> >>> Steven M. Lewis PhD
> >>> 4221 105th Ave NE
> >>> Kirkland, WA 98033
> >>> 206-384-1340 (cell)
> >>> Skype lordjoe_com
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Joseph Echeverria
> >> Cloudera, Inc.
> >> 443.305.9434
> >>
> >
> >
> >
> > --
> > Harsh J
>
>
>
> --
> Harsh J
>

--
Nitin Khandelwal

Re: Any good gui tools for working with hdfs

Harsh J 2011-12-21, 18:51

Yeah seems like its only working with 0.20.2 release at this point (203-205 upgraded the version, so that's the issue - compatibility broken). I repacked their jars with newer hadoop classes (DFSClient especially) and I think it works. I didn't test everything though.

Checking out their svn code and trying to build is turning out to be a nightmare (ant seems to have a load of deps, and no easy way of hadoop jar replacement?), so I'll pass on fixing their upstream stuff for a while. If anyone understands their build system better - do take a crack at it - they're completely open source :)

On 21-Dec-2011, at 11:28 PM, Radim Kolar wrote:

> i got following errror in log when using mucommander
>
> 2011-12-21 18:53:12,565 WARN org.apache.hadoop.ipc.Server: Incorrect header or version mismatch from 88.102.**.**:11072 got version 3 expected version 4
>
> what hdfs version you have?
>