Hadoop, mail # user - Re: Accessing HDFS files from an servlet

Accessing HDFS files from an servlet

sushil sontakke 2012-04-13, 07:46

I want to know if there is any way of reading a file from HDFS using a
servlet . Suppose I have filename of a valid file situated over HDFS . How
do I generate a URL to display that file on a jsp page using some servlet
code .

Thank You .

Re: Accessing HDFS files from an servlet

Jessica Seastrom 2012-04-13, 11:40

Have you considered using Hoop?
http://www.cloudera.com/blog/2011/07/hoop-hadoop-hdfs-over-http/

On Fri, Apr 13, 2012 at 3:46 AM, sushil sontakke <[EMAIL PROTECTED]>wrote:

> I want to know if there is any way of reading a file from HDFS using a
> servlet . Suppose I have filename of a valid file situated over HDFS . How
> do I generate a URL to display that file on a jsp page using some servlet
> code .
>
> Thank You .
>

--

<http://www.cloudera.com>
Jessica Seastrom
Solutions Architect
Email: [EMAIL PROTECTED]
Mobile: 443.622.6707

Re: Accessing HDFS files from an servlet

Edward Capriolo 2012-04-13, 13:14

http://www.edwardcapriolo.com/wiki/en/Tomcat_Hadoop

Have all the hadoop jars and conf files in your classpath
--or-- construct your own conf and URI programatically

URI i = URI.create("hdfs://192.168.220.200:54310");
FileSystem fs = FileSystem.get(i,conf);

On Fri, Apr 13, 2012 at 7:40 AM, Jessica Seastrom <[EMAIL PROTECTED]>wrote:

> Have you considered using Hoop?
> http://www.cloudera.com/blog/2011/07/hoop-hadoop-hdfs-over-http/
>
> On Fri, Apr 13, 2012 at 3:46 AM, sushil sontakke <[EMAIL PROTECTED]>wrote:
>
>> I want to know if there is any way of reading a file from HDFS using a
>> servlet . Suppose I have filename of a valid file situated over HDFS . How
>> do I generate a URL to display that file on a jsp page using some servlet
>> code .
>>
>> Thank You .
>>
>
>
>
> --
>
> <http://www.cloudera.com>
> Jessica Seastrom
> Solutions Architect
> Email: [EMAIL PROTECTED]
> Mobile: 443.622.6707
>
>
>

Re: Accessing HDFS files from an servlet

sushil sontakke 2012-04-15, 09:12

i am getting this error on running that jsp as servlet
Apr 15, 2012 2:34:07 PM org.apache.catalina.core.AprLifecycleListener init
INFO: Loaded APR based Apache Tomcat Native library 1.1.22.
Apr 15, 2012 2:34:07 PM org.apache.catalina.core.AprLifecycleListener init
INFO: APR capabilities: IPv6 [false], sendfile [true], accept filters
[false], random [true].
Apr 15, 2012 2:34:07 PM org.apache.tomcat.util.digester.SetPropertiesRule
begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting
property 'source' to 'org.eclipse.jst.j2ee.server:UserInterface' did not
find a matching property.
Apr 15, 2012 2:34:08 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-apr-8080"]
Apr 15, 2012 2:34:08 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-apr-8009"]
Apr 15, 2012 2:34:08 PM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1985 ms
Apr 15, 2012 2:34:08 PM org.apache.catalina.core.StandardService
startInternal
INFO: Starting service Catalina
Apr 15, 2012 2:34:08 PM org.apache.catalina.core.StandardEngine
startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.26
Apr 15, 2012 2:34:09 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-apr-8080"]
Apr 15, 2012 2:34:09 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-apr-8009"]
Apr 15, 2012 2:34:09 PM org.apache.catalina.startup.Catalina start
INFO: Server startup in 846 ms
Apr 15, 2012 2:34:10 PM org.apache.catalina.core.ApplicationContext log
INFO: Marking servlet DisplayFile as unavailable
Apr 15, 2012 2:34:10 PM org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Allocate exception for servlet DisplayFile
java.lang.ClassNotFoundException: org.apache.hadoop.fs.FSDataInputStream
   at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1701)
   at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1546)
   at java.lang.Class.getDeclaredConstructors0(Native Method)
   at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)
   at java.lang.Class.getConstructor0(Unknown Source)
   at java.lang.Class.newInstance0(Unknown Source)
   at java.lang.Class.newInstance(Unknown Source)
   at
org.apache.catalina.core.DefaultInstanceManager.newInstance(DefaultInstanceManager.java:125)
   at
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1136)
   at
org.apache.catalina.core.StandardWrapper.allocate(StandardWrapper.java:857)
   at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:135)
   at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
   at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
   at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
   at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
   at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
   at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
   at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
   at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
   at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
   at
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1805)
   at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
   at java.lang.Thread.run(Unknown Source)