Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Bringing Hadoop to Fedora


Copy link to this message
-
Re: Bringing Hadoop to Fedora
Alejandro Abdelnur 2013-07-09, 00:53
suresh,

thanks for jumping in from webhdfs side.

apparently, the issues that were affecting us with jetty 6 have been address with newer jetty versions.

my take would be we move forward with the version change and we make sure webhdfs is not affected, do you have a test that exercises webhdfs under load? that would definitely help

thx
  
Alejandro
(phone typing)

On Jul 8, 2013, at 17:43, Suresh Srinivas <[EMAIL PROTECTED]> wrote:

> Isn't Jetty used by WebHDFS? Given that, Jetty performance is still
> important.
>
>
> On Mon, Jul 8, 2013 at 2:06 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:
>
>> [moving bigtop to bcc]
>>
>> Tim,
>>
>> Except from HADOOP-9680 which has significant code changes and some false
>> changes (which I did not go thru), all other changes seem OK.
>>
>> * Have you had a change to run ALL Hadoop testcases with them applied to
>> make sure there are not regression?
>>
>> * Have you look at the output of running 'mvn dependency:tree' without/with
>> the patches to verify unwanted dependencies are not sneaking in?
>>
>> On HADOOP-9650 (not included in the above list):
>>
>> Moving from Jetty6 to Jetty9 should be OK if there are not regressions. I
>> was chatting with ToddL about the issues we had before in and according to
>> Jetty guys they've been fixed in newer Jetty versions. Also, we are not
>> using Jetty for shuffle anymore (we use Netty), so Jetty is not 'stress'
>> that much anymore as it is used for the web UIs and for NN-SNN
>> checkpointing.
>>
>> From the patch some code changes are required, once that is take care we
>> should repeat the bullet item above for this patch.
>>
>> In addition, we have to see how this will play with other projects like
>> HBase that are using HttpServer from hadoop-common. I'll forward this part
>> of the email to their dev@ so they can watch/jump-in if necessary in the
>> JIRA.
>>
>> Thanks.
>>
>>
>> On Mon, Jul 8, 2013 at 11:28 AM, Tim St Clair <[EMAIL PROTECTED]> wrote:
>>
>>> Arun,
>>>
>>> I realize you're probably pretty busy (or on vacation), but I figured I
>>> would re-ping this thread to inquire about the status of the patch set
>>> listed below.?.?
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-9594
>>> https://issues.apache.org/jira/browse/HADOOP-9610
>>> https://issues.apache.org/jira/browse/HADOOP-9611
>>> https://issues.apache.org/jira/browse/HADOOP-9613
>>> https://issues.apache.org/jira/browse/HADOOP-9680 / 9623
>>>
>>> Cheers,
>>> Tim
>>>
>>> ----- Original Message -----
>>>> From: "Arun C Murthy" <[EMAIL PROTECTED]>
>>>> To: [EMAIL PROTECTED]
>>>> Cc: [EMAIL PROTECTED], [EMAIL PROTECTED],
>>> [EMAIL PROTECTED], [EMAIL PROTECTED]
>>>> Sent: Tuesday, June 11, 2013 12:54:11 PM
>>>> Subject: Re: Bringing Hadoop to Fedora
>>>>
>>>> Tim,
>>>>
>>>> This is great! I'll take a look at some of these patches, welcome!
>>>>
>>>> Arun
>>>>
>>>> On Jun 11, 2013, at 8:54 AM, Tim St Clair wrote:
>>>>
>>>>> Greetings Hadoop Development Community,
>>>>>
>>>>> As some of you are aware, we've been looking to bring the upstream
>> 2.X
>>>>> series into the Fedora channels.  We believe that there are several
>>>>> benefits that this can bring to the community:
>>>>>
>>>>> - Updated & streamlined rpm packaging
>>>>> - Deeper level of system integration
>>>>> - System managed dependencies (security + defect tracking)
>>>>> - Proving ground for OpenJDK7
>>>>> - Greater exposure for the community (both adoption and defect
>>> tracking)
>>>>> ...
>>>>>
>>>>> During our evaluation (
>> https://fedoraproject.org/wiki/Features/Hadoop
>>> ),
>>>>> we've come across some build issues as they relate to Fedora
>> packaging
>>>>> guidelines (http://fedoraproject.org/wiki/Packaging:Java).  Most of
>>> these
>>>>> issues (listed below) are very minor dependency changes, and we were
>>>>> wondering if there are folks who would be willing to review.
>>>>>
>>>>> fedora-patch-math: https