Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # dev - JobConf and MiniMapRedCluster


+
Jay Vyas 2013-04-17, 18:06
+
Harsh J 2013-04-18, 04:49
+
Jay Vyas 2013-04-18, 04:54
+
Harsh J 2013-04-18, 05:16
+
Jay Vyas 2013-04-18, 05:55
Copy link to this message
-
Re: JobConf and MiniMapRedCluster
Harsh J 2013-04-18, 06:11
Ah yes, we ended up un-deprecating the old API and are currently
supporting both across all releases. 1.x releases too got a few
patches un-deprecating the API classes.

On Thu, Apr 18, 2013 at 11:25 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> Okay, thanks.... Ill look into this JIRA .
>
> it is clear from some light googling that, at some point
>
> ** some version of JobConf was deprecated, and that maybe again it was undeprecated or maybe moved....**
>
> Will have to look into this more formally to really determine what's going on.
>
>
>
> On Apr 18, 2013, at 1:16 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Am not sure I totally understand yet. JobConf isn't deprecated, and is
>> still a (and the only) valid way to use the older mapred.* API.
>>
>> If you mean we should shift the tests over to the new API
>> (mapreduce.*, and Job) then am all for it.
>>
>> The Tool+Configured extensions are good for ToolRunner.run(…) invoked
>> classes, which I guess is also a good way to write a base test
>> invoking class, but you'd have to end up changing a lot of test
>> classes for this.
>>
>> On Thu, Apr 18, 2013 at 10:24 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>> Only one response, inline below... Certainly I will file a JIRA and some updates if this makes sense :)....
>>>
>>> Would love to bring the minimrcluster class up to date!
>>>
>>> On Apr 18, 2013, at 12:49 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>>
>>>> Why do you imagine a test case would need the Configured and Tool
>>>> interfaces, which are more useful for actual client apps?
>>>
>>> Because - the JobConf is deprecated --- then shouldn't the classes which depend upon it be update to use the Configured interface?
>>>
>>>
>>>> Or did you mean these should support running Tool apps?
>>>>
>>>>> Any plans to update this or should file a JIRA?
>>>>
>>>> No plans as far as I'm aware; please do file a JIRA with a patch if
>>>> this makes sense to improve. Also do check out trunk first.
>>>>
>>>> On Wed, Apr 17, 2013 at 11:36 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>>>> Hi guys:
>>>>>
>>>>> the MiniMapRedCluster seems like a very useful tool which I just discovered
>>>>> in this blog post :
>>>>>
>>>>> http://grepalex.com/2012/10/20/hadoop-unit-testing-with-minimrcluster/
>>>>>
>>>>> But it looks like MiniMapRedCluster
>>>>>
>>>>> http://svn.apache.org/viewvc/hadoop/common/tags/release-1.0.3/src/test/org/apache/hadoop/mapred/ClusterMapReduceTestCase.java?view=co
>>>>>
>>>>> is still using JobConf instead of the Configured/Tool interface.
>>>>>
>>>>> Any plans to update this or should file a JIRA?
>>>>>
>>>>> --
>>>>> Jay Vyas
>>>>> http://jayunit100.blogspot.com
>>>>
>>>>
>>>>
>>>> --
>>>> Harsh J
>>
>>
>>
>> --
>> Harsh J

--
Harsh J