Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # dev - JobConf and MiniMapRedCluster


Copy link to this message
-
Re: JobConf and MiniMapRedCluster
Harsh J 2013-04-18, 05:16
Am not sure I totally understand yet. JobConf isn't deprecated, and is
still a (and the only) valid way to use the older mapred.* API.

If you mean we should shift the tests over to the new API
(mapreduce.*, and Job) then am all for it.

The Tool+Configured extensions are good for ToolRunner.run(…) invoked
classes, which I guess is also a good way to write a base test
invoking class, but you'd have to end up changing a lot of test
classes for this.

On Thu, Apr 18, 2013 at 10:24 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> Only one response, inline below... Certainly I will file a JIRA and some updates if this makes sense :)....
>
> Would love to bring the minimrcluster class up to date!
>
> On Apr 18, 2013, at 12:49 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Why do you imagine a test case would need the Configured and Tool
>> interfaces, which are more useful for actual client apps?
>
> Because - the JobConf is deprecated --- then shouldn't the classes which depend upon it be update to use the Configured interface?
>
>
>> Or did you mean these should support running Tool apps?
>>
>>> Any plans to update this or should file a JIRA?
>>
>> No plans as far as I'm aware; please do file a JIRA with a patch if
>> this makes sense to improve. Also do check out trunk first.
>>
>> On Wed, Apr 17, 2013 at 11:36 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>> Hi guys:
>>>
>>> the MiniMapRedCluster seems like a very useful tool which I just discovered
>>> in this blog post :
>>>
>>> http://grepalex.com/2012/10/20/hadoop-unit-testing-with-minimrcluster/
>>>
>>> But it looks like MiniMapRedCluster
>>>
>>> http://svn.apache.org/viewvc/hadoop/common/tags/release-1.0.3/src/test/org/apache/hadoop/mapred/ClusterMapReduceTestCase.java?view=co
>>>
>>> is still using JobConf instead of the Configured/Tool interface.
>>>
>>> Any plans to update this or should file a JIRA?
>>>
>>> --
>>> Jay Vyas
>>> http://jayunit100.blogspot.com
>>
>>
>>
>> --
>> Harsh J

--
Harsh J