Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # dev - JobConf and MiniMapRedCluster


+
Jay Vyas 2013-04-17, 18:06
+
Harsh J 2013-04-18, 04:49
+
Jay Vyas 2013-04-18, 04:54
+
Harsh J 2013-04-18, 05:16
Copy link to this message
-
Re: JobConf and MiniMapRedCluster
Jay Vyas 2013-04-18, 05:55
Okay, thanks.... Ill look into this JIRA .

it is clear from some light googling that, at some point

** some version of JobConf was deprecated, and that maybe again it was undeprecated or maybe moved....**
 
Will have to look into this more formally to really determine what's going on.

On Apr 18, 2013, at 1:16 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Am not sure I totally understand yet. JobConf isn't deprecated, and is
> still a (and the only) valid way to use the older mapred.* API.
>
> If you mean we should shift the tests over to the new API
> (mapreduce.*, and Job) then am all for it.
>
> The Tool+Configured extensions are good for ToolRunner.run(…) invoked
> classes, which I guess is also a good way to write a base test
> invoking class, but you'd have to end up changing a lot of test
> classes for this.
>
> On Thu, Apr 18, 2013 at 10:24 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>> Only one response, inline below... Certainly I will file a JIRA and some updates if this makes sense :)....
>>
>> Would love to bring the minimrcluster class up to date!
>>
>> On Apr 18, 2013, at 12:49 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>>> Why do you imagine a test case would need the Configured and Tool
>>> interfaces, which are more useful for actual client apps?
>>
>> Because - the JobConf is deprecated --- then shouldn't the classes which depend upon it be update to use the Configured interface?
>>
>>
>>> Or did you mean these should support running Tool apps?
>>>
>>>> Any plans to update this or should file a JIRA?
>>>
>>> No plans as far as I'm aware; please do file a JIRA with a patch if
>>> this makes sense to improve. Also do check out trunk first.
>>>
>>> On Wed, Apr 17, 2013 at 11:36 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>>> Hi guys:
>>>>
>>>> the MiniMapRedCluster seems like a very useful tool which I just discovered
>>>> in this blog post :
>>>>
>>>> http://grepalex.com/2012/10/20/hadoop-unit-testing-with-minimrcluster/
>>>>
>>>> But it looks like MiniMapRedCluster
>>>>
>>>> http://svn.apache.org/viewvc/hadoop/common/tags/release-1.0.3/src/test/org/apache/hadoop/mapred/ClusterMapReduceTestCase.java?view=co
>>>>
>>>> is still using JobConf instead of the Configured/Tool interface.
>>>>
>>>> Any plans to update this or should file a JIRA?
>>>>
>>>> --
>>>> Jay Vyas
>>>> http://jayunit100.blogspot.com
>>>
>>>
>>>
>>> --
>>> Harsh J
>
>
>
> --
> Harsh J
+
Harsh J 2013-04-18, 06:11