Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> JobConf and MiniMapRedCluster


+
Jay Vyas 2013-04-17, 18:06
+
Harsh J 2013-04-18, 04:49
+
Jay Vyas 2013-04-18, 04:54
Copy link to this message
-
Re: JobConf and MiniMapRedCluster
Am not sure I totally understand yet. JobConf isn't deprecated, and is
still a (and the only) valid way to use the older mapred.* API.

If you mean we should shift the tests over to the new API
(mapreduce.*, and Job) then am all for it.

The Tool+Configured extensions are good for ToolRunner.run(…) invoked
classes, which I guess is also a good way to write a base test
invoking class, but you'd have to end up changing a lot of test
classes for this.

On Thu, Apr 18, 2013 at 10:24 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> Only one response, inline below... Certainly I will file a JIRA and some updates if this makes sense :)....
>
> Would love to bring the minimrcluster class up to date!
>
> On Apr 18, 2013, at 12:49 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Why do you imagine a test case would need the Configured and Tool
>> interfaces, which are more useful for actual client apps?
>
> Because - the JobConf is deprecated --- then shouldn't the classes which depend upon it be update to use the Configured interface?
>
>
>> Or did you mean these should support running Tool apps?
>>
>>> Any plans to update this or should file a JIRA?
>>
>> No plans as far as I'm aware; please do file a JIRA with a patch if
>> this makes sense to improve. Also do check out trunk first.
>>
>> On Wed, Apr 17, 2013 at 11:36 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>> Hi guys:
>>>
>>> the MiniMapRedCluster seems like a very useful tool which I just discovered
>>> in this blog post :
>>>
>>> http://grepalex.com/2012/10/20/hadoop-unit-testing-with-minimrcluster/
>>>
>>> But it looks like MiniMapRedCluster
>>>
>>> http://svn.apache.org/viewvc/hadoop/common/tags/release-1.0.3/src/test/org/apache/hadoop/mapred/ClusterMapReduceTestCase.java?view=co
>>>
>>> is still using JobConf instead of the Configured/Tool interface.
>>>
>>> Any plans to update this or should file a JIRA?
>>>
>>> --
>>> Jay Vyas
>>> http://jayunit100.blogspot.com
>>
>>
>>
>> --
>> Harsh J

--
Harsh J
+
Jay Vyas 2013-04-18, 05:55
+
Harsh J 2013-04-18, 06:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB