Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Re: [jira] [Resolved] (PIG-2650) Convenience mock Loader and Storer to simplify unit testing of Pig scripts


Copy link to this message
-
Re: [jira] [Resolved] (PIG-2650) Convenience mock Loader and Storer to simplify unit testing of Pig scripts
Julien Le Dem 2012-04-26, 23:34
What about setting the goal for test-commit target under a minute and test target under an hour ?
opinions ?
Julien

On Apr 26, 2012, at 3:02 PM, Alan Gates wrote:

> One other caveat I'd like to add, we should never ever check in interface changes on branches.  You could argue that falls under disruptive changes, but I think they're worth calling out.
>
> I'm definitely +1 on checking this in though.  In general I'd like to figure out how we can use mock objects to speed up our unit testing.
>
> Alan.
>
> On Apr 26, 2012, at 2:57 PM, Daniel Dai wrote:
>
>> Yes, agree. Bug fix or non-disruptive changes only.
>>
>> On Thu, Apr 26, 2012 at 2:55 PM, Thejas Nair <[EMAIL PROTECTED]> wrote:
>>> In my opinion, we should only commit changes to released branches that are
>>> either critical bug fixes, or very useful minor changes which are not likely
>>> to affect the stability of the branch.
>>>
>>> This change would fall into 2nd category.
>>>
>>> Thanks,
>>> Thejas
>>>
>>>
>>>
>>> On 4/26/12 2:32 PM, Bill Graham wrote:
>>>>
>>>> What's fair game to commit to the the 0.10 branch? Just bug fixes, or are
>>>> new small features that didn't make it into 0.10 ok?
>>>>
>>>> On Thu, Apr 26, 2012 at 2:15 PM, Daniel Dai<[EMAIL PROTECTED]>  wrote:
>>>>
>>>>> I am fine with it. Please also include the following tiny patch to fix
>>>>> hadoop 23 build after the patch.
>>>>>
>>>>> --- pig/trunk/ivy.xml (original)
>>>>> +++ pig/trunk/ivy.xml Thu Apr 26 21:11:36 2012
>>>>> @@ -178,7 +178,7 @@
>>>>>   <dependency org="net.java.dev.javacc" name="javacc"
>>>>> rev="${javacc.version}"
>>>>>     conf="compile->master"/>
>>>>>   <dependency org="junit" name="junit" rev="${junit.version}"
>>>>> -      conf="test->default"/>
>>>>> +      conf="compile->master"/>
>>>>>   <dependency org="com.google.code.p.arat" name="rat-lib"
>>>>> rev="${rats-lib.version}"
>>>>>     conf="releaseaudit->default"/>
>>>>>   <dependency org="org.codehaus.jackson" name="jackson-mapper-asl"
>>>>> rev="${jackson.version}"
>>>>>
>>>>> Daniel
>>>>>
>>>>> On Thu, Apr 26, 2012 at 2:07 PM, Julien Le Dem<[EMAIL PROTECTED]>
>>>>> wrote:
>>>>>>
>>>>>> I'm planning to commit this in 0.10 branch as well
>>>>>> The patch has only new files so it will apply cleanly.
>>>>>> Any objection?
>>>>>> Julien
>>>>>>
>>>>>>
>>>>>> On Apr 26, 2012, at 1:30 PM, Julien Le Dem (JIRA) wrote:
>>>>>>
>>>>>>>
>>>>>>>    [
>>>>>
>>>>>
>>>>> https://issues.apache.org/jira/browse/PIG-2650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>>>>>>>
>>>>>>>
>>>>>>> Julien Le Dem resolved PIG-2650.
>>>>>>> --------------------------------
>>>>>>>
>>>>>>>      Resolution: Fixed
>>>>>>>   Fix Version/s: 0.11
>>>>>>>
>>>>>>>> Convenience mock Loader and Storer to simplify unit testing of Pig
>>>>>
>>>>> scripts
>>>>>>>>
>>>>>>>>
>>>>>
>>>>> --------------------------------------------------------------------------
>>>>>>>>
>>>>>>>>
>>>>>>>>               Key: PIG-2650
>>>>>>>>               URL: https://issues.apache.org/jira/browse/PIG-2650
>>>>>>>>           Project: Pig
>>>>>>>>        Issue Type: New Feature
>>>>>>>>          Reporter: Julien Le Dem
>>>>>>>>          Assignee: Julien Le Dem
>>>>>>>>           Fix For: 0.11
>>>>>>>>
>>>>>>>>       Attachments: PIG-2650-a.patch, PIG-2650-b.patch,
>>>>>
>>>>> PIG-2650-c.patch, PIG-2650.patch
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> A test would look as follows:
>>>>>>>> {code}
>>>>>>>> PigServer pigServer = new PigServer(ExecType.LOCAL);
>>>>>>>> TupleFactory tf = TupleFactory.getInstance();
>>>>>>>> Data data = Storage.resetData(pigServer.getPigContext());
>>>>>>>> data.set("foo", Arrays.asList(
>>>>>>>>   tf.newTuple("a"),
>>>>>>>>   tf.newTuple("b"),
>>>>>>>>   tf.newTuple("c")
>>>>>>>>   ));
>>>>>>>> pigServer.registerQuery("A = LOAD 'foo' USING mock.Storage();");
>>>>>>>> // some complex script to test
>>>>>>>> pigServer.registerQuery("STORE A INTO 'bar' USING mock.Storage();");
>>>>>>>> Iterator<Tuple>  out = data.get("bar").iterator();