test stubs should fail?

[Jeff Brown-4]

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Burt Beckwith-2]

+1

Burt

On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[Colin Harrington]

(+1)

If we are going to generate them then I would want them to fail and have action required to write a real test or to remove the empty test.

Colin Harrington
[hidden email]

On Mon, Jun 6, 2011 at 9:39 PM, Burt Beckwith <[hidden email]> wrote:

+1

Burt

On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[basejump (Josh)]

+1

We always seem to end up with at least a few tests in our projects that do nothing because they were auto generated and do nothing but pass.

Easy to fix in our company templates but think it should be part of the default grails templates

On Jun 6, 2011, at 9:43 PM, Colin Harrington wrote:

(+1)

If we are going to generate them then I would want them to fail and have action required to write a real test or to remove the empty test.

Colin Harrington
[hidden email]

On Mon, Jun 6, 2011 at 9:39 PM, Burt Beckwith <[hidden email]> wrote:

+1

Burt

On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[pmcneil]

+1

On 07/06/11 12:51, basejump (Josh) wrote:

+1

We always seem to end up with at least a few tests in our projects that do nothing because they were auto generated and do nothing but pass.

Easy to fix in our company templates but think it should be part of the default grails templates

On Jun 6, 2011, at 9:43 PM, Colin Harrington wrote:

(+1)

If we are going to generate them then I would want them to fail and have action required to write a real test or to remove the empty test.

Colin Harrington
[hidden email]

On Mon, Jun 6, 2011 at 9:39 PM, Burt Beckwith <[hidden email]> wrote:

+1

Burt

On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[Gert Wohlgemuth]

i agree completely with this.

On Mon, Jun 6, 2011 at 9:07 PM, Peter McNeil <[hidden email]> wrote:

+1

On 07/06/11 12:51, basejump (Josh) wrote:

+1

We always seem to end up with at least a few tests in our projects that do nothing because they were auto generated and do nothing but pass.

Easy to fix in our company templates but think it should be part of the default grails templates

On Jun 6, 2011, at 9:43 PM, Colin Harrington wrote:

(+1)

If we are going to generate them then I would want them to fail and have action required to write a real test or to remove the empty test.

Colin Harrington
[hidden email]

On Mon, Jun 6, 2011 at 9:39 PM, Burt Beckwith <[hidden email]> wrote:

+1

Burt

On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

-- 
web: http://nerderg.com
Twitter: http://twitter.com/pmcneil

--
------------------------------------------------------------
Lead Developer - Fiehnlab, UC Davis

gert wohlgemuth
[hidden email]

work:
http://fiehnlab.ucdavis.edu/staff/wohlgemuth

phone:
[phone]

coding blog
http://codingandmore.blogspot.com

------------------------------------------------------------
you're "a long time dead", as they say

[Bernardo Gomez Palacio]

+1 on eliminating the sense of false security.

On Jun 6, 2011 9:20 PM, "wohlgemuth" <[hidden email]> wrote:
> i agree completely with this.
>
> On Mon, Jun 6, 2011 at 9:07 PM, Peter McNeil <[hidden email]> wrote:
>
>> +1
>>
>>
>> On 07/06/11 12:51, basejump (Josh) wrote:
>>
>> +1
>> We always seem to end up with at least a few tests in our projects that do
>> nothing because they were auto generated and do nothing but pass.
>> Easy to fix in our company templates but think it should be part of the
>> default grails templates
>>
>> On Jun 6, 2011, at 9:43 PM, Colin Harrington wrote:
>>
>> (+1)
>>
>> If we are going to generate them then I would want them to fail and have
>> action required to write a real test or to remove the empty test.
>>
>> Colin Harrington
>> [hidden email]
>>
>>
>> On Mon, Jun 6, 2011 at 9:39 PM, Burt Beckwith <
>> [hidden email]> wrote:
>>
>>> +1
>>>
>>> Burt
>>>
>>>
>>> On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:
>>>
>>>> Most of the default generated unit test stubs have an empty test method
>>>> in them. I think the test should be hard coded to explicitly fail to draw
>>>> attention to the fact that there is an empty or useless test there.
>>>>
>>>> What say ye?
>>>>
>>>>
>>>>
>>>> jb
>>>>
>>>> --
>>>> Jeff Brown
>>>> [hidden email]
>>>> SpringSource - A Division Of VMware
>>>> http://www.springsource.com/
>>>>
>>>> Autism Strikes 1 in 166
>>>> Find The Cause ~ Find The Cure
>>>> http://www.autismspeaks.org/
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe from this list, please visit:
>>>>
>>>> http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>> --
>> web: http://nerderg.com
>> Twitter: http://twitter.com/pmcneil
>>
>>
>
>
> --
> ------------------------------------------------------------
> Lead Developer - Fiehnlab, UC Davis
>
> gert wohlgemuth
> [hidden email]
>
> work:
> http://fiehnlab.ucdavis.edu/staff/wohlgemuth
>
> phone:
> [phone]
>
> coding blog
> http://codingandmore.blogspot.com
>
> ------------------------------------------------------------
> you're "a long time dead", as they say

[robpatrick]

+1

On 7 Jun 2011, at 05:07, Peter McNeil <[hidden email]> wrote:

+1

On 07/06/11 12:51, basejump (Josh) wrote:

+1

We always seem to end up with at least a few tests in our projects that do nothing because they were auto generated and do nothing but pass.

Easy to fix in our company templates but think it should be part of the default grails templates

On Jun 6, 2011, at 9:43 PM, Colin Harrington wrote:

(+1)

If we are going to generate them then I would want them to fail and have action required to write a real test or to remove the empty test.

Colin Harrington
[hidden email]

On Mon, Jun 6, 2011 at 9:39 PM, Burt Beckwith <[hidden email]> wrote:

+1

Burt

On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[John Fletcher-3]

If you take out the testSomething() method I think it gives you an error "no tests to run" or something - that might be appropriate. Just comment that line.

 

John

2011/6/7 Rob Patrick <[hidden email]>

+1

On 7 Jun 2011, at 05:07, Peter McNeil <[hidden email]> wrote:

+1

On 07/06/11 12:51, basejump (Josh) wrote:

+1

We always seem to end up with at least a few tests in our projects that do nothing because they were auto generated and do nothing but pass.

Easy to fix in our company templates but think it should be part of the default grails templates

On Jun 6, 2011, at 9:43 PM, Colin Harrington wrote:

(+1)

If we are going to generate them then I would want them to fail and have action required to write a real test or to remove the empty test.

Colin Harrington
[hidden email]

On Mon, Jun 6, 2011 at 9:39 PM, Burt Beckwith <[hidden email]> wrote:

+1

Burt

On Mon, Jun 6, 2011 at 10:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

-- 
web: http://nerderg.com
Twitter: http://twitter.com/pmcneil

[Ted Naleid]

+1 

but while we're on the topic, I'd also like a switch to not generate a unit test class.  Often, the first type of test that I use for TDD is an integration test (especially with services), and not a unit test so I need to go through additional work to remove that unit test or move it over to the integration folder.  This would get more in my way if it's autogenerating a failing test that I don't plan on using.

On Mon, Jun 6, 2011 at 9:15 PM, Jeff Brown <[hidden email]> wrote:

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.

What say ye?



jb

--
Jeff Brown
[hidden email]
SpringSource - A Division Of VMware
http://www.springsource.com/

Autism Strikes 1 in 166
Find The Cause ~ Find The Cure
http://www.autismspeaks.org/


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[Graeme Rocher-4]

On Tue, Jun 7, 2011 at 5:10 PM, Ted Naleid <[hidden email]> wrote:
> +1
> but while we're on the topic, I'd also like a switch to not generate a unit
> test class.  Often, the first type of test that I use for TDD is an
> integration test (especially with services), and not a unit test so I need
> to go through additional work to remove that unit test or move it over to
> the integration folder.  This would get more in my way if it's
> autogenerating a failing test that I don't plan on using.

-1

We should be identifying why is it you need to write an integration
test and improving Grails to remove this requirement

Cheers

> On Mon, Jun 6, 2011 at 9:15 PM, Jeff Brown <[hidden email]> wrote:
>>
>> Most of the default generated unit test stubs have an empty test method in
>> them.  I think the test should be hard coded to explicitly fail to draw
>> attention to the fact that there is an empty or useless test there.
>>
>> What say ye?
>>
>>
>>
>> jb
>>
>> --
>> Jeff Brown
>> [hidden email]
>> SpringSource - A Division Of VMware
>> http://www.springsource.com/
>>
>> Autism Strikes 1 in 166
>> Find The Cause ~ Find The Cure
>> http://www.autismspeaks.org/
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>
>

--
Graeme Rocher
Grails Project Lead
SpringSource - A Division of VMware
http://www.springsource.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Ted Naleid]

On Tue, Jun 7, 2011 at 10:12 AM, Graeme Rocher <[hidden email]> wrote:

On Tue, Jun 7, 2011 at 5:10 PM, Ted Naleid <[hidden email]> wrote:
> +1
> but while we're on the topic, I'd also like a switch to not generate a unit
> test class.  Often, the first type of test that I use for TDD is an
> integration test (especially with services), and not a unit test so I need
> to go through additional work to remove that unit test or move it over to
> the integration folder.  This would get more in my way if it's
> autogenerating a failing test that I don't plan on using.

-1

We should be identifying why is it you need to write an integration
test and improving Grails to remove this requirement

Most of the time it's for services that are executing HQL queries with joins.  I use build-test-data to create the test data and then can execute real HQL queries and assert that they're working like I want them to.

Sometimes it's also easier to let calls go out to other spring injected services for the real processing to happen rather than mocking out a service method.  Unit testing methods completely in isolation gives me confidence that a particular method is working, but not confidence that the system works as a whole.  Integration tests give me that.

My current project has 2368 unit tests and 616 integration tests, so I do favor unit tests for their speed, but ~20% of the time, doing an integration test feels appropriate to me.

-Ted

On 08/06/2011, at 4:28 AM, Ted Naleid wrote:

> Most of the time it's for services that are executing HQL queries with joins.  I use build-test-data to create the test data and then can execute real HQL queries and assert that they're working like I want them to.
>
> Sometimes it's also easier to let calls go out to other spring injected services for the real processing to happen rather than mocking out a service method.  Unit testing methods completely in isolation gives me confidence that a particular method is working, but not confidence that the system works as a whole.  Integration tests give me that.
>
> My current project has 2368 unit tests and 616 integration tests, so I do favor unit tests for their speed, but ~20% of the time, doing an integration test feels appropriate to me.

+1

At least with 1.3, unit tests are just not reliable (too many differences in behaviour to production) and the setup makes them too fragile.

Why are we even creating these rather pointless stub files?



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[John Fletcher-3]

Well it saves you from creating them and kind of reminds you that you should. And don't forget that such things are important for new developers - it shows where such files should go and demonstrates a kind of "best practice" - create a service, create some tests. I know that it seems rather obvious to those on this list that unit tests go into test/unit and have a name like BookServiceTests but a surprisingly high proportion of those coming to grails (especially from Java) may have never, or rarely, written tests in their life. It's a big jump to start learning and investing in something that doesn't actually give your app any new features so it's not a bad idea to make this as easy and obvious as possible. Maybe the template should even contain a comment with a link to the testing page of the doco. Or maybe not.

 

A further alternative would be a config option allowing you to switch off this auto-generation of test files. Or a switch to the create command that enables/disables it.

 

I'm for keeping the auto-generation behaviour. As for fail by default, go for it if you like. Further to my last email, I just checked and if you comment out the void testSomething() method, you get the following result when you run the test

initializationError

Error

No runnable methods

java.lang.Exception: No runnable methods

 

Not necessarily sure if that's the clearest so perhaps the other suggestion of throwing an error directly is better.

John
 

2011/6/8 Luke Daley <[hidden email]>

On 08/06/2011, at 4:28 AM, Ted Naleid wrote:

> Most of the time it's for services that are executing HQL queries with joins.  I use build-test-data to create the test data and then can execute real HQL queries and assert that they're working like I want them to.
>
> Sometimes it's also easier to let calls go out to other spring injected services for the real processing to happen rather than mocking out a service method.  Unit testing methods completely in isolation gives me confidence that a particular method is working, but not confidence that the system works as a whole.  Integration tests give me that.
>
> My current project has 2368 unit tests and 616 integration tests, so I do favor unit tests for their speed, but ~20% of the time, doing an integration test feels appropriate to me.

+1

At least with 1.3, unit tests are just not reliable (too many differences in behaviour to production) and the setup makes them too fragile.

Why are we even creating these rather pointless stub files?

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

[Ted Naleid]

On Wed, Jun 8, 2011 at 2:26 AM, John Fletcher <[hidden email]> wrote:

A further alternative would be a config option allowing you to switch off this auto-generation of test files. Or a switch to the create command that enables/disables it.

Yep, this was my original suggestion, just a switch for those who know enough about grails/testing to turn off autogeneration of the unit test.

I agree with creating tests by default and think it's a good practice in most situations, and I think that having the default be a failing test is good too.  For me, it's not that I don't want a test, it's that I want a different kind of test than the default one that I want 80% of the time.

This isn't a huge deal for me, I can easily set up a zsh function to whack any autogenerated tests if this bugs me enough:

# delete autogenerated unit test from last "grails create*" command. ex: grails create-domain com.example.Foo

alias dut='rm -i test/unit/$(echo ${${(z)$(fc -ln 0 | grep grails\ create | tail -1)}[3]} | sed -e 's/\\\\./\\\\//g')Tests.groovy'

done :)

-Ted

[Gavin-15]

On 08/06/11 04:28, Ted Naleid wrote:

> Most of the time it's for services that are executing HQL queries with
> joins.  I use build-test-data to create the test data and then can
> execute real HQL queries and assert that they're working like I want
> them to.
>
> Sometimes it's also easier to let calls go out to other spring
> injected services for the real processing to happen rather than
> mocking out a service method.  Unit testing methods completely in
> isolation gives me confidence that a particular method is working, but
> not confidence that the system works as a whole.  Integration tests
> give me that.

Same here.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Predrag Knezevic]

+1

On Tue, Jun 7, 2011 at 4:15 AM, Jeff Brown <[hidden email]> wrote:

> Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.
>
> What say ye?
>
>
>
> jb
>
> --
> Jeff Brown
> [hidden email]
> SpringSource - A Division Of VMware
> http://www.springsource.com/
>
> Autism Strikes 1 in 166
> Find The Cause ~ Find The Cure
> http://www.autismspeaks.org/
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Marco Vermeulen]

+1
This is something that has bothered me from day 1

Jeff Brown-4 wrote

Most of the default generated unit test stubs have an empty test method in them.  I think the test should be hard coded to explicitly fail to draw attention to the fact that there is an empty or useless test there.