Add basic pytest tests for umdp3_conformance.py#216
Add basic pytest tests for umdp3_conformance.py#216R Sharp (r-sharp) wants to merge 19 commits intoMetOffice:mainfrom
Conversation
…es which git / VS Code are marking as in conflict. Annoyingly, this seems to have introduced a bug as well.
…esolving clashes. But looking at it, I really can't fathom out 'how'
…n't edited this file...
…iority checks missing from umdp3_conformance.py. That was fun
…sts - I think the power has gone to it's head...
…ext line where it no longer works...
…rmatter, or was it the other way round...
Sam Clarke-Green (t00sa)
left a comment
There was a problem hiding this comment.
Looks sensible. I agree with the developer that some of the tests, e.g. the argument processing, are re-checking existing functionality, e.g. argparse. If values are required or the testing is for logical correctness, either runtime error handling or asserts in the base could would be better options.
Erica Neininger (ericaneininger)
left a comment
There was a problem hiding this comment.
This looks fine, barring a missing Copyright notice please.
Comments above, regarding the need for complex unit tests to test basic functionality, or testing library functionality, are noted.
With the ideology that the test should be written first and the code written to pass the test, Co-Pilot concocting some of these apparently pointless tests are more understandable - but as is also noted, they may not be worth maintaining.
| @@ -0,0 +1,637 @@ | |||
| import sys | |||
There was a problem hiding this comment.
This new file should probably have a copyright notice. The tested source in umdp3_conformance.py doesn't have one either.
| import sys | |
| # ----------------------------------------------------------------------------- | |
| # (C) Crown copyright Met Office. All rights reserved. | |
| # The file LICENCE, distributed with this code, contains details of the terms | |
| # under which the code may be used. | |
| # ----------------------------------------------------------------------------- | |
| import sys |
PR Summary
Sci/Tech Reviewer: Sam Clarke-Green (@t00sa)
Code Reviewer: Erica Neininger (@ericaneininger)
Adding tests to test the tester not tests to to test the tests the tester runs.
If these are tests to test the tester, where's the tests to test the tests that test the tester.
A range of pytest unit tests to test section of umdp3-conformance.py.
Some of these were suggested by CoPilot as not only as things that could/should be tested, but CoPilot was also asked to supply the original code to form the tests. (mostly where monkeypatch was involved as I'd never really played with that before).
Particularly in the case of the CoPilot written ones, they work, they seem to go to great lengths to set up objects (real or patched) and then essentially test what are trivial actions/logic in the original code.
As a method of quickly highlighting when changes are made to those objects - great.
As ways of ensuring the code does what you expect - less so. However I think integration testing would be required for most of that, not unit testing.
Code Quality Checklist
Testing
These tests have been run. They all pass.
They don't affect the actual code, so that hasn't been run in this instance.
Security Considerations
AI Assistance and Attribution
I used (I believe) GitHub CoPilot (within VSCode) to list the things it thought I should test and to generate some of this code.
Sci/Tech Review
(Please alert the code reviewer via a tag when you have approved the SR)
Code Review