We aren’t sure how it happened, but we ended up with the feature tests for the Costs to Expect API being a Postman collection. There are over four hundred requests in the test collection and thousands of assertions are run on the responses, somewhere in the region of three thousand.
We like Postman, we use it every day. We use Postman for development and our documentation, public and private. You can view the documentation for the Costs to Expect API at https://postman.costs-to-expect.com
We have come to the realisation, much too late, that testing is not something we should be doing in Postman. We have started moving/recreating our feature tests, we have opted for local PHPUnit tests.
Recreating our tests will be a slow process, every new release will include tests, however, we are not focusing on just tests. “Why?” you ask. Well, we have tests, they simply aren’t ideal.
We are aiming to have our local tests in place as soon as possible, as much as that is our goal, we must be realistic, we simply can’t justify spending weeks writing new tests when we have a suite of working tests.
Moving to local tests is paying off already. In the latest release, we improved the first-install process for the API and made the code more testable. These improvements are a direct result of moving to local tests.
We have added a section to the README detailing our progress. We don’t want this section to exist in the README for too long so will ensure we add a decent number of useful tests each release.