The wrong tests

We aren’t sure how it happened, but we ended up with the feature tests for the Costs to Expect API being a Postman collection. There are over four hundred requests in the test collection and thousands of assertions are run on the responses, somewhere in the region of three thousand.

We like Postman, we use it every day. We use Postman for development and our documentation, public and private. You can view the documentation for the Costs to Expect API at https://postman.costs-to-expect.com

We have come to the realisation, much too late, that testing is not something we should be doing in Postman. We have started moving/recreating our feature tests, we have opted for local PHPUnit tests.

Recreating our tests will be a slow process, every new release will include tests, however, we are not focusing on just tests. “Why?” you ask. Well, we have tests, they simply aren’t ideal.

We are aiming to have our local tests in place as soon as possible, as much as that is our goal, we must be realistic, we simply can’t justify spending weeks writing new tests when we have a suite of working tests.

Moving to local tests is paying off already. In the latest release, we improved the first-install process for the API and made the code more testable. These improvements are a direct result of moving to local tests.

We have added a section to the README detailing our progress. We don’t want this section to exist in the README for too long so will ensure we add a decent number of useful tests each release.

API – [v2.19.0]

With the impending releases we are making the API more testable and improving the first-install process.

Added

  • We have started transferring our Postman response tests to local feature tests.
  • We have added tests for the Authentication controller.
  • We have started writing tests for the ResourceTypeManage controller.
  • We have updated /auth/forgot-password and /auth/register, both now support a send GET parameter, if defined, no email will be issued.

Changed

  • We are tweaking the first install; we have squashed all the migrations and tweaked the Docker setup. We have added an initial-install.sql file, this includes the required data for the API.
  • We have made minor changes to how we return validation errors; we were calling exit and stalling our tests.
  • We have updated the responses for /auth/forgot-password and /auth/register; responses include the required follow-on URIs and parameters.
  • We have updated the README, we have added a Tests section and updated the setup steps.

API – [v2.18.0]

We are working towards releasing the beta for the Costs to Expect App and releasing other apps; we have therefore opened up the registration on the API.

Added

  • We have opened up registration on the API; you can register, login, and use all the expected authentication features.
  • We have added notification emails for registration and forgot password requests.

Changed

  • We have switched to Laravel Sanctum and removed all references to Laravel Passport, Sanctum makes more sense for our API.
  • We have updated to Laravel version 8.
  • We have tweaked our Docker setup and removed composer and phpunit.
  • Content updates

App – [v1.21.2]

We had to tweak our sign-in after a change to the Costs to Expect API, the API switched to Sanctum from Passport.

Changed

  • Switch to Sanctum on the Costs to Expect API, minor changes required.

Fixed

  • We have updated our cache invalidation; we will not clear as many cache entries on resource creation.

App – [v1.21.1]

We have made a number of fixes and tweaks, the changes can be seen below.

Changed

  • We have added named section links, you will be taken to the relevant data, not the top of a page.
  • We have updated our back-end dependencies.
  • We have added ‘expires’ to the “API Requests” table shown at the bottom on each page.

Fixed

  • With have fixed a small issue where the menu was not highlighting the active menu item.
  • We have updated the summary layouts for mobile devices; the summary counts have a little more breathing room.
  • We have increased our minimum TTL to 30 days. The one-day TTL was making a mockery of our asynchronous group requests when one or more requests return an empty response.
  • We have corrected an issue with our create resource forms; the ‘effective_date’ field HTML was malformed.
  • When you copy an expense of type ‘allocated-expense’, the effective date is copied.

App – [v1.21.0]

We have switched to using queues to clear cache.

Added

  • We have added migrations for the job and failed jobs table; we are switching to queues.
  • We have switched to queues for cache clearing. Approximately 50% of our clear cache requests will be delayed and processed later; the rest of the clears are synchronous; it depends on where we are sending the user next.
  • We have altered the TTL for API responses; the TTL is defined based on the age of the data in the API response, we use the X-Last-Updated header.

Changed

  • Switching to named routes, work in process.
  • Minor tweak to the meta viewport tag.
  • We have reviewed the App and removed a couple of single instance classes and found better homes for some others.