Comparing Cassettes

Comparison with the test server method

This approach differs from a test server because it focuses less on asserting that the request is made correctly. The main goal is to behave exactly like the real third-party service without writing code to emulate that third-party service. The workflow is, in fact, simpler than the test server, in that we wrap our code with use_cassette/2 and go on about our day.

However, cassettes present a similar problem to the test server: they can get out of sync with the actual API. The solution lies in keeping cassettes up-to-date.

Keeping cassettes up-to-date

Compared to a test server, we adopt a much simpler approach to keep the cassette up to date. We only have to delete the stale cassette and rerun our tests.

To further push the idea of keeping cassettes up to date, we can always force real requests when running tests in continuous integration (CI) server. This way, we’ll avoid making real HTTP requests when developing on our local machine, but the CI server (which usually runs much less frequently) will ensure that the cassettes haven’t gotten out-of-date.

Note: This approach heavily depends on what requests the real API implies.

In the weather API example, making real requests to /data/2.5/forecast is feasible. If it’s only done in CI, then it’s unlikely that we’ll negatively affect our rate limiting.

In other cases, making requests might cost money or break things, so making real requests on every CI run might not be ideal. Furthermore, we usually want CI to be reproducible and consistent between runs, and depending on the availability of an external API might not be feasible.

Favorite use case

Cassettes can be used as an external service that allow us to set up and tear down resources through its API.

For example, the weather API could expose endpoints to register and delete named queries. If we wanted to test that we can query the forecast through a named query, we could create the named query, test the appropriate functions, and delete the named query all in the same test. In this use case, the cassette merely becomes a cache of HTTP requests. Even when not using cassettes (such as in CI), the test would create and destroy all necessary resources to run the test, leaving the external service’s state unchanged.

Never reuse cassettes!

If we have two tests that make the same request, we might be tempted to use the same cassette, but there’s a good chance that at some point, the request made in one of the two tests will change slightly, and then things won’t work anymore.

If we use a different cassette in each test, we’re guaranteed not to mess things up in this regard.