The Hermetic End to End Testing Pattern
The hermetic end to end testing pattern is a pattern where:
-
The entire app is run as a cohesive whole.
-
The entire "outside world" for this app is run in a strictly controlled mock environment. For example, if it:
Calls an external sandbox REST API. Uses a staging database whose exact data isn't controlled. ** Accesses the the internet.
Then it is not hermetic. If it:
Calls a mock REST API (e.g. using wiremock or mitmproxy). Uses a locally built database with consistent pre-defined fixtures. ** Doesn't access the internet.
Then it is hermetic.
Benefits
Hermetic tests are:
- Faster.
- More consistent.
- Trivially parallelizable
Partially hermetic end to end tests
Partial hermeticism is when every part of the end to end test is mocked, but it can be run in a mode that partially interacts with the outside world.
An example of this would be a hermetic end to end test that calls a mocked Paypal REST API using wiremock or mitmprox, but which can be run in a mode that uses the sandboxed Paypal REST API instead.
Partial hermeticism can be used as:
- An efficient way of creating mocks (by recording the actual request / response).
- A way of consistently testing "outside world" changes - e.g. if an API the app calls still works the same way tomorrow as it did yesterday.