What White Box Testing Misses When Your Code Talks to External Services

something outside that boundary. A payment processing service with 87% code coverage sounds well-tested. Walk through what that coverage actually validates and the picture gets less reassuring. The tests exercise the request construction logic.

They exercise the response parsing logic. They exercise the retry logic. All of that runs against a mock of the payment gateway - a mock that was written when the integration was first built, reflecting what the gateway returned at that point in time. What the coverage number doesn't capture: whether the mock still reflects what the gateway actually returns.

Whether the retry logic handles the specific error format the gateway uses today, not the format it used when the mock was written. Whether the fields the parsing logic depends on are still present in the real response. The code is covered. The integration isn't tested.

White box testing measures whether the code does what the developer intended. That's a meaningful thing to measure for logic that's self-contained - algorithms, data transformations, business rules that don't depend on external state. For code that integrates with external services, the developer's intention is only half the equation. The other half is whether the external service behaves the way the developer assumed it would.

White box testing has no visibility into that half. It can tell you the request was constructed according to the developer's model of the API. It can't tell you whether that model matches how the API actually behaves today. The gap between "the code does what I intended" and "the integration works correctly" is exactly where production failures accumulate in services with external dependencies.

The mock is the problem, but it doesn't look like the problem. When a test passes against a mock, the implicit claim is that the code handles the dependency correctly. The test doesn't make this claim explicitly, but that's what passing means to the person reading the CI results. Green means the integration works.

What the mock actually validates: the code handles the simulated dependency correctly. Whether the simulation is accurate is a separate question that white box testing doesn't ask. External APIs change without coordinating with the teams consuming them. A field gets renamed.

A previously optional key becomes required. An error code gets restructured. The pagination behavior changes. None of these changes break the mock - the mock doesn't know about them.

The tests keep passing. The code that was validated against the old behavior gets deployed to production, where it encounters the new behavior for the first time. This isn't a testing failure in the sense of bad test design. It's a structural limitation of what white box testing can see.

Code coverage tools instrument the code under test. They can observe every branch and condition inside the service. What they can't observe is the behavior of the external service itself - what it actually returns under what conditions, how its response structure has evolved, what its failure modes look like under real load. That boundary is the one that matters most for integration reliability.

The bugs that white box testing catches well - logic errors, edge case handling, incorrect branch conditions - are the bugs that would have been caught in code review or local testing anyway. The bugs it misses - assumptions about external service behavior that are no longer accurate - are the ones that survive to production because nothing in the test suite was positioned to catch them. Observation-based testing approaches close this gap differently. Instead of asserting that the code handles a mock correctly, they record what the real service actually returns and validate the code against those observations.

When the external service changes, re-recording shows exactly what shifted. The team finds out before deployment rather than from a production failure. White box testing remains the right tool for validating internal logic. The retry algorithm, the request construction, the response parsing - these are code paths that benefit from branch coverage and condition testing.

What needs complementary coverage is the contract between the service and its external dependencies: whether the requests the service sends match what the external service expects, and whether the responses the external service returns match what the service's parsing logic assumes. That contract isn't visible inside the code. It exists between the service and the external world, and validating it requires something that can observe both sides - not just the internal logic, but the actual behavior of the dependency the code is written to handle. White box testing tells you the code is internally consistent.

It can't tell you whether the world the code was written to interact with still looks the way the code expects it to.