We had an integration that said it was succeeding. The logs said it worked. The request returned. No exception was thrown. Nothing showed up where it was supposed to.

That is the kind of bug that wastes time because the system is not just failing. It is lying about the failure.

Our definition of success was too weak

We were pushing notes into an external EHR. The integration decided whether a push had succeeded by checking for one thing: Was there an ErrorMessage? If not, we treated the request as successful.

That sounds reasonable until the vendor returns this:

{"Error": "...not valid"}

Not ErrorMessage. Error.

The request contained a failure. Our code did not recognize the field. So we logged success. Nothing landed.

Absence of an error is not proof of success

This was the first problem. We had defined success negatively. Not: “Did we get the thing we expected back?” But: “Did we see one of the failures we know how to recognize?”

Those are very different contracts. The vendor response could change shape. A different error field could appear. The response could be incomplete. The request could technically succeed while the operation itself failed.

Our code would still say everything was fine as long as ErrorMessage was missing. That is a fragile definition of success.

And in production, fragile success checks are often worse than obvious failures. An obvious failure gets investigated. A false success gets trusted.

There was a second trap

The note integration also stored the external note ID after creating a note. Later pushes reused that ID. That worked until a provider used the EHR's "Remove All" action.

From the EHR's point of view, that note was gone. From our point of view, we still had its ID. So we kept trying to update a note that no longer existed.

The EHR returned an error saying the ID was not valid. Our code ignored it because it was looking for the wrong error field. Then it logged success. On the next attempt, we tried the same stale ID again.

The system could get stuck repeating a request that could never work while telling us every attempt succeeded.

A refactor made the old assumption dangerous

There was another useful detail in this failure. The integration had previously checked whether the response actually contained an ID. At some point, a refactor replaced that with the error-field check.

The new version looked cleaner. It also removed the strongest evidence we had that the operation had actually worked. That turned an existing edge case into a silent one.

The problem was not simply that the vendor had an awkward API. We had weakened our own definition of success.

The fix

We changed both parts of the behaviour.

First, either Error or ErrorMessage meant failure. More importantly, we stopped treating "no recognised error" as enough. A successful create needed to return the thing we expected: a valid note ID.

Second, when the EHR told us the stored ID was no longer valid, we stopped retrying the same stale reference. We dropped it and created a new note. That let the integration recover instead of remaining stuck on a resource that no longer existed.

Success should have positive evidence

The most useful lesson from this incident was simple: Define success by what happened, not by what did not happen.

If you create a remote resource, success might mean:

  • An ID came back
  • The expected state changed
  • All required sections were accepted
  • The write can be read back
  • The downstream system confirms the operation

The exact proof depends on the system. But "we did not see an error we recognize" is usually not enough.

That rule gets even more important around third-party integrations. You do not control their response shapes. You do not control every state transition. You do not control what users can change directly in their system.

The only thing you control is how much evidence you require before calling something successful.

The part I would design differently now

I would make success explicit at the boundary. Not inferred. Not based on missing fields. Not based on a request completing without throwing.

For any external write, I want to be able to answer: “What concrete thing proves this operation happened?”

If the answer is vague, the integration is probably carrying more uncertainty than the logs admit.

This bug looked like a vendor problem. The more important failure was ours. We had taught the system to confuse silence with success.

STATUS ● SHIPPED

END OF FILE