The pattern was familiar to everyone on the team. Every second Friday, a release went out, and every second Friday something went slightly wrong: a missing environment variable, a migration that ran in the wrong order, a service that restarted twice. Each time, the Monday meeting began with the same unspoken question: who did it this time? The team lead was tired of it. The engineers were tired of being looked at. And nothing was improving. This is a story about how that team stopped asking who and started asking what, and how the problem, once seen clearly, turned out to have a modest and fixable cause.
The symptom: the same kind of failure, different faces
On the surface, the failures looked unrelated. One week a feature flag was left off. Another week a database change needed a manual step that only one engineer remembered. A third week a rollback script pointed at an old directory. People tended to describe each event as a separate accident, and to attach each to a person: the one who was on call, the one who wrote the script, the one who approved the change.
Looking at them together changes the picture. Three events in three months, each hinging on some hand-done action from whoever happened to be closest to the release at that point, is not three accidents. It is one pattern: the release depended on humans remembering steps that lived in no shared place.
The wrong diagnosis: carelessness
The easy conclusion was that the team needed to be more careful. It is also the most common conclusion in situations like this one, and it nearly always fails. Telling capable people to be careful is like telling a driver to avoid potholes on a road full of them. Everybody was already trying. What they lacked was a release process that made the right action obvious and the wrong action hard.
There was a second cost, quieter than the first. Because the Monday meetings felt like trials, people had begun to soften their accounts. Details went missing. One engineer admitted months later that he had known about the manual migration step for weeks but did not raise it, because raising it felt like volunteering for the next blame session.
The better question
The turning point came when the team lead changed one sentence in the meeting invitation. Instead of “Friday release review”, it read “What did our release process make easy to get wrong?” The group agreed to a rule for the hour: describe what the system showed and what people could do, not what people ought to have done.
The facilitator, a developer from another team, drew a line on the whiteboard and plotted the last three releases along it. At each point where a human performed a step by hand, she placed a mark. By the end of the exercise the board had eleven marks, all in the same stretch of the process, between “build passes” and “service healthy”.
The diagnosis: a handoff with no owner and no record
The marks pointed to something concrete. The steps between a passing build and a healthy service had grown over time. Each had been added to fix an earlier problem, and each lived in a different place: a wiki page, a chat message pinned two years ago, a script on someone’s laptop. No single document listed them, and nothing checked that they had been done.
This is a process gap in the plain sense of the term. It is not about skill, motivation or attention. It is that the system relied on memory for work that had no reason to be remembered. That observation reconnects with broader discussions about how teams change how they work, such as the one in managing culture change when adopting DevOps, where moving routine knowledge out of people’s heads and into the pipeline is a recurring theme.
The fix: one change, not eleven
The team could have tried to fix all eleven marks. They did not. They asked which single change would remove the most manual steps. The answer was to write the release sequence as an automated job, a short script that ran the migration, set the flags from a versioned file and verified that the new service answered a health check before declaring the release done.
One engineer volunteered to build it, and the team agreed to review how it behaved over the next two releases. The remaining manual steps went into a backlog, ranked by how often they had caused trouble.
- Cause named: release steps lived in many places, and nothing confirmed they had been done.
- Change chosen: an automated release job that runs the steps in order and checks health.
- Owner and date: one engineer, with a check at the second release after launch.
What happened next
The first release using the job went out without incident, though it needed a small correction to the order of two steps. The team treated that correction as normal, with no meeting about who had written the wrong order. By the third release the Monday review had become short, and sometimes unnecessary, which the team lead considered the clearest sign of progress.
It also changed how people spoke. Engineers began flagging oddities as they saw them, since doing so no longer felt like a confession. A suspicious log line on a Thursday afternoon became a five-minute conversation instead of a Friday surprise.
Looking further ahead
Teams that take this route often find their view of the future shifts too. Fewer manual steps means more room for design work, and review habits become part of how the organisation grows. Readers interested in where this kind of practice is heading in Japan and elsewhere may enjoy the outlook for DevOps in Japan and around the world, which sets this sort of everyday discipline in a wider frame.
A similar lesson from the dining world
The same attitude applies in any trade built from repeatable steps. In a kitchen, one burnt tray is a note on oven timing, never a judgement on the cook. As an aside for the curious, FindMyResto lets people hunt for somewhere to eat, open menus, follow directions and reserve a table, with owners able to publish listing pages, an illustration of how a consumer service is stitched together from many small tasks that can each be examined without fault-finding.
A quick diagnostic for your own team
If your own releases, deploys or handoffs keep going wrong, try these questions before the next review.
- Do the failures involve a step performed by hand?
- Is that step written down in exactly one place that everyone uses?
- Does anything check afterwards that the step was done?
- Would a newcomer, on their first day, be able to perform it correctly?
A single “no” in that list usually marks the gap, and the repair is typically smaller than the tension surrounding it.
The short version
The Friday failures were never about careless people. They were about a process that left too much to memory and offered no confirmation. Once the team described the system instead of the person, the cause was visible within an hour, and the fix took an afternoon. If the same kind of failure keeps recurring on your team, resist the urge to look for a culprit, ask what the process made easy, and choose one change you can finish this week.









