Finding the Bug Is Only the Beginning
Early in my career, I transitioned into testing games on iOS and Android devices. Although the platform was new to me, the process was familiar: find a bug, reproduce it, and log it. What I didn't realize was that mobile testing would help sharpen what I consider QA's most valuable tool: critical thinking.
During one project, we encountered a crash in a game we were testing exclusively on iOS devices. Shortly after I noticed the crash, I thought I had isolated its cause. However, I noticed it occurring on another iOS device, and with slightly different repro steps. After an exhaustive test session, we thought it was a “random” bug that seemed to have occurred without any rhyme or reason. So we logged the bug as occurring at random, with the summary being something close to “A crash occurs seemingly at random when loading X character into Y background.”
After the bug was logged, we had some back and forth with the engineer in the bug comments (which by the way, is wildly inefficient). Eventually it was assigned back to QA as “Need More Info” with a comment that QA should get a more reliable repro. We in QA got together and brainstormed on what the differences were between our devices who were seeing the crash and those who were not seeing the crash. It turns out, the crash was only occurring on devices that were set to “Low Power Mode”. I never owned an iOS device at the time, so I didn’t even know that “Low Power Mode” was a thing, much less how it affected software when it was enabled.
Bug investigation doesn’t stop when it’s discovered. Rather, the discovery is the beginning. It’s when the investigation ramps up. Once we realized there had to be another variable for the crash, our questions changed. Why did it only happen on some devices? What was different about those devices? What had we overlooked? Could we reproduce it consistently if we found the missing condition?
A bug may appear random, but software behaves deterministically. There is usually an underlying catalyst or combination of conditions waiting to be discovered. Investigating a bug means stepping back from the symptom and asking better questions. Reliable reproduction is often the result of that process, not the process itself.
If we had accepted that crash as "random," the investigation would have ended there. Instead, it led us to a condition none of us had even considered. That's what good QA looks like in practice. Finding the bug wasn't the end of the investigation. It was the beginning.