How and why we Bug Bash at LinkedIn and why you should too
If you have been doing any kind of software development you are familiar with the bug bash concept in one way or another, even though you may call it something else. I want to share with you what I have learned doing bug bashes at LinkedIn for the past 2+ years and how it gives us confidence for releasing code at the scale of, well, the planet.
The what
Bug Bash is a cross-functional event for stress testing every bit of a new feature planned for external release. The format of this event is highly collaborative and async and with today’s circumstances, fully virtual, however, it has always had an optional virtual attendance under regular business situations as well. Bug Bashes usually range somewhere between 1 to 2 hours in time and includes members from Marketing, Business Development, accessibility (a11y) as well as the usual suspects of Design, Product and, Engineering (perhaps also QA for some companies. At LinkedIn we do not have a dedicated team for QA as we all wear that hat).
The How
Preparation
Preparation is arguably the most important part of the bug bash. This is usually done by one person (or one per platform). A well-prepared bug bash is destined for success while a not-prepared bug bash is a recipe for wasting your colleague’s time and a post launch failure.
The good thing is that preparation is not complicated and can even be done by the team intern. For the rest of this section let’s say we came up with a new Navigation bar for LinkedIn. Make sure to send out the prep doc along with the event invite and for those taking part in the bug bash in an async manner a note on the deadline to file the bugs.
A Good bug bash doc should include the following:
- POCs
- Set up instructions
- Test cases and Categories to cover
- Bug reporting instructions
- Resources - specs and anything else that may be required
Below is an example of such bug bash doc:
Bug Bash event
Make sure to send out the prepared doc for the bug bash at least 24 hours ahead of time, giving everyone ample time to set up and address any issue before the event so that you are not wasting time trying to debug individual issues.
At LinkedIn we like to start meeting with a success statement, so for example: “This meeting/bug-bash will be a success if we will be able to cover all test cases & categories mentioned in the preparation doc for all platforms”
[For Engineers] If possible avoid bug bashing your code as you might be so close to it that miss issues. Instead, focus your time bug bashing other’s code or be the resource for others for the session.
Remember 1–2 hours is not enough time so you may need to divide and conquer. As long as you cover all categories and come up with solid documentation of the issues it is a success. Based on the scope of the bug bash and the number of issues found, you may have to set up another bug bash for a future time. There is no shame in multiple bug bashes, if anything it tells you that the bug bash was a success.
Post Bug bash
Remember the bug bash is async so they might be some more bugs trickling in after the event. With that said this is the time where the team huddles and assesses if they are aligned on the mission and vision and if so, given the level of feedback from the bug bash are they on track for the previously communicated release deadline and if need be, make appropriate adjustments and communications. This is the time to triage the tickets and start working on addressing all issues.
Finally, and perhaps optionally (depending on the scale of the feature) you may send out a follow-up communication on the bug bash event and the next steps to the stakeholders. As a bonus, this is the step that makes the difference between a good POC and a great one so don’t skip this if you want to excel at your position. Do not forget to thank and give credit to all involved. Teamwork makes the dream work!
Feature Release
Bug Bash is not the ultimate bug-finding solution so as always ensure you are ramping the feature incrementally and measure and solicit feedback at every ramp stage.
What it means for Bug Bash to be async
Don’t let someone’s inability to attend the bug bash meeting be a deterrence of them being another set of eyes on the deliverable. If you have put the effort into the preparation step it should not cost you any additional effort to have the bug bash set up as an async event.
But what does it mean for the bug bash to be async? Async here means that those colleagues can contribute at their own pace or time given the same set of instructions and set up that was provided as part of the preparation. Just make sure to communicate a clear deadline on when to complete the bug bash by.
A11y
I am going to highlight this one piece just because it has traditionally been an afterthought when developing software. A11y for so many reasons that I will spare you why is not optional nor even a fast follow-up for any release. So please use bug bash as an opportunity to hash out any shortcomings with a11y and better yet next time if you have not done so in this cycle make sure to include a section in both Product and Design specs for a11y.
The Why
If you are reading this you are most likely familiar with the why so let me tell you why we at LinkedIn put a big emphasis on this. LinkedIn is a big company with very complex and interconnected pieces serving an even more diverse audience, it is practically impossible for any group of engineers (regardless of size) to be able to think of every use case and scenario not to mention the crazy big landscape of devices and operating systems and platforms. When we bug bash we have the opportunity to get fresh sets of eyes on our deliverable that were not there in the development process or perhaps not for the same platform (iOS vs. Android vs. web). Finally, my favorite is having those colleagues that just have different usability habits or requirements such as keyboard-only or screenreader requirements (or any assistive technology for that matter).
What bug bash is not…
For all its usefulness bug bash is not the right venue for a few things. For example, bug bash is not the place to question the overall objective of the feature or the product. It is completely valid to ask the question of whether or not the deliverable is aligned with the vision of the product and the mission of the company but it is not the right venue to question the vision and the mission. Also, a bug bash is not the right venue to exert your personal preferences, ideally and hopefully at a time much before the bug bash those feedbacks are solicited and taken into account, however, and regardless, at this point, the idea is to release to a wider audience of internal or external users for implicit and explicit feedback. Finally and perhaps most importantly, bug bash is not the venue to point fingers at any one person or group for a mistake or to try and figure out the how and who for addressing the issues that surfaced.
I want to end this post with my gratitude to you for taking the time to read my thoughts. I also want to express my desire that we in the software development community have an opportunity and responsibility to make the web a more inclusive, safe, and accessible place for all.
Keep on coding!! ❤️ 💻
* Opinions are my own and not the views of LinkedIn. Examples in this blog post are just that, examples, and are not based on any actual works at LinkedIn or by any of its employees (including yours truly)
