iteration

❌ Extreme Testing

Episode Summary

JP and John walk through preventing bugs with automated tests. The practical real world walk throughs of their approaches to testing and how to prevent shipping bugs out into the world.

Episode Notes

Iteration — A weekly podcast about programming, development, and design through the lens of amazing books, chapter-by-chapter.

This episode uses Chapter 13 of Extreme programing as a jumping off point

Testing Early, Often, and Automated

Here is the dilemma in software development: defects are expensive, but eliminating defects is also expensive.

• Instead of talking about having a kick-ass automated test suite, frequently testing code, and all of that good stuff, we should talk about something a little different.

Q: What was the last big bug you can think of? How did you handle it?

• Health coaching group discussions, mobile users could not post into group discussions for 48 hours. It’s a core feature of the platform.
• React Native Upgrade → Weird android bug where people could not take a photo to upload their proof of funds

Q: What was the reason for the latest big bug?

• lack of automated integration tests in the Mobile app.
• We have some unit testing we also have full API tests, but not enough coverage in the integration.
• We made a small change to the group discussion model, the mobile app wasn’t updated to consider this change, timing the rollout.
• Android Weirdness, lack of QA, rushing to get to the latest and greatest

Q: How do you handle bugs in general?

• Testing is first line of defense.
• QA is backup (One other dev)
• App signal is the catch all (After deployment)
• Users are the last line of defense RABBITHOLE
  • Give users a clear escape hatch / line of communication.
  • So many times we’ve caught flawed design assumptions from that feedback. Users love it too.
  • Superlative language "Thank you so much, you are the best, we really really appreciate you sending that in and I'm so sorry if you are having issues. "

Q: John Ask: What’s the most common causes of bugs?

• Test data does not properly represent production data - phone formats / etc
• An Edge case that wasn’t considered
• The feature functions properly but the functionality is wrong / not what the stakeholder intended.
• Rollout was not planned, data transition or all platforms are not in sync.
• Browser or context considerations. Mobile vs tablet / android device universe.
• Lack of QA, lack of tests
• Rushing things

John — Summary / Thoughts on book overall

• Overall got the most out of this vs other books we've read

Picks

• JP: Heroku! Review Apps (spins up a new app every time a PR is opened); Pipelines
• John: Contentful + Rails to give clients ability to update copy and images on marketing pages.