- What are the primary objectives and goals of the testing phase?
- Product
- Main features of our product? What are the use cases?
- Which platform supported (OS, hardware, devices)
- Who are the target users?
- Test new feature addition or completely as a product? Or its core functionalities?
- What kind of features we want to test?
- How can something go wrong?
- Testing
- Do we have any tests? Do we have unit/integration? Do we have any manual? Do we have functional automated tests? Do you have non-func testing?
- Which features should be tested? And what should not be tested? Do you any have any priority list, or I need to decide?
- Is it right time for testing? If there is no project design document or a clear vision for the product, it may be too early for testing,
- Has testability been considered in the project design? Before a project gets too far into implementation, all scenarios must be designed as testable.
- What kind of testing we want to perform? Do we want to do functional and non-functional?
- How much time do we have, so that we will prioritize the scope and approaches.
- Do we have automation? If yes do we have a framework? Planned percentage of automation? End goal?
- Why do we need automation? Benefits from automation (Faster, reliable, cheaper in a long term) Automated tests can run faster and more frequently (reusable) than manual tests, allowing for quicker feedback on the software's quality. Early detection of bugs, . They save time that leads the shorter time-to-market of product.
- What is the end line or acceptance criteria? For example: 100% success on smoke, 90% on extended path. Requirements coverage 80%.
- Do we have any known bugs that should be verified?
- Release frequency, how and how frequently feedback/report should be provided?
- Tooling and infrastructure
- Do we have any tools, do we have any TAF that we can reuse in our project?
- For E2E testing, how will you build test environment/infrastructure?
- What builds are your tests running against? Which branch is a release candidate? Environments (QA, DEV, PROD)? How to use?
- 3rd parties? Any mocking?
- Risks
- What are the bounds to use? What is acceptable and not? What degree of false data is acceptable?. Ex: temperature, pressure
- What are the stress / failure conditions? Which situation is inevitable, how the failure looks like?
- Process (how to use tests during release, is there any scheduled testing?, how to handle bugs, monitoring with metrics)
- How can readers review the actual test cases? How to ensure that automated tests are correct?
- Are there test schedule requirements? (When provide feedback)
- How are builds and tests run continuously? CI/CD, small tests run of CI
- How to report results and monitor? Who will get report? Who will track reports? How quality will be monitored?
- How are tests used when releasing?
- How users will report bug?
- Metrics
- Do you have any general product health or quality goals and how will you measure success? Consider:
- Number of bugs caught by users in production
- Number of bugs caught in release testing
- Number of open bugs over time
- Code coverage
- Cost of manual testing
- Why are you running these tests?
- Why did you write that specific test?
- Why did you choose to automate this task over this other one?
- Why are we investing in this tool?
Add: Ensuring the quality and reliability of software applications before they are released to end-users, meeting customer expectations — follow requirements, and enhance the overall software development process, help reducing risks.
