TL;DR
Overall we perform functional and non-functional testing.
If we talk about levels we perform smoke, critical path, extended/regression.
Tests classifications
Detailed Testing Classification122.5KB
By code execution
Static testing (verification) — without running the code. Dynamic testing (validation) — with running the code. - Static testing does the verification process while Dynamic testing does the validation process.
- Static testing is about the prevention of defects whereas dynamic testing is about finding and fixing the defects.
- Static testing was done without executing the program whereas dynamic testing is done by executing the program.
- Static testing checks requirement documents, and design documents to find errors whereas Dynamic testing checks the functional behavior of software system, memory/CPU usage and overall performance of the system.
- Static testing techniques are structural and statement coverage while Dynamic testing techniques are Boundary Value Analysis & Equivalence Partitioning.
By access to application code and architecture (by knowledge of code)
- White box method — there is access to the code.
- Black box method — there is no access to the code.
- Gray box method — some of the code is accessible, some of it is not.
By automation level
- Manual testing — test cases are performed by a person.
- Automated testing — test cases are partially or fully performed by a special testing tool.
By testing level
- Unit testing — individual small parts of an application are tested.
- Integration testing — the interaction between several parts of the application is tested.
- System testing — the application is tested as a whole.
By functional testing level (By test importance)
Smoke testing — testing of the most important, most crucial functionality. - Installation Tests: Testing the installation process of the software.
- UI Tests: Checking if the user interface elements are visible and responsive.
Critical path testing — testing the functionality used by typical users in typical daily activities. - Integration Tests: Testing the interaction between different components or modules of the software to ensure they work together as intended.
- Performance Tests: Assessing the software's performance under different conditions to identify any bottlenecks or issues.
- Security Tests: Evaluating the software's security measures to protect against vulnerabilities and potential breaches.
Extended testing — testing all (remaining) functionality stated in the requirements. - Functional Tests: Testing each feature and functionality of the software to ensure they work as intended.
- Regression Tests: Re-running tests to ensure that new changes or fixes haven't introduced new defects.
- Load and Stress Tests: Assessing the software's behavior under heavy loads and stress conditions.
- Usability Tests: Evaluating the user-friendliness and ease of use of the software.
- Compatibility Tests: Checking if the software works correctly on different devices, browsers, and operating systems.
- Localization and Internationalization Tests: Verifying that the software is ready for different languages and regions.
By positivity approach (positive and negative test case)
- Positive test case: acceptable actions, valid data, by requirements
- Negative test case: unacceptable actions, invalid data, out of bound of requirements.
Given the following requirement: “The system shall allow a new class to be added to the schedule.”
The positive test case would ensure that a new class (with all valid data) can be added to the schedule.
Negative test cases would try to “break” the software by testing things like:
By formalization level
- Ad-hoc testing
- Exploratory
- Test case based
By application nature
- Web-app testing
- Mobile-app testing
- Desktop-app testing
Types of software testing
Functional testing — are we building the product right? Determine if a piece of software is acting in accordance with pre-determined requirements.
Functional testing is validating if a system works as intended, meeting any given criteria.
The main points of functional testing
- Understanding customer requirements
- Testing input data
- Gathering expected results
- Executing test cases
- Comparing actual vs. expected results
- Updating results according to test case
Functional testing best practices – focus on:
- GUI coverage: with different screen resolutions, GUI elements like font size, color, etc.
- Input domain coverage: taking valid and correct size with appropriate formats.
- Return of correct data with correct input.
- Order of functionality (i.e. Tab order).
- Non-functional testing: This type of testing focuses on non-functional aspects of the software, such as performance, scalability, security, usability, and compatibility with different environments.
Unit testing — determines whether individual software components perform as expected (function, data structure, classes). - By validating across valid and invalid range of inputs and output parameters.
- When we combine two or more classes, it will be easier to test. (Integration testing)
- Done by developer
Integration testing — check that various pieces of the software work together as expected. - Examines the flow of data from one module to another.
- Example: In music migrator I use → (Yandex and Spotify, i need to test the flow of music whether correct and synchronized in both servers)
System testing — assesses both functional and non-functional testing requirements. Performed on whole system by checking whether the system or application meets the requirement specification document.
Smoke tests — check if critical functions of product work. - Smoke tests are performed quickly to confirm whether to accept or reject the build.
- It is a subset of acceptance testing.
Sanity testing — done when a new change or bug fix is implemented. - Done to check that bugs have been fixed after the build.
- Covers certain modules, in which code changes have been made. Был изменён код для изменения пароля, то значит нам нужно тестировать логин и регистрацию тоже.
Regression testing — check if nothing is broken after change. - Regression testing - recheck what we have already tested, done after code changes, updates, improvements.
- Regression testing is responsible for the overall stability and functionality of the existing features.
- Even small changes or modifications to software can bring unexpected issues. Regression testing becomes very important to test whether existing functionality is impacted. Regression testing ensures every functionality is working properly after any software update or modification happens.
- Done before each release
- How to select test cases? (High prob with mistakes, rapidly changing, critical components)
- Smoke vs regression tests: smokes tests run quickly to confirm whether accept or reject the build. Regression testing is performed to check whether the changed code affects the existing feature or not. Smoke testing is performed on newly developed software, before running regression tests.
Regression testing is done when:
- Any patches added to software or configuration of software have been changed.
- If new Change Requests are implemented to the build.
- New Features are added or any functionality has been added.
- Existing functionality is modified with improvements.
Sometimes regression testing is implemented to:
- Find bugs that can found because of new changes or modifications done on the build.
- Check for side effects after fixing bugs or any changes made.
- E2E testing — checks if flow of data is maintained if different operations are performed.
Acceptance testing — verifies that a specification or contract’s criteria are satisfied in accordance with its delivery. Done by client. Acceptance testing is intended at ensuring that the product meets minimum defined standards of quality prior to it being accept by the client or customer. Often the client will have his end-users to conduct the testing to verify the software has been implemented to their satisfaction (this is called User acceptance testing or UAT). Often UAT tests processes outside of the software itself to make sure the whole solution works as advertised.
Alpha testing comes into when enough functionality has been reasonably completed to enable the first round of (end-to-end) system testing to commence. At this point the interface might not be complete and the system may have many bugs.
Beta testing states that the bulk of functionality and the interface has been completed and remaining work is aimed at improving performance, eliminating defects and completing cosmetic work. At this point many defects still remain but they are generally well understood. Beta testing is often associated with the first end-user tests.
Alpha Testing | Beta Testing |
Alpha testing performed by Testers who are usually internal employees of the organization | Beta testing is performed by Clients or End Users who are not employees of the organization |
Alpha Testing performed at developer's site | Beta testing is performed at a client location or end user of the product |
Reliability and Security Testing are not performed in-depth Alpha Testing | Reliability, Security, Robustness are checked during Beta Testing |
Alpha testing involves both the white box and black box techniques | Beta Testing typically uses Black Box Testing |
Alpha testing requires a lab environment or testing environment | Beta testing doesn't require any lab environment or testing environment. The software is made available to the public and is said to be real time environment |
Long execution cycle may be required for Alpha testing | Only a few weeks of execution are required for Beta testing |
Critical issues or fixes can be addressed by developers immediately in Alpha testing | Most of the issues or feedback is collected from Beta testing will be implemented in future versions of the product |
Alpha testing is to ensure the quality of the product before moving to Beta testing | Beta testing also concentrates on the quality of the product, but gathers users input on the product and ensures that the product is ready for real time users. |
A/B testing — allows you to compare 2 versions of something to learn which is more effective. "Do your users like version A or version B better?" Why use A/B testing?
- To understand the visitors pain points. Which version they like.
- Get better ROI from existing traffic
- Make low-risk modifications (subset of users)
- Experiment
Process:
- Identify problem → analyze user data → make hypothesis → conduct test → analyze data
- Exploratory testing: Testers explore the software without predefined test cases, using their intuition and experience to uncover defects.
- UI testing — ensuring that the software uses a consistent user experience and no visual or graphical elements on the screen are broken.
- Data quality testing and database integrity testing — are two closely related types of testing aimed at examining such data characteristics as completeness, consistency, integrity, structuredness, etc
- Performance testing — this assesses how well the software performs under various conditions, such as load, stress, and scalability, to ensure it can handle expected workloads.
- Security testing — this checks the software for vulnerabilities and weaknesses to ensure that sensitive data and system integrity are not compromised.
- Compatibility testing — ensures that the software works correctly across different devices, operating systems, browsers, and network environments.
- Configuration testing
- Cross-browser testing
- Internationalization testing (i18n testing, globalization185 testing, localizability, testing) is a testing aimed to ensure that the product is ready to work in different languages and with different national and cultural characteristics.
- Accessibility testing is a testing aimed at investigating the suitability of a product to be used by people with disabilities
- Usability testing focuses on evaluating the software's user interface and overall user experience to ensure it is intuitive and user-friendly.
References
