Test automation framework

TL;DR

  • Common TAF layers
    • Core layer = test runner, logger, reporter, config layer, utility layer
    • Business layer = UI/API controllers, Page objects, API service contracts
    • Tests level = test data, tests

Test Automation Framework concept

  • Overall, make life easier and lower maintenance costs.
  • A testing framework is a set of guidelines used for creating and designing tests. A framework is designed to help QA team to test more efficiently. These guidelines could include coding standards, test data handling methods, object repositories, storing test results, info on how to access external resources.
  • Benefits:
    • improved test efficiency (doing the things right)
    • lower maintenance costs
    • max test coverage
    • reusability of code
  • Testing is ONLY about spending money, but ensuring quality and reputation. So we need to write tests faster. So what does it give us:
    • TAF increases repeatability of smoke, regression tests
    • saves time for complicated and repeatable tasks
    • adopting CI/CD
    • TAF speeds up data driven testing

TAF types

image
  • Behavior-driven or not? It has extra business layer, which is time consuming. if there is no need to use BDD, and only you understand that, don’t use it, if business people (BA or manual testers) want to create the test cases, and control them, use BDD. If you want your code to be the test cases use BDD. Otherwise, you can use BDD like approach, not using the BDD.
  • If data-driven, but how? Use DDT if you provide many different data and they have the same test steps. So, DDT is helpful to automate lots of test cases with single scenario code.
    • Ex: We have different metadata to search, TAG, AUTHOR, LAW_FIRM, COMPANY, data if different but test steps are the same.

How to start?

  • Make a research, ask what will be tested firstly, what is in priority, why testing particular things, what is in high risk.
  • Think about test data. CREATE NEW USERS EVERY TIME.

TAF Architecture

DOs:

  • Define expectations
  • Choose tools (runner, reporter, logger)
  • Define scope
  • Define approaches (DDT, BDT)
  • Configure test infrastructure
  • Define processes

DON’Ts:

  • Do not try to make the long-term decisions, requirement ofter change.
  • Don’t strict TAF layers
  • Do not concentrate on patterns. Solution: Focus on needs, don’t concentrate on patterns too much!

How to start (one possible way)? Start from 1 test, and build the whole architecture

  • Discover UI → Design page objects
  • Implement page objects (with locators)

Configuration of TAF

image

Layers

  • Why do we need the layers? It is general in software development, layers help us to isolate the code, like make them single responsibility.
  • Architecture basics:
    • Application channel - Selenium
    • Runner and Reporter - PyTest, unittest
    • Logger (logging all requests that were before failing, showing part of code that failed)
    • Data providers - API, txt, xml, csv, fake data
    • Config (concurrency, order of tests, builders (setup, teardown)) - browser type, hosts
    • Additional utils, functions, helpers
    • Tests
  • Common TAF layers
    • Core layer = config layer, utility layer, runner, logger, reporter, extensions
    • Business layer = UI/API controllers, Page objects, API service contracts
    • Tests level = test data, tests
‣
Another example

Why you need architecture?

  • Make the life easier
  • Forecast the changes you will need in future, and don’t cardinally change the architecture of whole project
    • For examples, if in the future you need to run tests in parallel, you need to think how to separate users, so that you test will not fail because of parallel execution. Test may create data in database, or delete, or update.

Core layer

  • Configuration
    • Environment config files
    • Constants
    • Common types (run in QA, DEV, PROD env)
    • Test settings
    • BEST PRACTICES: Use Singleton design pattern
  • Logger - implementation
    • PATTERN: Wrapper or Facade
  • Utilities and extensions - static, can be used by anyone
    • DB connections, data providers
    • Excel utility
  • Initialization helpers
    • WebDriver initialization
    • HTTP client creation
    • BEST PRACTICES: Use Factory design pattern
  • Reporter and Runner
    • Running category of tests, running in parallel, re-running tests after fail, running in command line
    • Base test controller
      • Base initialize/cleanup methods for test case
      • Base initialize/cleanup methods for test runs
      • Initialization of test settings
      • Logger init
      • Reporter init
    • Reporter: which info include (PASS, FAIL, SKIPPED)

Business layer

  • BDD. If you can’t, do BDD like style coding.
  • Business layer is
    • API/UI/Mobile controller layer
    • Page objects/API service contacts
      • ADVICE: if you test API it is good to have a Model object that receives API responses. And use model class in your tests, instead of JSON.
  • Business logic layer:
    • Page objects (web element, components, methods)
    • Steps (POs interaction, components interactions, business logic)
    • Tests (verifies/checks, uses steps)

Test layer

  • Consists of test data and tests (assertions)
  • Test logic should be reliable, clear, flexible, maintainable, stable
image

Best cases in building TAF

  1. API and UI
  2. Have access to DB
  3. Better YAML config
  4. Logger, request inspection
  5. Page objects should use more abstractions and functions
  6. Customize allure
  7. Take care the sacralization
  • Configurations: create config for each component separately. Sometimes we could use different environments.
  • Best practices
    • Code styling - black, pep8
    • Use PyTest fixtures maximum
      • @user() fixture test with authorization
      • Setup and teardown
    • Pytest markers VS Allure markers
      • Allure setup: add markers, add allure epic + feature
    • Better Pydantic
    • Better POM, create common functions to perform actions
    • Use Allure steps
    • Logging to track all requests and responses for given test still with allure steps
    • Test data: TAF: Global caching to speed up tests
    • Take into consideration Cross-Browser testing feature.
  • Setup GitLab CI/CD
    • Run on QA/Pre-PROD/PROD daily smoke?
    • Run regression before releases
  • Tool: Automation tool to compare two versions of APIs.

References

SuperMade with Super