A modern API testing tool for web applications built with Open API and GraphQL specifications.

Overview

Schemathesis

Build codecov.io status for master branch Version Python versions Documentation Status Gitter License

Schemathesis is a modern API testing tool for web applications built with Open API and GraphQL specifications.

It reads the application schema and generates test cases, which will ensure that your application is compliant with its schema.

The application under test could be written in any language; the only thing you need is a valid API schema in a supported format.

Simple to use and yet powerful to uncover hard-to-find errors thanks to the property-based testing approach backed by state-of-the-art Hypothesis library.

πŸ“£ Please fill out our quick survey so that we can learn how satisfied you are with Schemathesis, and what improvements we should make. Thank you!

Features

  • Content-Type, schema, and status code conformance checks for Open API;
  • Testing of explicit examples from the input schema;
  • Stateful testing via Open API links;
  • Concurrent test execution;
  • Targeted testing;
  • Storing and replaying network requests;
  • Built-in ASGI / WSGI application support;
  • Ready-to-go Docker image;
  • Configurable with user-defined checks, string formats, hooks, and targets.

Installation

To install Schemathesis via pip run the following command:

pip install schemathesis

Usage

You can use Schemathesis in the command line:

schemathesis run --stateful=links --checks all http://0.0.0.0:8081/schema.yaml

https://raw.githubusercontent.com/schemathesis/schemathesis/master/img/schemathesis.gif

Or in your Python tests:

import schemathesis

schema = schemathesis.from_uri("http://0.0.0.0:8081/schema.yaml")


@schema.parametrize()
def test_api(case):
    case.call_and_validate()

CLI is simple to use and requires no coding; the in-code approach gives more flexibility.

Both examples above will run hundreds of requests against the API under test and report all found failures and inconsistencies along with instructions to reproduce them.

πŸ’‘ See a complete working example project in the /example directory. πŸ’‘

Contributing

Any contribution to development, testing, or any other area is highly appreciated and useful to the project. For guidance on how to contribute to Schemathesis, see the contributing guidelines.

Support this project

Hi, my name is Dmitry! I started this project during my work at Kiwi.com. I am grateful to them for all the support they provided to this project during its early days and for the opportunity to evolve Schemathesis independently.

In order to grow the community of contributors and users, and allow me to devote more time to this project, please donate today.

Also, I occasionally write posts about Schemathesis in my blog and offer consulting services for businesses.

Commercial support

If you are interested in the effective integration of Schemathesis to your private project, you can schedule an appointment and I will help you do that.

Links

Additional content:

License

The code in this project is licensed under MIT license. By contributing to Schemathesis, you agree that your contributions will be licensed under its MIT license.

Comments
  • Store requests & responses

    Store requests & responses

    Closes #379

    • [x] Add command to cassette
    • [x] Documentation
    • [x] Reuse prepared request to avoid double computation
    • [x] CLI output about saved cassette file
    opened by Stranger6667 26
  • Add hooks to modify request before request is sent

    Add hooks to modify request before request is sent

    Description

    Schemathesis currently has hooks that allow users to extend the set of Schemathesis validation functions. I would like to be able to write hooks that allow me to alter the request before it is sent to the API. This will allow me to test specific behavior of the API.

    Example of one extended validation rule:

    Accept header MUST be used to indicate allowable request methods in a 405 response.
    

    To exercise this validation rule, we need to send a request with an operation that we know is not in the API definition for that endpoint. This idea is mentioned in #108, so this behavior may be included as part of the broader goal to generate invalid data in #65.

    However, some of the rules are likely too specific to be part of the default request generation. For example:

    Requests including Accept header with only formats not supported by the server MUST be
    rejected with a 406 status code and appropriate error response model.
    

    Design

    The validation hooks are simple and easy to use, and I think the request hooks should use the same design with different parameters. I think the should request hooks should take a request object and a case object (or any object that contains the API description for the endpoint associated with the request).

    Questions to Consider

    • Does mutating an initially valid request cause problems?
    • Do the extended validation rules still run when the status code is not in the API definition? 415 response, for example, may not be documented in the OpenAPI description.
    • Which requests do we modify? We only want to modify a subset of the requests. Maybe we generate a valid request for each hook in addition to the default requests. Each additional valid request will be modified by a single hook.
    Priority: Medium Type: Feature 
    opened by barrett-schonefeld 21
  • [FEATURE] Proxy traffic + disable certificate verification

    [FEATURE] Proxy traffic + disable certificate verification

    1: Add --proxy function to proxy traffic thru other tools.

    2: add --no-verify function to allow self signed certificate to proxy https thru tools.

    (Error example: Error: requests.exceptions.SSLError: HTTPSConnectionPool(host='', port=443): Max retries exceeded with url: /v0/docs/openapi.json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)')))

    Priority: Medium Type: Documentation 
    opened by wellenc-lex 20
  • validation of response schema not working for recursive schemas

    validation of response schema not working for recursive schemas

    In response_schema_conformance (in checks.py) schemathesis is validating the response schema after it has tried to resolve it. If the response schema is recursive this doesnt work. It would be better to directly pass the response schema to jsonschema.validate() instead of the resolved one.

    if i replace schema = case.endpoint.schema._get_response_schema(definition) with schema = responses[str(response.status_code)]['content'][content_type] (will fail if response has no content type) the validation works fine.

    since jsonschema.validate() can handle recursive schemas, this might also be a better fix for https://github.com/kiwicom/schemathesis/issues/435, validate the raw schema and then resolve it as much as needed to safe time wasted in unnecessary recursions.

    PS: just wanted to say thank you for schemathesis and not just keep opening new issues. I t has really helped us ;)

    Priority: Medium Type: Bug 
    opened by jeremyschiemann 15
  • Support multiple examples specified in Open API schema

    Support multiple examples specified in Open API schema

    As mentioned in #582 we can generate multiple examples via the examples keyword:

    parameters:
      - in: query
        name: limit
        schema:
          type: integer
          maximum: 50
        examples:       # Multiple examples
          zero:         # Distinct name
            value: 0    # Example value
            summary: A sample limit value  # Optional description
          max: # Distinct name
            value: 50   # Example value
            summary: A sample limit value
    
    Priority: Medium Type: Enhancement 
    opened by Stranger6667 14
  • Validate error for nullable properties since v1.3.0

    Validate error for nullable properties since v1.3.0

    Hi, I have schemathesis==1.3.3 and I have the same issue as #335:

    Details: 
    
    None is not of type 'string'
    
    Failed validating 'type' in schema['items']['properties']['family_name']:
        {'maxLength': 50,
         'nullable': True,
         'title': 'Family Name',
         'type': 'string'}
    
    On instance[0]['family_name']:
        None
    

    I tested older versions, This is working in v1.2.0, but regressed since v1.3.0

    Priority: High Type: Bug 
    opened by grigi 14
  • Falsifying examples are not shown in case of hypothesis errors

    Falsifying examples are not shown in case of hypothesis errors

    If the test run discovers multiple errors and hypothesis.errors.MultipleFailures: Hypothesis found 2 distinct failures. is raised, then all the other output is missing

    Also, it goes for timeout errors, we can display it as "E" in the CLI output and not stop the whole test suite

    Priority: High Type: Bug 
    opened by Stranger6667 14
  • [BUG] startup failure caused by dependencies, requires httpx

    [BUG] startup failure caused by dependencies, requires httpx

    Checklist

    Describe the bug Because the upstream starlette updated the dependencies,the following error occurs ModuleNotFoundError: No module named 'httpx'

    To Reproduce

    https://github.com/HXSecurity/DongTai/actions/runs/3132544194/jobs/5084991907 File "/home/runner/.local/bin/schemathesis", line 5, in from schemathesis.cli import schemathesis File "/home/runner/.local/lib/python3.8/site-packages/schemathesis/init.py", line 7, in from . import auth, fixups, graphql, hooks, runner, serializers, targets File "/home/runner/.local/lib/python3.8/site-packages/schemathesis/graphql.py", line 3, in from .specs.graphql import nodes File "/home/runner/.local/lib/python3.8/site-packages/schemathesis/specs/graphql/init.py", line 1, in from .loaders import from_asgi, from_dict, from_file, from_path, from_url, from_wsgi File "/home/runner/.local/lib/python3.8/site-packages/schemathesis/specs/graphql/loaders.py", line 9, in from starlette.testclient import TestClient as ASGIClient File "/home/runner/.local/lib/python3.8/site-packages/starlette/testclient.py", line 15, in import httpx ModuleNotFoundError: No module named 'httpx'

    Expected behavior It seems that dependencies that are still in version 0.x.y should not be used openly, which means their api is still changing frequently.

    Environment (please complete the following information):

    • OS:Linux
    • Python version: 3.8
    • Schemathesis version: 3.17.2

    Additional context looks related to this https://github.com/encode/starlette/pull/1376

    It looks like this issue is not fully resolved. https://github.com/schemathesis/schemathesis/issues/1417

    Type: Bug Status: Review Needed 
    opened by Bidaya0 13
  • [FEATURE] A way to refresh auth

    [FEATURE] A way to refresh auth

    Is your feature request related to a problem? Please describe. Currently, if the auth token expires during the test run, there is no built-in way to refresh it. It will be nice to have some way to register a function that will refresh auth. And maybe some refresh interval

    Describe the solution you'd like A decorator that will register such a function that will return headers required for authorization

    CLI implementation notes

    Here are my thoughts about implementing this feature.

    It might look like this for the end-user - similar to other hooks:

    import schemathesis
    
    
    @schemathesis.register_auth_provider(refresh_interval=600)  # in seconds
    def auth_provider(auth: AuthStorage):
        response = requests.post(...)
        token = response.json()["access_token"]
        ...  # Set the auth
    

    This decorator may store the function & refresh period in some global variable (it is global for a run anyway) that will be available for the Schemathesis internals.

    Then whenever Schemathesis makes a network call (or WSGI / ASGI calls) we check whether we need to call this function (depending on the time since the last auth function call + some leeway) and call it on some longer-lived storage for auth. E.g. we might need to put the auth into query params or into headers (usually it is the case). This might require some small object that will allow the user to set the auth according to their needs. Then we can properly set the auth in those network calls. It might be something like:

    # Schemathesis internals
    class AuthStorage:
        ...  # attrs for storing query / header values
    
        def set_query(self, name: str, value: str) -> None
            self.query[name] = value
        ...  # similarly for other auth types
    ...
    # End-user code
    
    @schemathesis.register_auth_provider(...)
    def set_auth(auth: AuthStorage):
        ...  # get the token
        auth.set_query("api_token", token)
    
    # Adjust network calls
    def _network_test(case, ..., auth: Optional[AuthStorage]):
        if auth is not None:
            set_auth(case, auth)  # Calls the auth provider function if needed
        ...  # make the call
    

    This storage lifetime is bound to the runner, so it could be initialized somewhere here. Also, I think some parts of the code above surely could be factored out into something more isolated.

    Keep in mind, that it requires synchronization as it could be accessed from multiple threads (e.g. lock over accessing the auth)

    Python test implementation notes

    It might be similar in Python tests. E.g., the end-user registers some auth provider on the schema object or globally:

    schema = ...
    
    # Creates a pytest fixture that provides a single `set_auth` method.
    @schema.register_auth_provider(refresh_interval=600)
    def auth(auth_storage):
        ...
    
    
    @schema.parametrize()
    def test_api(case, auth):
        auth.set_auth(case)  # Does all the stuff under the hood
    
    Priority: High Type: Feature Hacktoberfest Difficulty: Medium 
    opened by Stranger6667 13
  • Response code range not supported

    Response code range not supported

    Thanks for adding check of response code!

    Response codes may contain a range of code in the format XXX https://swagger.io/specification/#responsesObject

    My output

            if errors:
    >           raise AssertionError(*errors)
    E           AssertionError: Received a response with a status code, which is not defined in the schema: 404
    E           
    E           Declared status codes: 4XX, 200
    
    Priority: High Type: Feature 
    opened by MissiaL 13
  • [Questions] Access to data created by the schemathesis

    [Questions] Access to data created by the schemathesis

    Hi, I would like to access the data created by schemathesis. Currently, I run this command to test the API. I see the data logged by the API.

    schemathesis run -D negative http://localhost:9990/openapi.json -E "/cumulative_proba"

    I would love to see the data generated and use it to make my own list of specific cases, where I know the responses. I see some information in the example and Unicode folder under .hypothesis but can't decode them to the raw input values that is sent to API.

    Please let me know, Is there any way to get the data or generate like 100 examples, that I can see the exact values of the data generated.

    thank you so much

    opened by pdubey2018 12
  • [BUG] st --report, generated report on app.schemathesis.io doesn't show errors

    [BUG] st --report, generated report on app.schemathesis.io doesn't show errors

    Checklist

    [x] I checked the FAQ section of the documentation [x] I looked for similar issues in the issue tracker

    Describe the bug I run st run -E /api/invalid https://example.schemathesis.io/openapi.json --report

    It generated report at https://app.schemathesis.io/r/SPJFZ/, but the report doesn't show any errors, despite that this run generated one error:

    ____________________________________________________________ POST /api/invalid _______________
    InvalidSchema: Invalid type name
    
    

    To Reproduce See Describe the bug

    Expected behavior Report at https://app.schemathesis.io/r/SPJFZ/ should diplay the above error.

    Environment (please complete the following information):

    • OS: Windows 10
    • Python version: 3.10.5
    • Schemathesis version: 3.17.5
    • Spec version: Open API 3.0.2
    Type: Bug Status: Review Needed 
    opened by robert-dzikowski 1
  • [FEATURE] CLI option for queries

    [FEATURE] CLI option for queries

    Describe the solution you'd like I would like to have the option of providing a query value in the CLI to make sure all the tests I perform use this specific value. This query option should function similarly to the --header option that Schemathesis currently has.

    Describe alternatives you've considered I have the option of enforcing the values by using an enum instead of an example, and by setting the query to REQUIRED. This however would tell readers of the schema that this value is always used, which is not how the API actually works. I do not want a schema that is just for testing, I want to be able to provide the schema to consumers of the API with the correct context.

    Status: Review Needed Type: Feature 
    opened by hoog1511 1
  • [BUG] starlette 0.21.0 uses httpx testclient

    [BUG] starlette 0.21.0 uses httpx testclient

    Checklist

    Describe the bug starlette==0.21.0 updated to use httpx for the testclient instead of requests.

    In particular starlette.testclient.TestClient now inherits httpx.Client.

    To Reproduce Use Case.call_asgi(...) in a unit test.

    Error:

      File "/mnt/win/work/nightcrawler/tests/schema/lookup/test_txt.py", line 22, in test_txt
        response = case.call_asgi(headers={'Authorization': 'Bearer supersecret'})
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/schemathesis/models.py", line 397, in call_asgi
        return self.call(base_url=base_url, session=client, headers=headers, **kwargs)
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/schemathesis/models.py", line 327, in call
        response = session.request(**data)  # type: ignore
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/starlette/testclient.py", line 448, in request
        return super().request(
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/httpx/_client.py", line 803, in request
        request = self.build_request(
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/httpx/_client.py", line 346, in build_request
        headers = self._merge_headers(headers)
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/httpx/_client.py", line 413, in _merge_headers
        merged_headers.update(headers)
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/httpx/_models.py", line 199, in update
        headers = Headers(headers)
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/httpx/_models.py", line 79, in __init__
        self._list = [
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/httpx/_models.py", line 85, in <listcomp>
        for k, v in headers
    ValueError: too many values to unpack (expected 2)
    Falsifying example: test_txt(
        case=Case(path_parameters={'domain': 'A.ac'}, headers={'Authorization': 'Bearer '}, query={}),
        self=<tests.schema.lookup.test_txt.TestTXTSchema testMethod=test_txt>,
    )
    

    Additional context I forced the headers to be a dict instead of a requests.structures.CaseInsensitiveDict and that furthered the state, but it next up failed with:

      File "/mnt/win/work/nightcrawler/tests/schema/lookup/test_txt.py", line 23, in test_txt
        case.validate_response(response)
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/schemathesis/models.py", line 424, in validate_response
        copied_response = copy_response(response)
      File "/mnt/win/work/nightcrawler/.pyenv/lib/python3.10/site-packages/schemathesis/utils.py", line 279, in copy_response
        response.freeze()
    AttributeError: 'Response' object has no attribute 'freeze'
    Falsifying example: test_txt(
        case=Case(path_parameters={'domain': 'A.ac'}, headers={'Authorization': 'Bearer '}, query={}),
        self=<tests.schema.lookup.test_txt.TestTXTSchema testMethod=test_txt>,
    )
    

    This seems to be caused by now getting a httpx.Response back instead of a requests.Response, and these objects are very different.

    Type: Bug Status: Review Needed 
    opened by grigi 6
  • [BUG] Stateful testing restarts too soon and API is not explored

    [BUG] Stateful testing restarts too soon and API is not explored

    Checklist

    Describe the bug Stateful testing seems a really powerful for automating the API exploration, but it's difficult to make it work. It just stops too soon and filters lot of data. I'll attacch a simple case where i get a subset of the schema, configured with links, and run it.

    I run the test with pytest -s --log-cli-level=INFO --junitxml=report.xml -v test.py so the output is shown as the tests goes, as you can see the machine is restarted a lot of times, and the actual examples wich are tried are just a few between each run.

    Test Output INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:63 PUT /api-portale/api/v1/organizations/{organization_id} INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:63 POST /api-portale/api/v1/contacts INFO root:state-machine-ad-hoc-service.py:63 POST /api-portale/api/v1/contacts INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:55 RESTART

    INFO root:state-machine-ad-hoc-service.py:63 GET /api-portale/api/v1/contacts/{contact_id} INFO root:state-machine-ad-hoc-service.py:63 POST /api-portale/api/v1/organizations INFO root:state-machine-ad-hoc-service.py:63 POST /api-portale/api/v1/contacts INFO root:state-machine-ad-hoc-service.py:63 PUT /api-portale/api/v1/organizations/{organization_id} INFO root:state-machine-ad-hoc-service.py:63 POST /api-portale/api/v1/organizations INFO root:state-machine-ad-hoc-service.py:55 RESTART

    To Reproduce

    I've attacched the openapi yaml, this is the test code, note that rising up the max_examples and stateful_step_count doesn't improve the behaviour.

    import schemathesis
    import pytest
    
    from hypothesis.stateful import initialize, run_state_machine_as_test
    from hypothesis import settings, reproduce_failure, Phase, Verbosity, HealthCheck
    
    from schemathesis.models import Case
    from schemathesis.utils import NOT_SET
    
    import logging
    
    logger = logging.getLogger(__file__)
    logger.setLevel(logging.INFO)
        
    path = "schema-path.yaml"
    base_url = "http://localhost:8080/api-portale/api/v1/"
    
    schema = schemathesis.from_path(
        path,
        base_url = base_url
    )
    
    auth_headers = { 'X-Cart-username': 'gestore' }
    
    
    OrganizationWorkflow = schema.clone(operation_id=['addContact', 'getContact', 'createOrganization', 'updateOrganization']).as_state_machine()
    
    class SimpleWorkflow(OrganizationWorkflow):
    
        def setup(self):
            logging.info("RESTART\n\n")
        #end
    
        def get_call_kwargs(self, case):
            return {"headers": auth_headers }
        #end
    
        def before_call(self, case : Case):  
            logging.info(case.operation.verbose_name)
        #end
    
        def validate_response(self, response, case):
            case.validate_response(response, checks=(schemathesis.checks.status_code_conformance,))
        #end
    
    #end
    
    test_case_1 = SimpleWorkflow.TestCase
    
    

    Expected behavior The full machine gets explored by schemathesis. This is a simple case, but rarely it can follow the link createOrganization -> updateOrganization because with these few tries it cannot build randomly a valid Organization.

    Furthermore, when I raise the max_examples or the stateful_step_count, the tests stops with: hypothesis.errors.FailedHealthCheck: It looks like your strategy is filtering out a lot of data.

    Nothing really changes if I suprress this healthCheck, the state machine still stops too soon.

    openapi.yaml.txt

    Type: Bug Status: Review Needed 
    opened by fscarlato 0
  • [BUG] Schemathesis Can't handle a simple recursive reference:

    [BUG] Schemathesis Can't handle a simple recursive reference:

    Checklist

    I'm still getting: hypothesis.errors.Unsatisfiable: Currently, Schemathesis can't generate data for this operation due to recursive references in the operation definition. See more information in this issue - https://github.com/schemathesis/schemathesis/issues/947

    And the schema is a simple Parent-Child relationship:

    CategoryChildCreate:
      required:
        - category_id
        - category_label
      type: "object"
      properties:
        category_id:
          $ref: "#/components/schemas/CategoryId"
        category_label:
          type: "string"
        description:
          type: string
          maxLength: 255
        image:
          type: "string"
          format: "base64"
        children:
          type: array
          items:
            $ref: "#/components/schemas/CategoryChildCreate"
    

    If this is a recursion level problem, why it isn't solved by defining a maximum depth? Can this maximum depth be specified from the command line?

    Type: Bug Status: Review Needed 
    opened by fscarlato 0
  • [QUESTION] How to authenticate with http-only cookie ?

    [QUESTION] How to authenticate with http-only cookie ?

    Hello,

    I hope you don't mind this question.

    I am using FastAPI, and my authentification consist in a http-only cookie being set. Usually sharing a common TestClient is enough to keep the cookie set between requests. But I can't find a way to share a TestClient for the auth request and the case call request.

    I need a context for the TestClient too (with TestClient(my_app) as session:), as I need to fire the startup and shutdown event for the database connection.

    Is there a way to share a common session between the auth and the case.call ?

    Thanks for your time

    Type: Documentation 
    opened by triplem161 1
Releases(v3.17.5)
  • v3.17.5(Nov 8, 2022)

  • v3.17.4(Oct 19, 2022)

  • v3.17.3(Oct 10, 2022)

  • v3.17.2(Aug 27, 2022)

  • v3.17.1(Aug 19, 2022)

  • v3.17.0(Aug 17, 2022)

    From this release, you can simultaneously run positive & negative tests via CLI by passing -D all

    :rocket: Added

    • Support for exception groups in newer Hypothesis versions. #1592
    • A way to generate negative and positive test cases within the same CLI run via -D all.

    :bug: Fixed

    • Allow creating APIs in Schemathesis.io by name when the schema is passed as a file.
    • Properly trim tracebacks on Hypothesis>=6.54.0.
    • Skipping negative tests when they should not be skipped.

    :wrench: Changed

    • pytest: Generate positive & negative within the same test node.
    • CLI: Warning if there are too many HTTP 403 API responses.
    • Runner: BeforeExecution.data_generation_method and AfterExecution.data_generation_method changed to lists of DataGenerationMethod as the same test may contain data coming from different data generation methods.
    Source code(tar.gz)
    Source code(zip)
  • v3.16.5(Aug 11, 2022)

  • v3.16.4(Aug 8, 2022)

  • v3.16.3(Aug 8, 2022)

    A handful of new usability improvements!

    :rocket: New

    • CLI: --max-failures=N option to exit after first N failures or errors. #1580
    • CLI: --wait-for-schema=N option to automatically retry schema loading for N seconds. #1582
    • CLI: Display old and new payloads in st replay when the -v option is passed. #1584

    :bug: Fixed

    • Internal error on generating negative tests for query parameters with explode: true.
    Source code(tar.gz)
    Source code(zip)
  • v3.16.2(Aug 5, 2022)

    More improvements on user experience!

    From now Schemathesis will warn you if you specify an invalid API location and all responses are 404

    WARNING: All API responses have a 404 status code. Did you specify the proper API location?
    

    :rocket: New

    • after_load_schema hook, which is designed for modifying the loaded API schema before running tests. For example, you can use it to add Open API links to your schema via schema.add_link
    • New utf8_bom fixup. It helps to mitigate JSON decoding errors inside the response_schema_conformance check when the payload contains BOM. #1563
    • CLI: Warning if all responses are 404

    :bug: Fixed

    • docs: Description of -v or --verbosity option for CLI.

    :wrench: Changed

    • Execute before_call / after_call hooks inside the call_* methods. It makes them available for the pytest integration.
    Source code(tar.gz)
    Source code(zip)
  • v3.16.1(Jul 29, 2022)

    A little patch release that improves user experience!

    From now Schemathesis will warn you if you forgot to specify proper API credentials

    WARNING: Most of the responses from `GET /api/secret/` have a 401 status code. Did you specify proper API credentials?
    

    Also, we're extending SaaS test reports a bit! Schemathesis will send us information about what CLI options you use, so we can provide you with suggestions on how you can do more with CLI! Don't worry, all the data is anonymized and there is no sensitive info.

    What's Changed

    • Add SCHEMATHESIS_BASE_URL environment variable for specifying --base-url in CLI
    • Collect anonymized CLI usage data in reports
    • Warning if the API returns too many HTTP 401
    Source code(tar.gz)
    Source code(zip)
  • v3.16.0(Jul 22, 2022)

    Hello friends!

    Schemathesis 3.16.0 brings a brand new GitHub integration and many readability improvements to the test results. You will have much less visual clutter reading Schemathesis reports, and the failures will be much more concise from now.

    :octocat: GitHub Integration

    Now you can use a native GitHub application to collect & display API fuzzing results directly in your PR! It makes Schemathesis much simpler to use in your GitHub Actions workflows and brings more Schemathesis.io features closer to you:

    • API Issues Dashboard: Debug issues faster with all the report details in a UI;
    • Additional checks: Get your API checked more thoroughly by extra conformance checks;

    Grab your installation now!

    After installation, you'll be redirected to SaaS to grab your access token, which you need to add to your workflow.

    Depending on your application deployment, you'll need to add a new step to your GitHub Actions workflow. You can use this code sample to test your API in a pull request or run tests against a publicly resolvable API:

    jobs:
      api-tests:
        runs-on: ubuntu-20.04
        steps:
          - uses: schemathesis/action@v1
            with:
              # API schema location
              schema: 'http://127.0.0.1:5001/openapi.json'
              # Set your token from secrets
              token: ${{ secrets.SCHEMATHESIS_TOKEN }}
    

    You can find other deployment scenarios in our documentation

    Then you'll get your report:

    Schemathesis Report

    Read the full integration guide in our docs.

    A few notes:

    • Report upload size has some limitations now. My guess is that 250kb should be enough for most cases - let me know if I am wrong;
    • Using reporting with public repositories is free. Private ones are available on the paid plan;

    What is next?

    This release doesn't support GitHub organizations. The support will be added soon as well as more features like:

    • Precise API Coverage: Per-keyword coverage achieved by the test suite;
    • Stateful Flows: See how stateful tests are exercising your API endpoints in flow diagrams;
    • Schema fixes: Fix your API schema immediately by applying an autogenerated patch.

    Your feedback is important :heart:

    Please, let us know what you think about the integration in this discussion.

    :bug: Fixed

    • Compatibility with hypothesis >= 6.49 in #1538.
    • Support for Hypothesis' skipping tests when the generate phase is disabled.
    • Generating headers with schemas of array or object types.
    • Do not report optional headers missing.

    :wrench: Changed

    • Pytest: Turn off schema validation by default. Pass validate_schema=True to the schema loader of your choice to keep the old behavior.
    • Code samples: Do not display irrelevant HTTP headers. It affects ones automatically set by requests when test cases go to the application under test.
    • CLI: Display the test environment metadata only if the -v option is present.

    :fire: Removed

    • SaaS: Implicit reporting when the proper credentials were specified. From now on, every report upload requires the explicit --report CLI option.
    • CLI: Text representation of HTTP requests in the CLI output.
    Source code(tar.gz)
    Source code(zip)
  • v3.15.6(Jun 23, 2022)

  • v3.15.5(Jun 21, 2022)

  • v3.15.4(Jun 6, 2022)

    This release makes Schemathesis work on schemas that do not comply with the Open API spec due to YAML specifics (missing quotes around keys that can be parsed as integers, booleans, etc)

    Added

    • Support generating data for Open API request payloads with wildcard media types. #1526

    Changed

    • Mark tests as skipped if there are no explicit examples and --hypothesis-phases=explicit is used. #1323
    • Parse all YAML mapping keys as strings, ignoring the YAML grammar rules. For example, on: true will be parsed as {"on": True} instead of {True: True}. Even though YAML does not restrict keys to strings, in the Open API and JSON Schema context, this restriction is implied because the underlying data model comes from JSON.
    • INTERNAL: Improve flexibility of event serialization.
    • INTERNAL: Store request / response history in SerializedCheck.
    Source code(tar.gz)
    Source code(zip)
  • v3.15.3(May 28, 2022)

    Fixed

    • Deduplication of failures caused by malformed JSON payload. #1518
    • Do not re-raise InvalidArgument exception as InvalidSchema in non-Schemathesis tests. #1514
    Source code(tar.gz)
    Source code(zip)
  • v3.15.2(May 9, 2022)

    This is a small bugfix release :)

    :bug: Fixed

    • Avoid generating negative query samples that result in an empty query.
    • Editable installation via pip.
    Source code(tar.gz)
    Source code(zip)
  • v3.15.1(May 3, 2022)

    This release adds a few improvements that simplify customization and fixes a few minor issues that appear when running in Docker.

    :rocket: Added

    • OpenAPI: Expose APIOperation.get_security_requirements that returns a list of security requirements applied to the API operation
    • Attach originally failed checks to "grouped" exceptions.

    :bug: Fixed

    • Internal error when Schemathesis doesn't have permission to create its hosts.toml file.
    • Do not show an internal Hypothesis warning multiple times when the Hypothesis database directory is not usable.
    • Do not print not relevant Hypothesis reports when run in CI.
    • Invalid verbose_name value in SerializedCase for GraphQL tests.
    Source code(tar.gz)
    Source code(zip)
  • v3.15.0(May 1, 2022)

    Major improvements for GraphQL GraphQL_Logo

    This release introduces a bunch of nice improvements for GraphQL testing - now you can test mutations, extend Schemathesis with custom scalars, and run tests at least twice as fast as before!

    Other things include cleaner pytest output, human-readable cassette payloads, reporting flaky failures, better negative testing, and much more! See below

    :rocket: Added

    • GraphQL: Mutations supports. Schemathesis will generate random mutations by default from now on.
    • GraphQL: Support for registering strategies to generate custom scalars.
    • Custom auth support for schemas created via from_pytest_fixture.

    :wrench: Changed

    • Do not encode payloads in cassettes as base64 by default. This change makes Schemathesis match the default Ruby's VCR behavior and leads to more human-readable cassettes. Use --cassette-preserve-exact-body-bytes to restore the old behavior. #1413
    • Bump hypothesis-graphql to 0.9.0.
    • Avoid simultaneous authentication requests inside auth providers when caching is enabled.
    • Reduce the verbosity of the pytest output. A few internal frames and the "Falsifying example" block are removed from the output.
    • Skip negative tests on API operations that are not possible to negate. #1463
    • Make it possible to generate negative tests if at least one parameter can be negated.
    • Treat flaky errors as failures and display a full report about the failure. #1081
    • Do not duplicate failing explicit examples in the HYPOTHESIS OUTPUT CLI output section. #881

    :bug: Fixed

    • GraphQL: Semantically invalid queries without aliases.
    • GraphQL: Rare crashes on invalid schemas.
    • Internal error inside BaseOpenAPISchema.validate_response on requests>=2.27 when response body contains malformed JSON. #1485
    • schemathesis.from_pytest_fixture: Display each failure if Hypothesis found multiple of them.

    🏎️ Performance

    • GraphQL: Over 2x improvement from internal optimizations.
    Source code(tar.gz)
    Source code(zip)
  • v3.14.2(Apr 21, 2022)

    πŸŽ‰ Schemathesis.io: schema-based API testing as a service. πŸŽ‰

    This release adds support for Schemathesis.io - you can upload your test results and visualize them. Or, you can signup to a Pro account and use our servers to run enhanced tests against your API.

    Feel free to join our Discord - we'd love to hear your feedback!

    :rocket: CLI features

    • Support for auth customization & automatic refreshing. #966
    • Open API link name customization via the name argument to schema.add_link.
    • st as an alias to the schemathesis command line entrypoint.
    • st auth login / st auth logout to authenticate with Schemathesis.io.
    • X-Schemathesis-TestCaseId header to help to distinguish test cases on the application side. #1303
    • Support for comma separated lists in the --checks CLI option. #1373
    • Hypothesis Database configuration for CLI via the --hypothesis-database option. #1326
    • Make the SCHEMA CLI argument accept API slugs from Schemathesis.io.

    :bug: Bug fixes

    • Using @schema.parametrize with test methods on pytest>=7.0.
    • Show the proper Hypothesis configuration in the CLI output. #1445
    • Missing source attribute in the Case.partial_deepcopy implementation. #1429
    • Duplicated failure message from content_type_conformance and response_schema_conformance checks when the checked response has no Content-Type header. #1394
    • Not copied case & response inside Case.validate_response.
    • Ignored pytest.mark decorators when they are applied before schema.parametrize if the schema is created via from_pytest_fixture. #1378

    :wrench: Chores and Improvements

    • Enable Open API links traversal by default. To disable it, use --stateful=none.
    • Do not validate API schema by default. To enable it back, use --validate-schema=true.
    • Add the api_slug CLI argument to upload data to Schemathesis.io.
    • Show response status code on failing checks output in CLI.
    • Improve error message on malformed Open API path templates (like /foo}/). #1372
    • Improve error message on malformed media types that appear in the schema or in response headers. #1382
    • Relax dependencies on pyyaml and click.
    • Add --cassette-path that is going to replace --store-network-log. The old option is deprecated and will be removed in Schemathesis 4.0

    P.S. These release notes contain entries from multiple releases, as they were not properly displayed on GitHub before

    Source code(tar.gz)
    Source code(zip)
  • v3.14.1(Apr 18, 2022)

    What's Changed

    • fix: Compatibility with pytest>=7.0 by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1460

    Full Changelog: https://github.com/schemathesis/schemathesis/compare/v3.14.0...v3.14.1

    Source code(tar.gz)
    Source code(zip)
  • v3.14.0(Apr 17, 2022)

    What's Changed

    • chore: Validate requests kwargs to catch cases when the ASGI integration is used, but the proper ASGI client is not supplied by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1399
    • chore: Add st as an alias to schemathesis by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1400
    • chore: Enable Open API links traversal by default by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1401
    • chore: Do not validate API schema by default by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1402
    • feat: Add a command to authenticate with Schemathesis.io by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1403
    • fix(docs): Bump Sphinx version by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1406
    • chore: api_id CLI argument to upload data to Schemathesis.io by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1404
    • fix: Create auth file even if some parent dir does not exist by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1408
    • chore: Update pre-commit by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1416
    • chore: Deep-clone Case instances before passing to check functions by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1419
    • chore: Deep-clone Response instances before passing to check functions by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1420
    • chore: Pass custom formats to strategy factories by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1421
    • chore: Support for Hypothesis>=6.41.0 by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1426
    • chore: Ping SaaS on login by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1424
    • fix: Missing media_type in the Case.partial_deepcopy implementation. by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1428
    • refactor: Move data generation method choices into a separate variable by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1431
    • chore: Show response status code on failing checks output in CLI by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1433
    • fix: Missing source attribute in the Case.partial_deepcopy implementation by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1432
    • chore: Update CLI auth by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1430
    • feat: Open API link name customization by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1435
    • feat: X-Schemathesis-TestCaseId header by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1434
    • Update builds by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1436
    • fix: Duplicated failure message from content_type_conformance and response_schema_conformance checks by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1437
    • fix: Not copied case & response inside Case.validate_response by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1438
    • chore: Improve error message on malformed Open API path templates by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1440
    • fix: Ignored pytest.mark decorators when they are applied before schema.parametrize if the schema is created via from_pytest_fixture by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1441
    • feat: Support for comma separated lists in the --checks CLI option by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1442
    • Configure Hypothesis db by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1444
    • chore: Improve error messages for malformed media types by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1446
    • chore: Fail fast if Schemathesis.io responds with an error by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1447
    • chore: Add --cassette-path that is going to replace --store-network-log by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1448
    • fix: Compatibility with pytest-asyncio>=0.17.1 by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1453
    • test: Cleanup test code by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1455
    • feat: Make the SCHEMA CLI argument accept API slug from Schemathesis.io by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1456
    • test: Update Flask dependency by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1458
    • chore: Relax dependencies on pyyaml and click by @Stranger6667 in https://github.com/schemathesis/schemathesis/pull/1459

    Full Changelog: https://github.com/schemathesis/schemathesis/compare/v3.13.3...v3.14.0

    Source code(tar.gz)
    Source code(zip)
  • v3.13.9(Apr 14, 2022)

  • v3.13.8(Apr 5, 2022)

    :bug: Bug fixes

    • 72698c0 Missing media_type in the Case.partial_deepcopy implementation.

    :wrench: Chores and Improvements

    • f2be90d Release 3.13.8
    Source code(tar.gz)
    Source code(zip)
  • v3.13.7(Apr 2, 2022)

  • v3.13.6(Mar 31, 2022)

  • v3.13.5(Mar 31, 2022)

    :rocket: Features

    • 4c645b5 Extra requests related options for the replay command
    • cf03191 APIOperation.iter_parameters helper to iterate over all parameters.
    • ebf9061 Support for readOnly and writeOnly Open API keywords
    • a3392a7 Support client-side certificates in CLI
    • 2903305 New before_init_operation hook
    • f136a16 (internal) description attribute for all parsed parameters inside APIOperation.
    • f4576bd Schemathesis.io integration
    • 3a730d0 New process_call_kwargs CLI hook
    • 94ed201 New before_call and after_call CLI hooks.
    • 36829e2 Add ExecutionEvent.is_terminal attribute that indicates whether an event is the last one in the stream
    • b8d4484 A way to stop the Schemathesis runner's event stream manually via events.stop()
    • c76285c GraphQL support in CLI
    • e922209 Case.data_generation_method attribute that provides the information of the underlying data generation method
    • 9217358 Negative testing
    • 1cb291b Additional context for each failure coming from the runner.
    • d3e1766 (runner) Add BeforeExecution.verbose_name & SerializedCase.verbose_name that reflect specification-specific API operation name
    • b78a164 Support for disabling ANSI color escape codes via the NO_COLOR environment variable or the --no-color CLI option
    • 9ac1b75 More GraphQL loaders
    • 36c0c87 Support for data_generation_methods and code_sample_style in all GraphQL loaders
    • 22354a9 ASGI support for GraphQL schemas
    • 0bf11c7 Support for app & base_url arguments for the from_pytest_fixture runner
    • fa0820f from_wsgi loader for GraphQL apps
    • 517d5f2 before_generate_case hook
    • d0d743e Add event_type field to the debug output
    • c61dcee --debug-output-file CLI option to enable storing the underlying runner events in the JSON Lines format in a separate file for debugging purposes
    • 006b8a4 Display failing response payload in the CLI output
    • 4f94584 Add count_operations flag to runner.prepare
    • 23665e2 Support for external examples
    • e291644 Support YAML serialization
    • ea64c29 --dry-run CLI option. When applied, Schemathesis won't send any data to the server and won't perform any response checks
    • 9c13450 A helpful error message when an operation is not found during the direct schema access
    • 7c528f4 Support data generation for text/plain media types with charset
    • e8bd16c Base URL for GraphQL schemas
    • f724df9 Run response_schema_conformance check on media types that are encoded with JSON (#927)

    :bug: Bug fixes

    • 639b7ab (docs) Bump Sphinx version
    • 375f4f6 Use full_path in error messages in recoverable schema-level errors
    • f9ac0e2 Use the same correlation_id in BeforeExecution and AfterExecution events if the API schema contains an error that causes an InvalidSchema exception during test execution.
    • fe23f2e Properly handle error if Open API parameter doesn't have content or schema keywords
    • 5370c80 (example) asyncpg==0.23.0 installs from wheels
    • 3b518b9 (example) fix postgres version to current latest one
    • 4fd42cb (example) mount main.py into container
    • 92805d5 (example) recreate web app on failure
    • 3208712 (example) bind web app ports to 127.0.0.1
    • 89cda52 Generating illegal Unicode surrogates in queries
    • fe51d57 Not-escaped single quotes in generated Python code samples
    • ccb39b7 Improper handling of base_url in call_asgi, when the base URL has a non-empty base path
    • 45a0eb1 Generating values not compliant with the ECMAScript regex syntax
    • 380bcf4 Not raising an error when tests generated by schemas loaded with from_pytest_fixture match no API operations
    • b820385 Silently failing to detect numeric status codes when the schema contains a shared parameters key
    • 24155c9 Ignored hooks defined on a schema instance when it is loaded via from_pytest_fixture
    • 5ffba33 Respect the data_generation_methods config option defined on a schema instance when it is loaded via from_pytest_fixture
    • 589be99 Handle KeyboardInterrupt that happens outside of the main test loop inside the runner
    • 0ad3437 Unescaped quotes in generated Python code samples on some schemas
    • 700af30 Generation of invalid headers in some cases
    • 9577f0f (build) Unpinned Python version specifier in the pytest-legacy job
    • b765e8f Pass data_generation_method to GraphQLCase
    • 120cbcf Unresolvable dependency due to incompatible changes in the new hypothesis-jsonschema release
    • cf63173 (example) Docker setup for example project
    • a80424b (example) Typo in example project's schema
    • df14316 Properly report all failures when custom checks are passed to case.validate_response
    • 54cce04 Stop worker threads on failures with exit_first enabled
    • b1172a1 Rewrite not resolved remote references to local ones
    • 167b8a9 Re-used referenced objects during inlining
    • 471cebd Stop workers on interrupted event (#1239)
    • c8179b9 TypeError on case.call with bytes data on GraphQL schemas
    • 6168c22 Internal error in make_case calls for GraphQL schemas.
    • 0bec4b3 Preserve non-body parameter types in requests during Open API runtime expression evaluation
    • 9395a74 TypeError during negative testing on Open API schemas with parameters that have non-default style value
    • 424dfa2 KeyError when the response_schema_conformance check is executed against responses without schema definition
    • 703430c When EventStream.stop is called, the next event always is the last one
    • 4dbf750 Incorrect deduplication applied to response schema conformance failures that happen to have the same failing validator but different input values
    • cd91522 Compatibility with newer Hypothesis versions
    • f9fa796 Open API style & explode for parameters derived from security definitions
    • 6e72c3f Apply the Open API's style & explode keywords to explicit examples
    • 586ba77 Disable filtering optimization for headers when there are keywords other than type
    • 31d322f Too much filtering in headers that have schemas with the pattern keyword
    • 288bf17 Invalid multipart payload generated for unusual schemas for the multipart/form-data media type
    • 1b8da90 Ignored explicit Content-Type override in Case.as_requests_kwargs
    • bb6fe04 Internal error on unusual schemas for the multipart/form-data media type
    • acd2ac4 Ignored Open API specific keywords & types in schemas with deeply nested references
    • dbd4cfd Ignored $ref keyword in schemas with deeply nested references.
    • 46ec17d Missing support for date string format
    • 5841dda UnicodeEncodeError when sending application/octet-stream payloads that has no format: binary in their schemas
    • 5098a1d Bump minimum hypothesis-graphql version to 0.4.1
    • f8f571b (windows) UnicodeDecodeError during schema loading via the from_path loader if it contains certain Unicode symbols
    • f8d90b9 Using parametrized pytest fixtures with the from_pytest_fixture loader
    • d558063 Return type of make_case for GraphQL schemas
    • dbd6720 Silently ignoring some incorrect usages of @schema.given`
    • 90198ec Missing @schema.given implementation for schemas created via the from_pytest_fixture loader
    • c7cf6d8 (deps) Do not use importlib-metadata==3.8 in dependencies
    • 8d665f7 Encoding for response payloads displayed in the CLI output
    • 5d25d55 Output types for evaluation results of $response.body and $request.body runtime expressions
    • d6eb7ac Missing body parameters during Open API links processing in CLI
    • a13ffce Displaying wrong headers in the FAILURES block of the CLI output
    • d77c9a3 UnicodeDecodeError on cURL command generation
    • 1043c40 Wrong test results in some cases when the tested schema contains a media type that Schemathesis doesn't know how to work with
    • 4df34f9 Percent-encode . and .. strings in path parameters
    • 5355c13 loosen importlib-metadata version constraint
    • d97019d Prevent a small terminal width causing a crash when printing percentage
    • 3224601 Return a default terminal size to prevent crashes on systems with zero-width terminals (some CI/CD servers)
    • 11cb8b9 Docker tags for Buster-based images
    • 079f645 Prevent a TypeError when additional_checks is a list
    • 07c52a5 Programmatic addition of Open API links via add_link
    • 09fa0c2 Generating stateful tests, with common parameters behind a reference
    • c3a787b Flaky Hypothesis error during explicit examples generation
    • 63f4dea Processing parameters common for multiple API operations if they are behind a reference
    • 3c0883c Import in tests
    • 9e74ec6 Generating incomplete explicit examples
    • e1a54c0 CLI execution stops on errors during example generation
    • b8f42df Handling of API operations that contain reserved characters in their paths
    • f1566f7 CLI crash on schemas with operation names longer than the current terminal width
    • 7ab5c86 Invalid types in x-examples
    • 5ca14bf TypeError during nullable array parameter serialization
    • 4774398 Collecting nullable parameters that are behind references
    • 27bce30 Explicit examples for multipart forms
    • d578519 A possibility to override APIStateMachine.step
    • d25efba Unsatisfiable error in stateful testing
    • afa3daa Skip explicit examples generation if this phase is disabled via config
    • a5d3c65 Do not suggest to disable schema validation if it is already disabled
    • 9b93056 Apply body-level hooks
    • fad2c19 Internal error if filling missing explicit examples led to Unsatisfiable errors
    • bce08c4 --exitfirst CLI option trims the progress bar output when a failure occurs
    • b63a578 Use typing instead of typing_extensions
    • 152ccb8 Excessive reference inlining in large schemas
    • 44c6549 CLI failure if the tested operation is GET and has payload examples (#930)
    • 530d5b6 Distinguishing between no payload requests & null payload (#929)
    • 6f6cff0 Allow sending null as request payload if the schema expects it (#922)

    :wrench: Chores and Improvements

    • fbe5e04 Release 3.13.5
    • 542e7a2 Deep-clone Case instances before passing to check functions
    • f4ead9e Release 3.13.4
    • b552584 Support Werkzeug>=2.1.0
    • 7a5a8fe Pin aevea/action-kaniko
    • 7ca318e Validate requests kwargs to catch cases when the ASGI integration is used, but the proper ASGI client is not supplied
    • 2fee032 Release 3.13.3
    • c810f3e Update pre-commit
    • 38a582e Release 3.13.2
    • e933553 Use Schemathesis default User-Agent when communicating with SaaS
    • a84fe70 Release 3.13.1
    • 24442ba Release 3.13.0
    • aebe61a Always show traceback for errors in Schemathesis.io integration
    • 2815f96 Update Schemathesis.io integration
    • 2eaea40 Release 3.12.3
    • a395fcd Disable the duplicate-code pylint lint
    • 57d9c74 Release 3.12.2
    • d146ea6 Release 3.12.1
    • 4e2d446 Release 3.12.0
    • c337784 Update typing-extensions requirement to >=3.7,<5
    • 26d7905 Update pre-commit
    • 23dd6c8 Drop Python 3.6 support
    • e927c51 Release 3.11.7
    • 7a10fbf Support Python 3.10
    • 48ee34c Update dev dependencies
    • 02935ff Upgrade pre-commit
    • ac453f2 Release 3.11.6
    • accb4da Release 3.11.5
    • dded3b1 Generate tests for API operations with the HTTP TRACE method on Open API 2.0
    • d56afa2 Send response.elapsed to Schemathesis.io
    • 0e6f204 Release 3.11.4
    • 7621f40 Minor changes to the Schemathesis.io integration
    • bee03a4 Add AfterExecution.data_generation_method
    • d97a102 Release 3.11.3
    • 19ee5f9 Release 3.11.2
    • 8026971 Use str for subtest context
    • 5250ea4 Raise an error if a test function wrapped with schema.parametrize matches no API operations
    • 8fd0f41 Use verbose_name & data_generation_method parameters to subtest context instead of path & method
    • 69c9e96 (test) Disable deadline for some flaky tests
    • 87ac234 update release notary
    • 9a7c239 Release 3.11.1
    • b980cfa Update click and pyyaml versions, update tests to reflect that click
    • 5a09a03 Release 3.11
    • bd4b9c5 Use case-insensitive dictionary for Case.headers
    • e966976 Bound validators caching during negative testing
    • 94b642d Bump default_language_version for pre-commit
    • b3d0faa Minor changes for Schemathesis.io integration
    • d31b017 GraphQL schema loaders now accept single DataGenerationMethod instances for the data_generation_methods argument
    • f485bcf Add AfterExecution.verbose_name
    • 94ee105 Update the bug report template
    • 56bc46b Fix typing for BeforeExecution.from_operation
    • 11471d8 Store data_generation_method in BeforeExecution
    • 78dd617 Improve reporting of jsonschema errors which are caused by non-string object keys
    • 7e71b00 Update pre-commit
    • b1fc60e Show cURL code samples by default instead of Python
    • 6987f7b Move data generation method to strategy factory mapping to the module level
    • 8d49a88 DataGenerationMethod.all shortcut to get all possible enum variants.
    • c7527af Release 3.10.0
    • 3f04815 Improve error messages when the loaded API schema is not in JSON or YAML
    • 576dba4 Improve error messages for Schemathesis.io
    • 84473eb Add an extra message in case of errors with Schemathesis.io
    • 669755c Show Schemathesis.io test report URL
    • 5780fd2 Open API schema loaders now accept single DataGenerationMethod instances for the data_generation_methods argument
    • 608216f Relax colorama and typing-extensions
    • e14df88 Add tool.poetry.urls section
    • 829fcb9 Reduce Docker images size
    • b6d75cd Add timeouts when loading external schema components or external examples
    • 2418188 Do not use header filtration when not necessary
    • affc9a8 Minor refactoring of reference inlining
    • e75345c Change OpenAPI20CompositeBody.definition type to List[OpenAPI20Parameter]
    • fd86bad Pin werkzeug to >=0.16.0
    • f196825 Remove unused parameter_cls attributes from BaseOpenAPISchema
    • 72a946c Remove duplicated keyword
    • 3b8ee7f Release 3.9.7
    • 7c50df3 Check non-string response status codes when Open API links are collected
    • dd247d2 Release 3.9.6
    • d4560b4 Release 3.9.5
    • 4700a6c Release 3.9.4
    • e16d42a Remove unused imports
    • 5e18c2b Release 3.9.3
    • bbcbd92 Release 3.9.2
    • 9a6cc1d Return response from case.call_and_validate
    • be2f772 Release 3.9.1
    • 7ed2306 Convert all FailureContext class attributes to instance attributes
    • b6b3c59 Add SerializedCase.cookies
    • 01064a7 Add API schema to the Initialized event
    • 6fad51c ExecutionEvent.asdict adds the event_type field which is the event class name
    • 6256be0 Release 3.9.0
    • a9fbccd Add EventStream.finish method
    • 250c20e Avoid ``pytest`warnings when internal Schemathesis classes are in the test module scope
    • 7d8c0be Release 3.8.0
    • 622a1c9 Improve APIOperation.make_case behavior
    • 4d08fd2 Bump hypothesis-jsonschema version
    • 37b1000 Python values of True, False and None are converted to their JSON equivalents when generated for path parameters or query
    • 0b90566 Raise UsageError if schema.parametrize or schema.given are applied to the same function more than once
    • d30e9c0 Release 3.7.8
    • a6112dc (deps) Update dependencies in /example
    • 8411718 Release 3.7.7
    • 0daefff Release 3.7.6
    • 909205d Release 3.7.5
    • 996bc8e Release 3.7.4
    • bf894c8 Make SerializedCase.path_template return path templates as they are in the schema, without base path
    • 6dcce2e Release 3.7.3
    • 04b24b4 Release 3.7.2
    • bd066f0 Add SerializedCase.media_type attribute
    • 02df4dc Release 3.7.1
    • 5593a81 Various internal changes
    • bc3c08b Release 3.7.0
    • 06ecf49 Extend the default test duration deadline to 15 seconds
    • 6dd1586 Set default timeout for network requests to 10 seconds.
    • 7f26e1f Use different exception classes for not_a_server_error and status_code_conformance checks.
    • af7cde3 Release 3.6.11
    • c3339d7 Release 3.6.10
    • d7a0e9e Bump hypothesis-jsonschema version
    • 844655a Explicitly add colorama to dependencies
    • b1d45b8 Fix hypothesis-jsonschema version constraint
    • 86b396b Release 3.6.9
    • ce92e34 Release 3.6.8
    • 0779b97 Relax dependency on starlette to >=0.13,<1
    • 9297654 Release 3.6.7
    • 8ca014c Update pre-commit hooks
    • 59840e1 Call shutdown on all handlers before CLI exit
    • a7d2bf1 Release 3.6.6
    • 667d8b7 Improve error message for failing Hypothesis deadline healthcheck in CLI
    • b8bed68 Release 3.6.5
    • e362292 Generate valid header values for Bearer auth by construction rather than by filtering
    • d414477 (deps) bump urllib3 from 1.25.6 to 1.25.8 in /example
    • 9d9255b Release 3.6.4
    • e85a13d Bump minimum hypothesis-graphql version to 0.5.0
    • aa632df Remove obsolete pylint and mypy tox envs
    • 1050b81 Release 3.6.3
    • 95ec3ef (deps) bump asyncpg from 0.19.0 to 0.21.0 in /example
    • d786d33 Release 3.6.2
    • 6d7ddad Remove unused imports
    • 24212a9 Release 3.6.1
    • 3b10a2b (deps) Bump Sphinx version
    • 60ed847 Release 3.6.0
    • 301dfae Fix tox warning
    • 1f69add Deprecate schemathesis.runner.prepare
    • 04f22e8 Initial support for GraphQL schemas in Schemathesis runner
    • a2d9beb Loaders unification
    • 7586416 Improve error handling on from_path incorrect usage
    • ced53fc Validate schema paths in ASGI / WSGI loaders
    • 08ba562 Validate GraphQL loaders in Schemathesis runner
    • a25b91c Fix invalid import
    • 70754a5 Add blank impl for GraphQLSchema.get_stateful_tests
    • 5307af4 Unify test IDs
    • 3355ce9 Use different DB entries for each API operation in CLI
    • d689d77 Update GraphQL loader
    • e69a2ea Do not show overly verbose raw schemas in Hypothesis output for failed GraphQL tests
    • c46d5b8 Generate separate tests for each field for GraphQL schemas
    • 7d87261 Release 3.5.3
    • 2825d3e (deps) bump pyyaml from 5.1.2 to 5.4 in /example
    • 43419ba Release 3.5.2
    • 93b6800 Prefix worker thread names with schemathesis_
    • 89f059d Release 3.5.1
    • 99013f9 Release 3.5.0
    • 6e17291 Release 3.4.1
    • 2206999 Release 3.4.0
    • f32c681 Do not store absent request/response payloads as empty strings in VCR cassettes
    • 9524c08 (deps) bump jinja2 from 2.10.3 to 2.11.3 in /example
    • 66eb76e Release 3.3.1
    • 2356141 (internal) Make SerializedCheck.example and Check.example not optional
    • 3afd29c Release 3.3.0
    • 46c1162 A way to control which code sample style to use
    • 89384ca Add extra information to events, emitted by the Schemathesis runner.
    • fdeb037 Release 3.2.2
    • f0e0b0a Support Hypothesis 6
    • caa2c9f Release 3.2.1
    • 6db2490 Release 3.2.0
    • 7bae578 Sort keys when serializing to JSON in hash calculation
    • 13810ff Release 3.1.3
    • 069f5cd Release 3.1.2
    • d97ddbe Release 3.1.1
    • 73b51d8 (deps) bump aiohttp from 3.6.2 to 3.7.4 in /example
    • 3c07cac Release 3.1.0
    • 9f3cd95 Support the latest cryptography version in Docker images
    • e54b764 Release 3.0.9
    • 59091d1 Release 3.0.8
    • d6e7c2f Release 3.0.7
    • c32eda0 Release 3.0.6
    • 29ea09c Release 3.0.5
    • 8cee5de Bump mypy to 0.800
    • 6514888 Release 3.0.4
    • 53ab237 Use the same ref resolving approach in get_operation_by_id
    • f436963 Release 3.0.3
    • 67c1a0f Release 3.0.2
    • dcab4bc Release 3.0.1
    • 16daedb Release 3.0.0
    • 1d793e3 Continue testing after schema parsing errors if possible
    • 418b56e Update pre-commit hooks
    • 882d12f Export from_aiohttp in the public API
    • 21072ae Do not export Stateful in the public API
    • de8fca0 Do not export init_default_strategies in the public API
    • 40b1860 Catch extra Hypothesis output
    • 82b3094 A better error message for invalid regex syntax
    • 8401998 Improve error messages for serialization errors (#998)
    • 11b28c8 Fix a typo
    • a61453e Allow the user in docker image to write to /app
    • 079da46 Do not require --base-url for schemas loaded via a file when dry run is enabled
    • 5d9e82e Deduplicate collected errors that don't come from failed checks
    • 441f825 Display all non-check failures in CLI tests
    • a543240 Shorter error messages on invalid schemas in some cases
    • 1c2c91f Allow running before_add_examples hooks even if default examples generation failed
    • ffdcee6 Fix passing the all option to the test server
    • e85d4e0 Use Python 3.9 for Schemathesis's Docker image
    • 000c9c7 Remove deprecated seed-isort-config pre-commit hook
    • bec459e Update pre-commit to v3.4.0
    • f4a7fd7 Disallow using --auth together with --header that sets the Authorization header.
    • 7b7e6ea Enforce path parameters to be required if schema validation is disabled
    • 4f5ecef Use keyword arguments for get_case_strategy call
    • 7266446 Unify the test server

    :package: Other

    • ea4855b Fix grammar
    • 6603099 Update UTM labels in links
    • 909a661 Update image links
    • 21622f5 Add a link to SaaS
    • b0ec4de Update SaaS tutorial
    • b378064 Disable deadline for test_read_only
    • bb03393 Update README
    • bdfb0f1 Update Changelog
    • 536fb28 Update README
    • 8a556ee Increase the value of max_examples to reduce test flakiness
    • 8e2eb7c Install setuptools explicitly
    • ad9a502 Clarify readme
    • 18da638 Fix RST syntax
    • 279e32b Increase max_examples in test_hidden_failure to decrease the number of flaky test runs
    • 169c5b4 Add more Docker usage examples
    • 70e3489 Use the most recent poetry version
    • 3316723 Add suppress_health_check to test_pet to avoid flakiness
    • 2482fda Remove test that relies on old Hypothesis behavior
    • 9adce81 Add links to Deriving Semantics-Aware Fuzzers from Web API Schemas
    • 67577ff Update LICENSE
    • 562d3fa Use longer timeouts for test_app on Windows
    • d9300c0 Update changelog
    • 85e3932 Update changelog
    • ed3762b Add a note about filters being treated as regular expressions
    • 5630bd5 Add missing section to changelog.rst
    • aa6ded7 Update FAQ
    • d377303 Update changelog
    • 239315a Generate tests on demand when multiple workers are used during CLI runs
    • b804378 Add a recipe for per-route timeouts
    • eea6e61 Remove MutationResult.is_success and MutationResult.is_failure helpers
    • e8823d8 Cache split_schema results for Open API schemas during negative testing
    • 63f8323 Cache get_full_path for Open API schemas
    • 905a113 Properly cache rewritten_components
    • c34b332 (oas3) Inline only components/schemas before passing schemas to hypothesis-jsonschema
    • 0f4fff6 Use sets during negative testing data generation
    • b39cc91 Fix RST syntax
    • 722ed8a Avoid copy.deepcopy calls in some cases
    • d8bbc71 Dramatically improve CLI startup performance
    • e0403bd Cache rewritten schema components for Open API schemas
    • 1a07106 Avoid extra resolve_all call in some cases
    • ebaa7e8 Avoid calling deepcopy in some cases
    • 3019796 Fix flaky test
    • 5684c82 Remove obsolete hook mention
    • 4ae7f01 Clarify FAQ entry
    • c274ade Add a new FAQ entry
    • c55fde2 Update Docker docs
    • 60cb07e Add missing changelog entry
    • b0d5177 Adjust changelog
    • ec91777 Document CLI options & add a recipe for HTTP(S) proxy
    • 8c50c3b Clarify communication language
    • 583d344 Disable deadline on flaky test
    • 5ce74cc Use poetry<1.1.9
    • 7530ff0 Minor changelog correction
    • 212ef23 Update Service docs
    • e2c7a5f Add a link to a walkthrough
    • 97b47b5 Add tests for schema mutations
    • 822b084 Add extra tests for is_valid_header
    • 5980df1 Add missing changelog entry
    • e5178f3 Disable some health checks for slow tests
    • d7e90c0 Add more tests
    • 3080028 Fix typo in help message for --header
    • 975fb44 Fix Service tests on Windows
    • 4a217d8 Use result.verbose_name unconditionally in display_subsection
    • b0ca226 Use cls in class methods instead of concrete classes
    • 47813ef Improve the way internal events are handled in CLI
    • 3d5d093 Update GraphQL docs
    • 9fcad07 Fix RST syntax
    • 12b0049 Disable deadline for occasionally slow tests
    • 5e46a7d Simplify examples serialization
    • 00d3796 Improve schemas to avoid unneeded filtering
    • bd0cc79 Reduce the amount of filtering needed to generate valid headers and cookies
    • c5e5770 Clarify scope for the add_case hook
    • bb7a8a3 Remove unused fixtures
    • efd1969 Fix corpus tests
    • 42a2113 Fix insufficient validation in test apps
    • a371c3b Update notes on stateful testing in CLI
    • 88818f6 Disable deadline in flaky tests
    • 4e2114c Avoid flakiness in test_path_parameters_encoding
    • 9382d71 Pin the sphinx_rtd_theme version
    • a2b0de3 Fix argument order in hook examples
    • 9afd280 Fix example for APIStateMachine.get_call_kwargs
    • 0f50b02 (runner) Move fixups installation out of Schemathesis runner
    • 5b9d757 (runner) Convert Hypothesis configuration options earlier
    • 4652ec5 (test) Schema loading tests
    • 6d77911 (test) Restructure test app
    • beca586 (test) Group tests for code samples generation
    • 2281c04 (test) Remove unused imports
    • c607b1b Remove low-level data generation tests
    • 2c5437e Add Effective API schemas testing to the list of additional content
    • 309001b Move schemathesis.loaders to schemathesis.specs.openapi.loaders
    • ec54781 (test) Reuse fixtures in tests for pytest_subtests integration
    • bf60043 use functioning fixup name in example
    • f293c32 Update GraphQL schema in CONTRIBUTING
    • 86830b1 Add the from_wsgi loader to the public API docs
    • 9010191 Minor corrections
    • 92a4ab4 Add comments
    • 64f6985 Update docs
    • 9d695ae Add an internal caching layer for data generation strategies
    • c173aa5 Fix typos & include available fixups
    • c8de514 Add recipes section
    • bb5bfbe Remove unnecessary new lines in changelog
    • e08f625 Remove obsolete mentions of form_data
    • a2f164a Build docker images with buster as base image
    • 7311d1f Document how to run custom checks via the Python API
    • faa5120 Do not use f-strings without interpolation
    • 067d82c Add docstrings
    • 8cea838 Remove not used HookLocation
    • 8459dd9 Add register_check to the API reference
    • 0fbf8e1 Add basic documentation for the public API
    • 08018ef Disable the filter_too_much health check in test_global_body_hook
    • a863007 Disable Hypothesis health checks for test_openapi_links_multiple_threads
    • 63622e7 Use --hypothesis-max-examples in a slow test
    • 711fb32 Split corpus tests into more jobs
    • cf8c4d7 Decrease the maximum number of examples in the CI Hypothesis profile
    • bf063c0 Add manual triggers
    • 2f08618 Ignore thread exceptions in test for keyboard interrupt
    • 6738258 Add missing imports to code examples
    • b0b6ea6 Use blacken-docs
    • a9cb16e Add a CI job to check docs
    • 3394c55 Rename endpoint to operation where necessary
    • a0ec86a Improve processing of API operation definitions
    • 881c35e Add tests on schemas from Open API catalog
    • 5f14f45 Document corner cases for stateful tests
    • b337551 Decrease memory usage
    • 103186d Use compiled versions of Open API spec validators
    • 74ddafe Add more tests on nullable parameters serialization
    • 4258045 Suppress Hypothesis health checks in test_optional_form_parameters
    • 4f0c8e7 Disable derandomization in stateful tests
    • b6cbbc8 Fix the schema_url fixture
    • 3358c83 Fix Hypothesis warnings
    • 0e5705d Rewrite the tests for stateful feature
    • 93c7c95 Fix scheduled jobs
    • 25bb3c4 Fix mutmut test error
    • 8ce5d35 Update FAQ
    • 40858b5 (changelog) Link additional issue
    • 70d48c4 Do not use conditionals for detecting the content of Hypothesis enums
    • a261895 (cli) Show option default values in the CLI help output
    • fbe6e23 (cli) Group CLI parameters help output
    • 7cb6392 Provide additional info in the test server output
    • 71fb6ec Move exit code decision out from the output handler
    • 57a702f Clarify low variance in generated data
    • d9ed57e Fix test_global_body_hook test
    • 036fc49 Add example project
    • 6d60cdc Remove deprecated git checkout HEAD^2 step from the CodeQL job
    • 2e9794d Disable the too_slow health check for some tests
    • 2ca9f50 Add a rare possibility into an assertion inside test_hypothesis_failed_event
    • c37304c Additional input validation for test server
    • 7646027 Improve reliability of the test server
    • 2e28b7e Add a test for interactions recording in ASGI apps
    • 7298a0a Remove not needed condition
    • f510a39 Remove not needed code for GraphQL schemas
    • 7bb50bc Remove deprecated stateful testing integration from the pytest plugin. (#933)
    • c6a9129 Minor refactorings & additional tests
    • 86c52cb Add more tests (#931)
    • 0a9e271 Fix displaying GIF on the PyPI page
    • d2e5168 Add more tests (#926)
    • edf9352 Use assume in some pytester-style tests (#924)
    • faabe7e Rename get_hypothesis_conversions -> get_parameter_serializer
    • 2dcadf3 Open API parameters handling
    Source code(tar.gz)
    Source code(zip)
  • v3.13.4(Mar 29, 2022)

    :bug: Bug fixes

    • 639b7ab (docs) Bump Sphinx version

    :wrench: Chores and Improvements

    • f4ead9e Release 3.13.4
    • b552584 Support Werkzeug>=2.1.0
    • 7a5a8fe Pin aevea/action-kaniko
    • 7ca318e Validate requests kwargs to catch cases when the ASGI integration is used, but the proper ASGI client is not supplied

    :wrench: Chores and Improvements

    • 41ab56e Release 2.8.6
    • fd54e0d Bump hypothesis-jsonschema version
    • baa97df Support for Werkzeug>=2.1.0
    • 9e1fb3d Update graphql-server
    • b1e77fd Update pre-commit

    :package: Other

    • c90a7b8 Fix failing tests
    Source code(tar.gz)
    Source code(zip)
  • v3.13.3(Feb 20, 2022)

    :rocket: Features

    • 4c645b5 Extra requests related options for the replay command

    :wrench: Chores and Improvements

    • 2fee032 Release 3.13.3
    • c810f3e Update pre-commit

    :package: Other

    • ea4855b Fix grammar
    • 6603099 Update UTM labels in links
    • 909a661 Update image links
    Source code(tar.gz)
    Source code(zip)
Owner
Schemathesis.io
A tool that generates and runs test cases for Open API / GraphQL based apps
Schemathesis.io
PacketPy is an open-source solution for stress testing network devices using different testing methods

PacketPy About PacketPy is an open-source solution for stress testing network devices using different testing methods. Currently, there are only two c

null 4 Sep 22, 2022
A framework-agnostic library for testing ASGI web applications

async-asgi-testclient Async ASGI TestClient is a library for testing web applications that implements ASGI specification (version 2 and 3). The motiva

null 122 Nov 22, 2022
A testing system for catching visual regressions in Web applications.

Huxley Watches you browse, takes screenshots, tells you when they change Huxley is a test-like system for catching visual regressions in Web applicati

Facebook Archive 4.1k Nov 30, 2022
pytest plugin for distributed testing and loop-on-failures testing modes.

xdist: pytest distributed testing plugin The pytest-xdist plugin extends pytest with some unique test execution modes: test run parallelization: if yo

pytest-dev 1.1k Dec 30, 2022
PENBUD is penetration testing buddy which helps you in penetration testing by making various important tools interactive.

penbud - Penetration Tester Buddy PENBUD is penetration testing buddy which helps you in penetration testing by making various important tools interac

Himanshu Shukla 15 Feb 1, 2022
pytest_pyramid provides basic fixtures for testing pyramid applications with pytest test suite

pytest_pyramid pytest_pyramid provides basic fixtures for testing pyramid applications with pytest test suite. By default, pytest_pyramid will create

Grzegorz ŚliwiΕ„ski 12 Dec 4, 2022
WEB PENETRATION TESTING TOOL πŸ’₯

N-WEB ADVANCE WEB PENETRATION TESTING TOOL Features ?? Admin Panel Finder Admin Scanner Dork Generator Advance Dork Finder Extract Links No Redirect H

null 56 Dec 23, 2022
AutoExploitSwagger is an automated API security testing exploit tool that can be combined with xray, BurpSuite and other scanners.

AutoExploitSwagger is an automated API security testing exploit tool that can be combined with xray, BurpSuite and other scanners.

null 6 Jan 28, 2022
Language-agnostic HTTP API Testing Tool

Dredd β€” HTTP API Testing Framework Dredd is a language-agnostic command-line tool for validating API description document against backend implementati

Apiary 4k Jan 5, 2023
βœ… Python web automation and testing. πŸš€ Fast, easy, reliable. πŸ’ 

Build fast, reliable, end-to-end tests. SeleniumBase is a Python framework for web automation, end-to-end testing, and more. Tests are run with "pytes

SeleniumBase 3k Jan 4, 2023
frwk_51pwn is an open-sourced remote vulnerability testing and proof-of-concept development framework

frwk_51pwn Legal Disclaimer Usage of frwk_51pwn for attacking targets without prior mutual consent is illegal. frwk_51pwn is for security testing purp

51pwn 4 Apr 24, 2022
Web testing library for Robot Framework

SeleniumLibrary Contents Introduction Keyword Documentation Installation Browser drivers Usage Extending SeleniumLibrary Community Versions History In

Robot Framework 1.2k Jan 3, 2023
hyppo is an open-source software package for multivariate hypothesis testing.

hyppo (HYPothesis Testing in PythOn, pronounced "Hippo") is an open-source software package for multivariate hypothesis testing.

neurodata 137 Dec 18, 2022
Voip Open Linear Testing Suite

VOLTS Voip Open Linear Tester Suite Functional tests for VoIP systems based on voip_patrol and docker 10'000 ft. view System is designed to run simple

Igor Olhovskiy 17 Dec 30, 2022
A command-line tool and Python library and Pytest plugin for automated testing of RESTful APIs, with a simple, concise and flexible YAML-based syntax

1.0 Release See here for details about breaking changes with the upcoming 1.0 release: https://github.com/taverntesting/tavern/issues/495 Easier API t

null 909 Dec 15, 2022
splinter - python test framework for web applications

splinter - python tool for testing web applications splinter is an open source tool for testing web applications using Python. It lets you automate br

Cobra Team 2.6k Dec 27, 2022
splinter - python test framework for web applications

splinter - python tool for testing web applications splinter is an open source tool for testing web applications using Python. It lets you automate br

Cobra Team 2.3k Feb 5, 2021
Scalable user load testing tool written in Python

Locust Locust is an easy to use, scriptable and scalable performance testing tool. You define the behaviour of your users in regular Python code, inst

Locust.io 20.4k Jan 4, 2023
Scalable user load testing tool written in Python

Locust Locust is an easy to use, scriptable and scalable performance testing tool. You define the behaviour of your users in regular Python code, inst

Locust.io 15.3k Feb 8, 2021