Convenient tools for using Swagger to define and validate your interfaces in a Pyramid webapp.

Fixes #81. This commit delegates swagger validation to py package swagger_spec_validator. This commit only covers the validation part of swagger 1.2/2.0 spec and in no way handles the serving of 2.0 compatible endpoint.

It currently breaks py33 and py34 tests. This should be rectified with the merge of Yelp/swagger_spec_validator#11.

Another change made is regarding the swagger spec file convention between 1.2 and 2.0. Validation check first tries to find Swagger 2.0 spec (which should be just a single file, named, swagger.json) otherwise it tries to find Swagger 1.2 spec. swagger_spec_validator reads the spec again to decide whether the spec is a 1.2 or a 2.0. I am checking these here because, convention wise, api_docs.json is not a correct name for a 2.0 swagger schema. More such changes would need to be made when pyramid_swagger handles 2.0 schema completely.

• This takes the swagger2.0 feature branch and merges it into master
• The last four commits are the only additional changes
• Major version bump to 2.0.0-rc1

I have been getting "Could not find a matching Swagger operation for GET request api/v1/users" from one of my ... experiments.

After digging in to it a bit, I found that the problem was either that route_info['route'].path in tween.py:get_op_for_request() unexpectedly does not start with a "/", or that the map used in the Spec from bravado_core unexpectedly does start the requests with a "/".

This is with Python 3.5.0, pyramid 1.5.7, pyramid-swagger 2.2.2, bravado-core 3.1.1

Adds support for validating arguments of "paramType": "form". Fixes #53.

Looking for some feedback to make sure I'm doing it right and not missing anything.

This is a rough first cut for routing request handling through a tween to bravado-core.

Notes:

• Swagger 1.2 is not in the code path at all.
• tween20.py created to isolate Swagger 2.0 support for the time being
• Depends on https://github.com/Yelp/swagger-py/pull/140 being merged
• Manually tested using example_happyhour and works wonderfully
• Couldn't get dependency_links working in setup.py so unit tests still a WIP

Will re-submit when a swagger2.0 branch is created upstream.

This adds a new config key to pyramid_swagger and passes it on to bravado-core. When use_models is set to False, models are not generated from the incoming request. This retains compatibility with existing Swagger 1.2 views where models are represented as dicts.

I believe this resolves #73

2b092b1 flatten out the call-graph in ingest.py. While I was trying to grok this module I noticed that the call graph was pretty deep, but a lot of parameters weren't being used for anything except being passed down to the next function. With a couple small changes I was able to get everything called directly from compile_swagger_schema() which is the entrypoint to this module. My feeling is that this makes it easier to understand because from a single function (compile_swagger_schema()) you can get a sense of the whole module. The call graph is never really more than depth 2. It also simplies the return value of the functions to single values instead of having to return complex tuples from many of them.

0a44a58 was just a sed. register_swagger_endpoints made me think of #3. In this case "swagger" should be pretty obvious since the package is already called pyramid_swagger, so I thought register_api_doc_endpoints was more accurate.

c259561 addresses the first point of #73. Adds an option to completely disable /api-docs registration, leaving it up to the user. Docs updated.

4c624c3 starts to get more involved in trying to address the second point of #73. I noticed that compile_swagger_schema() (and validate_swagger_schema() under that) were being called from two places. I guess this is because pyramid_swagger has two primary functions: serving api docs, and the tween for request/response validation. Each was calling compile/validate. It seemed like instead of having them both do the work, we could move that work to be a the first step of including this package. The compile and validate can be done up-front, and the artifact ( the SwaggerSchema) just gets added to the settings. I think we could also make an interface and have this registered to the registry redirectly (let me know if that feels like a better approach, I wasn't exactly sure what the function was to do that).

This change supports the second point in #73 by allowing a user to disable this compile_swagger_schema() step entirely. This would allow them to construct their own SwaggerSchema in whatever method they want, add it to the settings (or registry) and everything else in the package should work the same way.

This change still needs some documentation about the SwaggerSchema interface, and how you would build one yourself (and possibly more tests), but I wanted to get it out and get some feedback on it before spending more time on it.

• Swagger 1.2 is not in the code path at all.
• tween20.py created to isolate Swagger 2.0 support
• Manually tested using example_happyhour and works wonderfully
• using requirements.txt for deps until packaged on pypi

With the last release was added the dereference_served_schema parameter. As described by the documentation the parameter is not working well in case of recursive model definition.

The pull request is trying to fix this issue.

The idea is to create a big self-containing swagger specification which could contain $ref tags. Having internal references allows the generation of recursive models and without preventing the generation of a unique swagger specification file

Ping: @striglia, @lucagiovagnoli, @sjaensch

Using the schema below, if I try to view /swagger.json, I get a RuntimeError from excessive recursion. This does not occur in 2.2.3. Last few lines of the stack trace:

  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 113, in resolve_refs
    return resolve_ref(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 104, in resolve_ref
    return resolve_refs(spec, spec_dict)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 115, in resolve_refs
    new_dict[key] = resolve_refs(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 115, in resolve_refs
    new_dict[key] = resolve_refs(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 115, in resolve_refs
    new_dict[key] = resolve_refs(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 113, in resolve_refs
    return resolve_ref(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 104, in resolve_ref
    return resolve_refs(spec, spec_dict)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 115, in resolve_refs
    new_dict[key] = resolve_refs(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 115, in resolve_refs
    new_dict[key] = resolve_refs(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 115, in resolve_refs
    new_dict[key] = resolve_refs(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 113, in resolve_refs
    return resolve_ref(spec, subval)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/pyramid_swagger/api.py", line 101, in resolve_ref
    with spec.resolver.resolving(url):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/jsonschema/validators.py", line 327, in resolving
    url, resolved = self.resolve(ref)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/jsonschema/validators.py", line 335, in resolve
    url = self._urljoin_cache(self.resolution_scope, ref)
  File "/Users/bwarfield/.virtualenvs/popeye/lib/python2.7/site-packages/functools32/functools32.py", line 394, in wrapper
    result = cache[key]
RuntimeError: maximum recursion depth exceeded in cmp

Repro case schema:

   {
    "swagger": "2.0",
    "info": {
        "title": "Bug demo",
        "description": "Recursive object definitions cause problems", 
        "version": "0.1.0"
    },
    "definitions": {
        "widget": {
            "description": "A widget which may have multiple generations of child widgets",
            "properties": {
                "name": {
                    "type": "string",
                    "minLength": 1,
                    "maxLength": 50
                },
                "children": {
                    "type": "array",
                    "items": {
                        "$ref":"#/definitions/widget"
                    }
                }
            }
        }
    },
    "paths": {
        "/resources/widget/{code}" : {
            "parameters" : [
                {
                    "name": "code",
                    "description": "The CODE value for a widget",
                    "in": "path",
                    "type": "string",
                    "required": true
                }
            ],
            "get": {
                "summary": "View a single widget",
                "produces": ["application/json"],
                "parameters": [],
                "responses": {
                    "200": {
                        "description":"Widget local content (without parents or descendents)",
                        "schema": {"$ref":"#/definitions/widget"}
                    },
                    "404": {"description": "The widget does not exist"}
                }
            }
        }
    }    
}

Bonjour, j'ai essayé de créer un projet pyramid en utilisant le tuto suivant: http://docs.pylonsproject.org/docs/pyramid/en/latest/narr/project.html Puis je voulais intégrer le pyramid_swagger en utilisant le lien suivant http://pyramid-swagger.readthedocs.org/en/latest/configuration.html

j'ai rien obtenu que des erreurs qui finissent pas

When I use a .ini file for configuration, some variables are passed to bravado-core in the form of str and always treated as True. Since all variables except formats are boolean, they should be converted using asbool(): https://bravado-core.readthedocs.io/en/stable/config.html

The same thing happens for enable_api_doc_views in __init__.py, and thus I fixed.

Performing response validation is important to ensure that the provided responses are "good", but validating responses has impact on the overall response time. According to the profiling tests on bravado-core validating response is responsible for ~2/3 of the response unmarshaling time.

Name (time in ms)                                      Min                   Max                  Mean             StdDev                Median                IQR            Outliers       OPS            Rounds  Iterations
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
large_objects[not_validate-full-deref-100]          6.9844 (1.0)         26.9779 (1.0)          7.2673 (1.0)       1.7472 (4.23)         7.0800 (1.0)       0.0764 (1.0)           1;8  137.6017 (1.0)         131           1
large_objects[not_validate-with-refs-100]          10.7621 (1.54)        31.4821 (1.17)        11.2619 (1.55)      2.3074 (5.59)        10.9386 (1.55)      0.0955 (1.25)          2;7   88.7950 (0.65)         86           1
large_objects[validate-full-deref-100]             20.2593 (2.90)        40.8244 (1.51)        20.9659 (2.88)      2.9113 (7.05)        20.4715 (2.89)      0.2009 (2.63)          1;6   47.6965 (0.35)         49           1
large_objects[validate-with-refs-100]              28.0228 (4.01)        30.2325 (1.12)        28.4346 (3.91)      0.4131 (1.0)         28.3672 (4.01)      0.2481 (3.25)          2;2   35.1684 (0.26)         36           1
large_objects[not_validate-full-deref-1000]        71.5863 (10.25)       94.4015 (3.50)        75.5684 (10.40)     7.5661 (18.32)       72.4621 (10.23)     1.2367 (16.18)         2;3   13.2330 (0.10)         15           1
large_objects[not_validate-with-refs-1000]        109.7313 (15.71)      131.4665 (4.87)       113.2843 (15.59)     6.9370 (16.79)      110.8814 (15.66)     1.3490 (17.65)         2;2    8.8273 (0.06)         15           1
large_objects[validate-full-deref-1000]           201.3343 (28.83)      224.4196 (8.32)       206.3566 (28.40)     7.4360 (18.00)      203.7418 (28.78)     2.3413 (30.63)         2;2    4.8460 (0.04)         15           1
large_objects[validate-with-refs-1000]            280.9136 (40.22)      304.3246 (11.28)      284.2836 (39.12)     5.8347 (14.12)      282.3307 (39.88)     3.0268 (39.60)         1;1    3.5176 (0.03)         15           1
large_objects[not_validate-full-deref-5000]       366.9331 (52.54)      398.9429 (14.79)      383.4994 (52.77)    14.7368 (35.68)      391.4602 (55.29)    28.8145 (377.00)        9;0    2.6076 (0.02)         15           1
large_objects[not_validate-with-refs-5000]        556.1794 (79.63)      591.9109 (21.94)      573.9847 (78.98)    14.5869 (35.31)      581.6492 (82.15)    26.4046 (345.47)        6;0    1.7422 (0.01)         15           1
large_objects[validate-full-deref-5000]         1,021.2229 (146.22)   1,063.9244 (39.44)    1,038.9285 (142.96)   14.9543 (36.20)    1,044.3556 (147.51)   26.0122 (340.33)        5;0    0.9625 (0.01)         15           1
large_objects[validate-with-refs-5000]          1,411.4028 (202.08)   1,458.9073 (54.08)    1,433.4528 (197.25)   15.1330 (36.63)    1,433.0088 (202.40)   25.3398 (331.54)        4;0    0.6976 (0.01)         15           1
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Raw data in https://travis-ci.org/Yelp/bravado-core/jobs/536472510#L574 and https://termbin.com/6wyi

Currently pyramid-swagger does allow an "on/off" toggle for response validation. I'm wondering if we might have response validation performed on a percentage of traffic, such that we might have some confidence on the quality of the responses without impacting all the requests.

It would be nice to have a new configuration that might look like

{
    '<pyramid_route_name>': route_validation_percentage,  # value between 0 and 1
    None: general_validation_percentage,  # value between 0 and 1 # If the value is missing it is assumed to be 1
}

The idea is to have:

• pyramid_swagger.enable_response_validation: enabling or disabling response validation
• if response validation is enabled then if:
  • the new configuration map is defined then response validation will respect the defined percentages
  • the new configuration map is not defined then all the responses are validated

Before moving on with PR-ing the idea I would like to have some input. (ie. Does the idea make sense? Do the configurations be eventually be easily understandable?)

cc: @sjaensch , @striglia

This would be a breaking change so something for major release.

Basicly:

instead of doing something like:

pyramid_swagger.schema_directory = schemas/live/here

We would do:

pyramid_swagger.schema_location = my_app.module

This would be more in line with what pyramid uses, besides for applications that are installed via pip providing schema directory would be problematic, my approach will work correctly if you are doing something like setup.py develop or if you have pip install my_app.

To get actual absolute file location from schema definitio one could do then:

schema_dir = pkg_resources.resource_filename('myapp', 'module')

pkg_resources also has tools for directly fetching the contents of schema as string if we want that.

Hi, per previous discussion I've created a PR that provides /api-explorer URL that serves Swagger UI using locally served spec files. Right now it uses CDN to serve most of static files to keep pyramid_swagger as small as possible, however template code is ready for local static file serving if needed (I used that first before I got the idea to use CDN).

https://github.com/striglia/pyramid_swagger/pull/226 PR adding the functionality.

Previously, setting pyramid_swagger.enable_api_doc_views = false in a configuration .ini file would not disable the api_doc views, because the string 'false' is truthy.

• Convert the string-value of settings.get('pyramid_swagger.enable_api_doc_views', True) to a bool using pyramid.settings.asbool . (https://docs.pylonsproject.org/projects/pyramid/en/latest/api/settings.html)