Dredd API Document Testing with python hook

To validate the API implementation as documented in the API specification document. Dredd is use for verifying the conformance of responses per the api document — yaml file. Dredd cannot be used for functional testing.

Installation and Setup

URL: https://dredd.org/en/latest/index.html

sudo npm install dredd — global

pip install dredd_hooks

Testing Phases

Before any testing can be done, the API document needs to pass validation. Use the following tools to validate the document.

API document validation

In this phase, we want to focus on ensuring the API document conforms to the specification that it was intended for. For Dredd usage, the documentation will also need to pass Dredd parser before it can be used. It is likely that if the documentation passes specification validation, then it Dredd will have no issue with parsing the document.

Specification validation

The API specification file will likely be available prior to implementation. We should be able to take this file to test its conformance to the specification listed in the file.

Tools use: — http://editor.swagger.io/ and Linter for OpenAPI Document

• http://editor.swagger.io: You can import the API document and see specification errors after it was imported.

Structural error

• Common errors:
• Semantic error at paths: this error refers to the referenced schema name having a space. This error can be ignored since fixing it will break Dredd.
• Structural error at responses: this error refers to missing description field or property in the referenced schema or responses. This error MUST be fixed in order to pass Dredd parsing.
• Linter: Tool to validate the OpenAPI documents.
• URL: https://github.com/IBM/openapi-validator/blob/main/README.md

1. npm install ibm-openapi-validator 2
2. lint-openapi <spec-file.yaml>

• Dredd: Dredd will only execute tests once the document passes Dredd parsing. To perform Dredd parsing:
• dredd <spec-file.yaml> anyhost
• Common error is missing example or default values.

If there are errors in the outputs, open a ticket and have Dev team fix the document.

Evaluate warnings generated by lint-openapi tool and open a ticket accordingly.

Implementation

Get List of Transactions

Before any work can be done, we need to get a list of all transactions from the API Specification Document. Run the command below:

dredd <API Spec> any — names | grep info

Get Umer Farooq’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

See output:

info: /planets > List all planets > 200 > application/json

info: /planets/{planetId} > Fetch one planet by position > 200 > application/json

Dredd Limitations

Due to Dredd’s limited support for OpenAPI 3.x, there are currently limitations and false negatives reported by Dredd we need to consider. Below are the list of known issues encountered and the workaround for each issue.

1. Redirection: Dredd does not support redirect. If you encounter 308 status during your execution, then a discussion with DevOps is necessary to remove redirection before further Dredd development can be done.
2. Non-required properties are treated as required: per OpenApi specification, all properties are treated as optional unless it is specified as required. Dredd will sometimes report the missing properties in a response as failure even when it is not defined to be required. The work around is to fix it in the before_validation hook such as the case for file-ingestion error response.

• Error response specification: only code is required
• Handling of default response.
• Example of default response in API document

default:

description: default error response

content:

application/json:

schema:

$ref: ‘#/components/schemas/ErrorResponse’

• Issue: If your API spec document has default listed in the responses, then Dredd will interpret it as the response code 200, but it expects an ErrorResponse based on the specification for default response.
• Workaround implemented in hook files for now.

1. Add the transaction to the stash variable in after_each hook

@hooks.after_each

def my_after_each(transaction):

print(‘after_each’)

# add rc 200 to stash to be skipped only if transaction is 2xx

if re.search(r’2\d\d’, transaction[‘name’]):

stash[re.sub(r’2\d\d’, ‘200’, transaction[‘name’])] = 1

2. Check to ensure the transaction had not been added to stash variable. If it had, then set the transaction to skipp

@hooks.before_each

def before_each(transaction):

# this is to ensure a transaction is executed exactly once

if transaction[‘name’] in stash:

transaction[‘skip’] = True

return