README

Copy Markdown

CI License Version Docs Read the blog post

[!IMPORTANT] This project is still in alpha/experimental stage. Bug reports and contributions are more than welcome

Cuerdo

Transform Arazzo documents into executable property-based tests.

Running an Arazzo Document

Why Cuerdo?

APIs are usually tested with a handful of "happy path" examples. Over time, bugs are discovered and added to the test suite manually. This process is slow, and many bugs reach production, impacting users.

Cuerdo automatically generates hundreds or thousands of test cases from you Arazzo workflows with automatically generated inputs, validating requests and responses against OpenAPI schemas and discovering edge cases that are commonly missed by example-based testing.

Features

  • Generates hundreds or thousands of test cases for each Arazzo workflow, exploring edge cases.
  • Validates every request and response against your OpenAPI schemas.
  • Generates reproducible failures and, when a bug is discovered, shrinks towards minimal inputs/payload.
  • Compatible with latest Arazzo 1.1 specification.
  • Supports custom input generation when domain-specific constraints cannot be expressed in JSON Schema.
  • Available as command-line tool, Docker image and Elixir library.

Quick Start

Download the executable from the releases page (no Elixir/Erlang required), and run it as

./cuerdo_linux_amd64 path/to/arazzo.yaml

Dockerfile

Run cuerdo from a Docker image. Keep in mind you need to mount the local files.

docker run --rm -v "$PWD:/documents" -w /documents igsomething/cuerdo test/support/arazzo.yaml

From Elixir

Add :cuerdo to the list of dependencies in mix.exs

def deps do
  [
    {:cuerdo, "~> 0.5"}
  ]
end

Execute workflows with automatically generated input as part of your test suite

defmodule MyTest do
  use Cuerdo.ArazzoCase

  arazzo_document_test document: YamlElixir.read_from_file!("arazzo.yaml")
end

Or execute a workflow directly

iex> inputs = %{"email" => "user@example.com", "password" => "securePassword"}
iex> document = YamlElixir.read_from_file!("arazzo.yaml")
iex> {:ok, context} = Cuerdo.Arazzo.run_workflow(inputs, "createUserWorkflow", document)
iex> Cuerdo.Arazzo.Context.workflow_outputs(context, "createUserWorkflow")
%{"token" => "userSessionToken"}

For more in-depth information and guides refer to any of the useful links

Contributions are welcome, please read Contributing before creating any issue or pull request.

Unsupported features and limitations

AsyncAPI

All steps and workflows are assumed to execute synchronously and in the order they are defined. AsyncAPI features and fields are unsupported as there is no current way of validating that a message was published to an out-of-band broker or queue

Workflow

  • dependsOn: The field is ignored. If a workflow depends on another workflow then it should define a step that references the dependency.
  • successActions and failureActions are ignored.

Step

  • channelPath, correlationId, action: Used exclusively by AsyncAPI.
  • onSuccess and onFailure: Same as successActions and failureActions from Workflow
  • in: "cookie"
  • dependsOn: Same as Workflow

Condition and Expression

  • xpath and any XML functionality is unsupported
  • type allows only strings. This means that JSONPath supports RFC-9535 version only, and JSON Pointer (RFC-6901). Non-standard and legacy JSONPath and JSONPointer versions are unsupported.