LaunchDarkly REST API
LaunchDarkly DocsSign in
HomeAPI ReferenceSDKsContext Plugin
Experiments

Create experiment

POST
/api/v2/projects/{projectKey}/environments/{environmentKey}/experiments

Create an experiment.

To run this experiment, you'll need to create an iteration and then update the experiment with the startIteration instruction.

Experiment types

Use the type field to specify the experiment type: experiment (default), mab (multi-armed bandit), or holdout.

For multi-armed bandit experiments, set reallocationFrequencyMillis on the iteration to control how often traffic is reallocated across variations.

Results analysis

Use the methodology field to specify the results analysis approach: bayesian (default) or frequentist.

Use the dataSource field to specify the source of metric data: launchdarkly (default), snowflake, or databricks.

Use the analysisConfig field to customize analysis settings such as the Bayesian threshold, significance threshold, or multiple comparison correction method.

To learn more, read Creating experiments.

Authorization

ApiKey read, write
Authorization<token>

In: header

Scope: read, write

Path Parameters

projectKey*string

The project key

Formatstring
environmentKey*string

The environment key

Formatstring

Request Body

application/json

name*string

The experiment name

description?string

The experiment description

maintainerId?string

The ID of the member who maintains this experiment

key*string

The experiment key

iteration*

Details on the construction of the initial iteration

holdoutId?string

The ID of the holdout

tags?array<string>

Tags for the experiment

methodology?string

The results analysis approach.

Value in

  • "bayesian"
  • "frequentist"
  • "export_only"
analysisConfig?

Details of the Analysis Configuration for this experiment

dataSource?string

The source of metric data in order to analyze results. Defaults to "launchdarkly" when not provided.

Value in

  • "launchdarkly"
  • "snowflake"
type?string

The type of experiment.

Value in

  • "experiment"
  • "mab"
  • "holdout"

Response Body

application/json

application/json

application/json

application/json

application/json

application/json

curl -X POST "https://example.com/api/v2/projects/string/environments/string/experiments" \  -H "Content-Type: application/json" \  -d '{    "name": "Example experiment",    "key": "experiment-key-123abc",    "iteration": {      "hypothesis": "Example hypothesis, the new button placement will increase conversion",      "metrics": [        {          "key": "metric-key-123abc"        }      ],      "treatments": [        {          "name": "Treatment 1",          "baseline": true,          "allocationPercent": "10",          "parameters": [            {              "flagKey": "example-flag-for-experiment",              "variationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05"            }          ]        }      ],      "flags": "{\\"example-flag-key\\": { \\"ruleId\\": \\"e432f62b-55f6-49dd-a02f-eb24acf39d05\\", \\"flagConfigVersion\\": 12, \\"notInExperimentVariationId\\": \\"e432f62b-55f6-49dd-a02f-eb24acf39d05\\" }}"    }  }'
{  "_id": "12ab3c45de678910fgh12345",  "key": "experiment-key-123abc",  "name": "Example experiment",  "description": "An example experiment, used in testing",  "_maintainerId": "12ab3c45de678910fgh12345",  "_creationDate": "1654104600000",  "environmentKey": "string",  "methodology": "bayesian",  "dataSource": "snowflake",  "archivedDate": "1654104600000",  "tags": [    "experiment",    "feature"  ],  "_links": {    "parent": {      "href": "/api/v2/projects/my-project/environments/my-environment",      "type": "application/json"    },    "self": {      "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment",      "type": "application/json"    }  },  "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40",  "currentIteration": {    "_id": "12ab3c45de678910fgh12345",    "hypothesis": "The new button placement will increase conversion",    "status": "running",    "createdAt": "1654104600000",    "startedAt": "1655314200000",    "endedAt": "1656610200000",    "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",    "winningReason": "We ran this iteration for two weeks and the winning variation was clear",    "canReshuffleTraffic": true,    "flags": {      "property1": {        "targetingRule": "fallthrough",        "targetingRuleDescription": "Customers who live in Canada",        "targetingRuleClauses": [          null        ],        "flagConfigVersion": 12,        "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05",        "_links": {          "self": {            "href": "/api/v2/flags/my-project/my-flag",            "type": "application/json"          }        }      },      "property2": {        "targetingRule": "fallthrough",        "targetingRuleDescription": "Customers who live in Canada",        "targetingRuleClauses": [          null        ],        "flagConfigVersion": 12,        "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05",        "_links": {          "self": {            "href": "/api/v2/flags/my-project/my-flag",            "type": "application/json"          }        }      }    },    "reallocationFrequencyMillis": 3600000,    "version": 0,    "primaryMetric": {      "key": "metric-key-123abc",      "_versionId": "string",      "name": "My metric",      "kind": "custom",      "isNumeric": true,      "eventKey": "event-key-123abc",      "_links": {        "self": {          "href": "/api/v2/metrics/my-project/my-metric",          "type": "application/json"        }      },      "isGroup": true,      "metrics": [        {          "key": "metric-key-123abc",          "_versionId": "version-id-123abc",          "name": "Example metric",          "kind": "custom",          "isNumeric": true,          "unitAggregationType": "sum",          "analysisType": "mean",          "eventKey": "event-key-123abc",          "analysisUnit": "string",          "_links": {            "self": {              "href": "/api/v2/metrics/my-project/my-metric",              "type": "application/json"            }          },          "nameInGroup": "Step 1",          "randomizationUnits": [            "user"          ],          "analysisUnits": [            "user"          ]        }      ],      "analysisType": "mean",      "analysisUnit": "user"    },    "primarySingleMetric": {      "key": "metric-key-123abc",      "_versionId": "version-id-123abc",      "name": "Example metric",      "kind": "custom",      "isNumeric": true,      "unitAggregationType": "sum",      "analysisType": "mean",      "eventKey": "event-key-123abc",      "analysisUnit": "string",      "_links": {        "self": {          "href": "/api/v2/metrics/my-project/my-metric",          "type": "application/json"        }      }    },    "primaryFunnel": {      "key": "metric-group-key-123abc",      "name": "My metric group",      "kind": "funnel",      "_links": {        "parent": {          "href": "/api/v2/projects/my-project",          "type": "application/json"        },        "self": {          "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",          "type": "application/json"        }      },      "metrics": [        {          "key": "metric-key-123abc",          "_versionId": "version-id-123abc",          "name": "Example metric",          "kind": "custom",          "isNumeric": true,          "unitAggregationType": "sum",          "analysisType": "mean",          "eventKey": "event-key-123abc",          "analysisUnit": "string",          "_links": {            "self": {              "href": "/api/v2/metrics/my-project/my-metric",              "type": "application/json"            }          },          "nameInGroup": "Step 1",          "randomizationUnits": [            "user"          ],          "analysisUnits": [            "user"          ]        }      ]    },    "randomizationUnit": "user",    "attributes": [      "string"    ],    "treatments": [      {        "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",        "name": "Treatment 1",        "allocationPercent": "10",        "baseline": true,        "parameters": [          {            "variationId": "string",            "flagKey": "string"          }        ]      }    ],    "secondaryMetrics": [      {        "key": "metric-key-123abc",        "_versionId": "version-id-123abc",        "name": "Example metric",        "kind": "custom",        "isNumeric": true,        "unitAggregationType": "sum",        "analysisType": "mean",        "eventKey": "event-key-123abc",        "analysisUnit": "string",        "_links": {          "self": {            "href": "/api/v2/metrics/my-project/my-metric",            "type": "application/json"          }        }      }    ],    "metrics": [      {        "key": "metric-key-123abc",        "_versionId": "string",        "name": "My metric",        "kind": "custom",        "isNumeric": true,        "eventKey": "event-key-123abc",        "_links": {          "self": {            "href": "/api/v2/metrics/my-project/my-metric",            "type": "application/json"          }        },        "isGroup": true,        "metrics": [          {            "key": "metric-key-123abc",            "_versionId": "version-id-123abc",            "name": "Example metric",            "kind": "custom",            "isNumeric": true,            "unitAggregationType": "sum",            "analysisType": "mean",            "eventKey": "event-key-123abc",            "analysisUnit": "string",            "_links": {              "self": {                "href": "/api/v2/metrics/my-project/my-metric",                "type": "application/json"              }            },            "nameInGroup": "Step 1",            "randomizationUnits": [              "user"            ],            "analysisUnits": [              "user"            ]          }        ],        "analysisType": "mean",        "analysisUnit": "user"      }    ],    "layerSnapshot": {      "key": "checkout-flow",      "name": "Checkout Flow",      "reservationPercent": 10,      "otherReservationPercent": 70    },    "covariateInfo": {      "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",      "fileName": "covariate.csv",      "createdAt": "1654104600000"    }  },  "type": "experiment",  "_access": {    "denied": [      {        "action": "string",        "reason": {          "resources": [            "proj/*:env/*;qa_*:/flag/*"          ],          "notResources": [            "string"          ],          "actions": [            "*"          ],          "notActions": [            "string"          ],          "effect": "allow",          "role_name": "string"        }      }    ],    "allowed": [      {        "action": "string",        "reason": {          "resources": [            "proj/*:env/*;qa_*:/flag/*"          ],          "notResources": [            "string"          ],          "actions": [            "*"          ],          "notActions": [            "string"          ],          "effect": "allow",          "role_name": "string"        }      }    ]  },  "draftIteration": {    "_id": "12ab3c45de678910fgh12345",    "hypothesis": "The new button placement will increase conversion",    "status": "running",    "createdAt": "1654104600000",    "startedAt": "1655314200000",    "endedAt": "1656610200000",    "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",    "winningReason": "We ran this iteration for two weeks and the winning variation was clear",    "canReshuffleTraffic": true,    "flags": {      "property1": {        "targetingRule": "fallthrough",        "targetingRuleDescription": "Customers who live in Canada",        "targetingRuleClauses": [          null        ],        "flagConfigVersion": 12,        "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05",        "_links": {          "self": {            "href": "/api/v2/flags/my-project/my-flag",            "type": "application/json"          }        }      },      "property2": {        "targetingRule": "fallthrough",        "targetingRuleDescription": "Customers who live in Canada",        "targetingRuleClauses": [          null        ],        "flagConfigVersion": 12,        "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05",        "_links": {          "self": {            "href": "/api/v2/flags/my-project/my-flag",            "type": "application/json"          }        }      }    },    "reallocationFrequencyMillis": 3600000,    "version": 0,    "primaryMetric": {      "key": "metric-key-123abc",      "_versionId": "string",      "name": "My metric",      "kind": "custom",      "isNumeric": true,      "eventKey": "event-key-123abc",      "_links": {        "self": {          "href": "/api/v2/metrics/my-project/my-metric",          "type": "application/json"        }      },      "isGroup": true,      "metrics": [        {          "key": "metric-key-123abc",          "_versionId": "version-id-123abc",          "name": "Example metric",          "kind": "custom",          "isNumeric": true,          "unitAggregationType": "sum",          "analysisType": "mean",          "eventKey": "event-key-123abc",          "analysisUnit": "string",          "_links": {            "self": {              "href": "/api/v2/metrics/my-project/my-metric",              "type": "application/json"            }          },          "nameInGroup": "Step 1",          "randomizationUnits": [            "user"          ],          "analysisUnits": [            "user"          ]        }      ],      "analysisType": "mean",      "analysisUnit": "user"    },    "primarySingleMetric": {      "key": "metric-key-123abc",      "_versionId": "version-id-123abc",      "name": "Example metric",      "kind": "custom",      "isNumeric": true,      "unitAggregationType": "sum",      "analysisType": "mean",      "eventKey": "event-key-123abc",      "analysisUnit": "string",      "_links": {        "self": {          "href": "/api/v2/metrics/my-project/my-metric",          "type": "application/json"        }      }    },    "primaryFunnel": {      "key": "metric-group-key-123abc",      "name": "My metric group",      "kind": "funnel",      "_links": {        "parent": {          "href": "/api/v2/projects/my-project",          "type": "application/json"        },        "self": {          "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",          "type": "application/json"        }      },      "metrics": [        {          "key": "metric-key-123abc",          "_versionId": "version-id-123abc",          "name": "Example metric",          "kind": "custom",          "isNumeric": true,          "unitAggregationType": "sum",          "analysisType": "mean",          "eventKey": "event-key-123abc",          "analysisUnit": "string",          "_links": {            "self": {              "href": "/api/v2/metrics/my-project/my-metric",              "type": "application/json"            }          },          "nameInGroup": "Step 1",          "randomizationUnits": [            "user"          ],          "analysisUnits": [            "user"          ]        }      ]    },    "randomizationUnit": "user",    "attributes": [      "string"    ],    "treatments": [      {        "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",        "name": "Treatment 1",        "allocationPercent": "10",        "baseline": true,        "parameters": [          {            "variationId": "string",            "flagKey": "string"          }        ]      }    ],    "secondaryMetrics": [      {        "key": "metric-key-123abc",        "_versionId": "version-id-123abc",        "name": "Example metric",        "kind": "custom",        "isNumeric": true,        "unitAggregationType": "sum",        "analysisType": "mean",        "eventKey": "event-key-123abc",        "analysisUnit": "string",        "_links": {          "self": {            "href": "/api/v2/metrics/my-project/my-metric",            "type": "application/json"          }        }      }    ],    "metrics": [      {        "key": "metric-key-123abc",        "_versionId": "string",        "name": "My metric",        "kind": "custom",        "isNumeric": true,        "eventKey": "event-key-123abc",        "_links": {          "self": {            "href": "/api/v2/metrics/my-project/my-metric",            "type": "application/json"          }        },        "isGroup": true,        "metrics": [          {            "key": "metric-key-123abc",            "_versionId": "version-id-123abc",            "name": "Example metric",            "kind": "custom",            "isNumeric": true,            "unitAggregationType": "sum",            "analysisType": "mean",            "eventKey": "event-key-123abc",            "analysisUnit": "string",            "_links": {              "self": {                "href": "/api/v2/metrics/my-project/my-metric",                "type": "application/json"              }            },            "nameInGroup": "Step 1",            "randomizationUnits": [              "user"            ],            "analysisUnits": [              "user"            ]          }        ],        "analysisType": "mean",        "analysisUnit": "user"      }    ],    "layerSnapshot": {      "key": "checkout-flow",      "name": "Checkout Flow",      "reservationPercent": 10,      "otherReservationPercent": 70    },    "covariateInfo": {      "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",      "fileName": "covariate.csv",      "createdAt": "1654104600000"    }  },  "previousIterations": [    {      "_id": "12ab3c45de678910fgh12345",      "hypothesis": "The new button placement will increase conversion",      "status": "running",      "createdAt": "1654104600000",      "startedAt": "1655314200000",      "endedAt": "1656610200000",      "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58",      "winningReason": "We ran this iteration for two weeks and the winning variation was clear",      "canReshuffleTraffic": true,      "flags": {        "property1": {          "targetingRule": "fallthrough",          "targetingRuleDescription": "Customers who live in Canada",          "targetingRuleClauses": [            null          ],          "flagConfigVersion": 12,          "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05",          "_links": {            "self": {              "href": "/api/v2/flags/my-project/my-flag",              "type": "application/json"            }          }        },        "property2": {          "targetingRule": "fallthrough",          "targetingRuleDescription": "Customers who live in Canada",          "targetingRuleClauses": [            null          ],          "flagConfigVersion": 12,          "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05",          "_links": {            "self": {              "href": "/api/v2/flags/my-project/my-flag",              "type": "application/json"            }          }        }      },      "reallocationFrequencyMillis": 3600000,      "version": 0,      "primaryMetric": {        "key": "metric-key-123abc",        "_versionId": "string",        "name": "My metric",        "kind": "custom",        "isNumeric": true,        "eventKey": "event-key-123abc",        "_links": {          "self": {            "href": "/api/v2/metrics/my-project/my-metric",            "type": "application/json"          }        },        "isGroup": true,        "metrics": [          {            "key": "metric-key-123abc",            "_versionId": "version-id-123abc",            "name": "Example metric",            "kind": "custom",            "isNumeric": true,            "unitAggregationType": "sum",            "analysisType": "mean",            "eventKey": "event-key-123abc",            "analysisUnit": "string",            "_links": {              "self": {                "href": "/api/v2/metrics/my-project/my-metric",                "type": "application/json"              }            },            "nameInGroup": "Step 1",            "randomizationUnits": [              "user"            ],            "analysisUnits": [              "user"            ]          }        ],        "analysisType": "mean",        "analysisUnit": "user"      },      "primarySingleMetric": {        "key": "metric-key-123abc",        "_versionId": "version-id-123abc",        "name": "Example metric",        "kind": "custom",        "isNumeric": true,        "unitAggregationType": "sum",        "analysisType": "mean",        "eventKey": "event-key-123abc",        "analysisUnit": "string",        "_links": {          "self": {            "href": "/api/v2/metrics/my-project/my-metric",            "type": "application/json"          }        }      },      "primaryFunnel": {        "key": "metric-group-key-123abc",        "name": "My metric group",        "kind": "funnel",        "_links": {          "parent": {            "href": "/api/v2/projects/my-project",            "type": "application/json"          },          "self": {            "href": "/api/v2/projects/my-project/metric-groups/my-metric-group",            "type": "application/json"          }        },        "metrics": [          {            "key": "metric-key-123abc",            "_versionId": "version-id-123abc",            "name": "Example metric",            "kind": "custom",            "isNumeric": true,            "unitAggregationType": "sum",            "analysisType": "mean",            "eventKey": "event-key-123abc",            "analysisUnit": "string",            "_links": {              "self": {                "href": "/api/v2/metrics/my-project/my-metric",                "type": "application/json"              }            },            "nameInGroup": "Step 1",            "randomizationUnits": [              "user"            ],            "analysisUnits": [              "user"            ]          }        ]      },      "randomizationUnit": "user",      "attributes": [        "string"      ],      "treatments": [        {          "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58",          "name": "Treatment 1",          "allocationPercent": "10",          "baseline": true,          "parameters": [            {              "variationId": "string",              "flagKey": "string"            }          ]        }      ],      "secondaryMetrics": [        {          "key": "metric-key-123abc",          "_versionId": "version-id-123abc",          "name": "Example metric",          "kind": "custom",          "isNumeric": true,          "unitAggregationType": "sum",          "analysisType": "mean",          "eventKey": "event-key-123abc",          "analysisUnit": "string",          "_links": {            "self": {              "href": "/api/v2/metrics/my-project/my-metric",              "type": "application/json"            }          }        }      ],      "metrics": [        {          "key": "metric-key-123abc",          "_versionId": "string",          "name": "My metric",          "kind": "custom",          "isNumeric": true,          "eventKey": "event-key-123abc",          "_links": {            "self": {              "href": "/api/v2/metrics/my-project/my-metric",              "type": "application/json"            }          },          "isGroup": true,          "metrics": [            {              "key": "metric-key-123abc",              "_versionId": "version-id-123abc",              "name": "Example metric",              "kind": "custom",              "isNumeric": true,              "unitAggregationType": "sum",              "analysisType": "mean",              "eventKey": "event-key-123abc",              "analysisUnit": "string",              "_links": {                "self": {                  "href": "/api/v2/metrics/my-project/my-metric",                  "type": "application/json"                }              },              "nameInGroup": "Step 1",              "randomizationUnits": [                "user"              ],              "analysisUnits": [                "user"              ]            }          ],          "analysisType": "mean",          "analysisUnit": "user"        }      ],      "layerSnapshot": {        "key": "checkout-flow",        "name": "Checkout Flow",        "reservationPercent": 10,        "otherReservationPercent": 70      },      "covariateInfo": {        "id": "74a49a2b-4834-4246-917e-5d85231d8c2a",        "fileName": "covariate.csv",        "createdAt": "1654104600000"      }    }  ],  "analysisConfig": {    "bayesianThreshold": "string",    "significanceThreshold": "string",    "testDirection": "string",    "multipleComparisonCorrectionMethod": "bonferroni",    "multipleComparisonCorrectionScope": "variations",    "sequentialTestingEnabled": true  },  "mutableFieldsByStatus": {    "not_started": {      "property1": [        "string"      ],      "property2": [        "string"      ]    },    "running": {      "property1": [        "string"      ],      "property2": [        "string"      ]    },    "stopped": {      "property1": [        "string"      ],      "property2": [        "string"      ]    }  }}

Get experiments GET

Get details about all experiments in an environment. ### Filtering experiments LaunchDarkly supports the `filter` query param for filtering, with the following fields: - `flagKey` filters for only experiments that use the flag with the given key. - `metricKey` filters for only experiments that use the metric with the given key. - `status` filters for only experiments with an iteration with the given status. An iteration can have the status `not_started`, `running` or `stopped`. For example, `filter=flagKey:my-flag,status:running,metricKey:page-load-ms` filters for experiments for the given flag key and the given metric key which have a currently running iteration. ### Expanding the experiments response LaunchDarkly supports five fields for expanding the "Get experiments" response. By default, these fields are **not** included in the response. To expand the response, append the `expand` query parameter and add a comma-separated list with any of the following fields: - `previousIterations` includes all iterations prior to the current iteration. By default only the current iteration is included in the response. - `draftIteration` includes the iteration which has not been started yet, if any. - `secondaryMetrics` includes secondary metrics. By default only the primary metric is included in the response. - `treatments` includes all treatment and parameter details. By default treatment data is not included in the response. - `analysisConfig` includes the analysis configuration for the experiment, such as the Bayesian threshold or significance threshold. For example, `expand=draftIteration,treatments` includes the `draftIteration` and `treatments` fields in the response. If fields that you request with the `expand` query parameter are empty, they are not included in the response.

Get experiment GET

Get details about an experiment. ### Expanding the experiment response LaunchDarkly supports five fields for expanding the "Get experiment" response. By default, these fields are **not** included in the response. To expand the response, append the `expand` query parameter and add a comma-separated list with any of the following fields: - `previousIterations` includes all iterations prior to the current iteration. By default only the current iteration is included in the response. - `draftIteration` includes the iteration which has not been started yet, if any. - `secondaryMetrics` includes secondary metrics. By default only the primary metric is included in the response. - `treatments` includes all treatment and parameter details. By default treatment data is not included in the response. - `analysisConfig` includes the analysis configuration for the experiment, such as the Bayesian threshold or significance threshold. For example, `expand=draftIteration,treatments` includes the `draftIteration` and `treatments` fields in the response. If fields that you request with the `expand` query parameter are empty, they are not included in the response.