Experiments
Get experiments
Get details about all experiments in an environment.
Filtering experiments
LaunchDarkly supports the filter query param for filtering, with the following fields:
flagKeyfilters for only experiments that use the flag with the given key.metricKeyfilters for only experiments that use the metric with the given key.statusfilters for only experiments with an iteration with the given status. An iteration can have the statusnot_started,runningorstopped.
For example, filter=flagKey:my-flag,status:running,metricKey:page-load-ms filters for experiments for the given flag key and the given metric key which have a currently running iteration.
Expanding the experiments response
LaunchDarkly supports five fields for expanding the "Get experiments" response. By default, these fields are not included in the response.
To expand the response, append the expand query parameter and add a comma-separated list with any of the following fields:
previousIterationsincludes all iterations prior to the current iteration. By default only the current iteration is included in the response.draftIterationincludes the iteration which has not been started yet, if any.secondaryMetricsincludes secondary metrics. By default only the primary metric is included in the response.treatmentsincludes all treatment and parameter details. By default treatment data is not included in the response.analysisConfigincludes the analysis configuration for the experiment, such as the Bayesian threshold or significance threshold.
For example, expand=draftIteration,treatments includes the draftIteration and treatments fields in the response. If fields that you request with the expand query parameter are empty, they are not included in the response.
Authorization
ApiKey read, writeAuthorization<token>
In: header
Scope: read, write
Path Parameters
projectKey*string
The project key
Format
stringenvironmentKey*string
The environment key
Format
stringQuery Parameters
limit?integer
The maximum number of experiments to return. Defaults to 20.
Format
int64offset?integer
Where to start in the list. Use this with pagination. For example, an offset of 10 skips the first ten items and then returns the next items in the list, up to the query limit.
Format
int64filter?string
A comma-separated list of filters. Each filter is of the form field:value. Supported fields are explained above.
Format
stringexpand?string
A comma-separated list of properties that can reveal additional information in the response. Supported fields are explained above.
Format
stringlifecycleState?string
A comma-separated list of experiment archived states. Supports archived, active, or both. Defaults to active experiments.
Format
stringResponse Body
application/json
application/json
application/json
application/json
application/json
application/json
application/json
curl -X GET "https://example.com/api/v2/projects/string/environments/string/experiments"{ "items": [ { "_id": "12ab3c45de678910fgh12345", "key": "experiment-key-123abc", "name": "Example experiment", "description": "An example experiment, used in testing", "_maintainerId": "12ab3c45de678910fgh12345", "_creationDate": "1654104600000", "environmentKey": "string", "methodology": "bayesian", "dataSource": "snowflake", "archivedDate": "1654104600000", "tags": [ "experiment", "feature" ], "_links": { "parent": { "href": "/api/v2/projects/my-project/environments/my-environment", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment", "type": "application/json" } }, "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40", "currentIteration": { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } }, "type": "experiment", "_access": { "denied": [ { "action": "string", "reason": { "resources": [ "proj/*:env/*;qa_*:/flag/*" ], "notResources": [ "string" ], "actions": [ "*" ], "notActions": [ "string" ], "effect": "allow", "role_name": "string" } } ], "allowed": [ { "action": "string", "reason": { "resources": [ "proj/*:env/*;qa_*:/flag/*" ], "notResources": [ "string" ], "actions": [ "*" ], "notActions": [ "string" ], "effect": "allow", "role_name": "string" } } ] }, "draftIteration": { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } }, "previousIterations": [ { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } } ], "analysisConfig": { "bayesianThreshold": "string", "significanceThreshold": "string", "testDirection": "string", "multipleComparisonCorrectionMethod": "bonferroni", "multipleComparisonCorrectionScope": "variations", "sequentialTestingEnabled": true }, "mutableFieldsByStatus": { "not_started": { "property1": [ "string" ], "property2": [ "string" ] }, "running": { "property1": [ "string" ], "property2": [ "string" ] }, "stopped": { "property1": [ "string" ], "property2": [ "string" ] } } } ], "total_count": 0, "_links": { "property1": { "href": "string", "type": "string" }, "property2": { "href": "string", "type": "string" } }}Update flag settings for context PUT
Enable or disable a feature flag for a context based on its context kind and key. In the request body, the `setting` should be the variation value to set for the context. It must match the flag's variation type. For example, for a boolean flag you can use `"setting": true` or `"setting": false` in the request body. For a string flag, you can use `"setting": "existing_variation_value_to_use"`. Omitting the `setting` attribute from the request body, or including a `setting` of `null`, erases the current setting for a context. If you previously patched the flag, and the patch included the context's data, LaunchDarkly continues to use that data. If LaunchDarkly has never encountered the combination of the context's key and kind before, it calculates the flag values based on the context kind and key.
Create experiment POST
Create an experiment. To run this experiment, you'll need to [create an iteration](https://launchdarkly.com/docs/api/experiments/create-iteration) and then [update the experiment](https://launchdarkly.com/docs/api/experiments/patch-experiment) with the `startIteration` instruction. ### Experiment types Use the `type` field to specify the experiment type: `experiment` (default), `mab` (multi-armed bandit), or `holdout`. For multi-armed bandit experiments, set `reallocationFrequencyMillis` on the iteration to control how often traffic is reallocated across variations. ### Results analysis Use the `methodology` field to specify the results analysis approach: `bayesian` (default) or `frequentist`. Use the `dataSource` field to specify the source of metric data: `launchdarkly` (default), `snowflake`, or `databricks`. Use the `analysisConfig` field to customize analysis settings such as the Bayesian threshold, significance threshold, or multiple comparison correction method. To learn more, read [Creating experiments](https://launchdarkly.com/docs/home/experimentation/create).