Holdouts
Get holdout
Get details about a holdout.
Expanding the holdout response
LaunchDarkly supports the following fields for expanding the "Get holdout" response. By default, these fields are not included in the response.
To expand the response, append the expand query parameter and add a comma-separated list with any of the following fields:
draftIterationincludes the iteration which has not been started yet, if any, for this holdout.previousIterationsincludes all iterations prior to the current iteration, for this holdout. By default only the current iteration is included in the response.rel-draftIterationincludes the iteration which has not been started yet, if any, for the experiments related to this holdout.rel-metricsincludes metrics for experiments related to this holdout.rel-previousIterationsincludes all iterations prior to the current iteration, for the experiments related to this holdout.rel-secondaryMetricsincludes secondary metrics for experiments related to this holdout.rel-treatmentsincludes all treatment and parameter details for experiments related to this holdout.secondaryMetricsincludes secondary metrics for this holdout. By default only the primary metric is included in the response.treatmentsincludes all treatment and parameter details for this holdout. By default treatment data is not included in the response.
For example, expand=draftIteration,rel-draftIteration includes the draftIteration and rel-draftIteration fields in the response. If fields that you request with the expand query parameter are empty, they are not included in the response.
Authorization
ApiKey read, writeAuthorization<token>
In: header
Scope: read, write
Path Parameters
projectKey*string
The project key
Format
stringenvironmentKey*string
The environment key
Format
stringholdoutKey*string
The holdout experiment key
Format
stringQuery Parameters
expand?string
A comma-separated list of properties that can reveal additional information in the response. Supported fields are explained above. Holdout experiment expansion fields have no prefix. Related experiment expansion fields have rel- as a prefix.
Format
stringResponse Body
application/json
application/json
application/json
application/json
application/json
application/json
curl -X GET "https://example.com/api/v2/projects/string/environments/string/holdouts/string"{ "_id": "string", "status": "created", "description": "string", "holdoutAmount": "string", "isDirty": true, "createdAt": 0, "updatedAt": 0, "baseExperiment": { "_id": "12ab3c45de678910fgh12345", "key": "experiment-key-123abc", "name": "Example experiment", "description": "An example experiment, used in testing", "_maintainerId": "12ab3c45de678910fgh12345", "_creationDate": "1654104600000", "environmentKey": "string", "methodology": "bayesian", "dataSource": "snowflake", "archivedDate": "1654104600000", "tags": [ "experiment", "feature" ], "_links": { "parent": { "href": "/api/v2/projects/my-project/environments/my-environment", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment", "type": "application/json" } }, "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40", "currentIteration": { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } }, "type": "experiment", "_access": { "denied": [ { "action": "string", "reason": { "resources": [ "proj/*:env/*;qa_*:/flag/*" ], "notResources": [ "string" ], "actions": [ "*" ], "notActions": [ "string" ], "effect": "allow", "role_name": "string" } } ], "allowed": [ { "action": "string", "reason": { "resources": [ "proj/*:env/*;qa_*:/flag/*" ], "notResources": [ "string" ], "actions": [ "*" ], "notActions": [ "string" ], "effect": "allow", "role_name": "string" } } ] }, "draftIteration": { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } }, "previousIterations": [ { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } } ], "analysisConfig": { "bayesianThreshold": "string", "significanceThreshold": "string", "testDirection": "string", "multipleComparisonCorrectionMethod": "bonferroni", "multipleComparisonCorrectionScope": "variations", "sequentialTestingEnabled": true }, "mutableFieldsByStatus": { "not_started": { "property1": [ "string" ], "property2": [ "string" ] }, "running": { "property1": [ "string" ], "property2": [ "string" ] }, "stopped": { "property1": [ "string" ], "property2": [ "string" ] } } }, "relatedExperiments": [ { "_id": "12ab3c45de678910fgh12345", "key": "experiment-key-123abc", "name": "Example experiment", "description": "An example experiment, used in testing", "_maintainerId": "12ab3c45de678910fgh12345", "_creationDate": "1654104600000", "environmentKey": "string", "methodology": "bayesian", "dataSource": "snowflake", "archivedDate": "1654104600000", "tags": [ "experiment", "feature" ], "_links": { "parent": { "href": "/api/v2/projects/my-project/environments/my-environment", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/environments/my-environment/experiments/my-experiment", "type": "application/json" } }, "holdoutId": "f3b74309-d581-44e1-8a2b-bb2933b4fe40", "currentIteration": { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } }, "type": "experiment", "_access": { "denied": [ { "action": "string", "reason": { "resources": [ "proj/*:env/*;qa_*:/flag/*" ], "notResources": [ "string" ], "actions": [ "*" ], "notActions": [ "string" ], "effect": "allow", "role_name": "string" } } ], "allowed": [ { "action": "string", "reason": { "resources": [ "proj/*:env/*;qa_*:/flag/*" ], "notResources": [ "string" ], "actions": [ "*" ], "notActions": [ "string" ], "effect": "allow", "role_name": "string" } } ] }, "draftIteration": { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } }, "previousIterations": [ { "_id": "12ab3c45de678910fgh12345", "hypothesis": "The new button placement will increase conversion", "status": "running", "createdAt": "1654104600000", "startedAt": "1655314200000", "endedAt": "1656610200000", "winningTreatmentId": "122c9f3e-da26-4321-ba68-e0fc02eced58", "winningReason": "We ran this iteration for two weeks and the winning variation was clear", "canReshuffleTraffic": true, "flags": { "property1": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } }, "property2": { "targetingRule": "fallthrough", "targetingRuleDescription": "Customers who live in Canada", "targetingRuleClauses": [ null ], "flagConfigVersion": 12, "notInExperimentVariationId": "e432f62b-55f6-49dd-a02f-eb24acf39d05", "_links": { "self": { "href": "/api/v2/flags/my-project/my-flag", "type": "application/json" } } } }, "reallocationFrequencyMillis": 3600000, "version": 0, "primaryMetric": { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" }, "primarySingleMetric": { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } }, "primaryFunnel": { "key": "metric-group-key-123abc", "name": "My metric group", "kind": "funnel", "_links": { "parent": { "href": "/api/v2/projects/my-project", "type": "application/json" }, "self": { "href": "/api/v2/projects/my-project/metric-groups/my-metric-group", "type": "application/json" } }, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ] }, "randomizationUnit": "user", "attributes": [ "string" ], "treatments": [ { "_id": "122c9f3e-da26-4321-ba68-e0fc02eced58", "name": "Treatment 1", "allocationPercent": "10", "baseline": true, "parameters": [ { "variationId": "string", "flagKey": "string" } ] } ], "secondaryMetrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } } } ], "metrics": [ { "key": "metric-key-123abc", "_versionId": "string", "name": "My metric", "kind": "custom", "isNumeric": true, "eventKey": "event-key-123abc", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "isGroup": true, "metrics": [ { "key": "metric-key-123abc", "_versionId": "version-id-123abc", "name": "Example metric", "kind": "custom", "isNumeric": true, "unitAggregationType": "sum", "analysisType": "mean", "eventKey": "event-key-123abc", "analysisUnit": "string", "_links": { "self": { "href": "/api/v2/metrics/my-project/my-metric", "type": "application/json" } }, "nameInGroup": "Step 1", "randomizationUnits": [ "user" ], "analysisUnits": [ "user" ] } ], "analysisType": "mean", "analysisUnit": "user" } ], "layerSnapshot": { "key": "checkout-flow", "name": "Checkout Flow", "reservationPercent": 10, "otherReservationPercent": 70 }, "covariateInfo": { "id": "74a49a2b-4834-4246-917e-5d85231d8c2a", "fileName": "covariate.csv", "createdAt": "1654104600000" } } ], "analysisConfig": { "bayesianThreshold": "string", "significanceThreshold": "string", "testDirection": "string", "multipleComparisonCorrectionMethod": "bonferroni", "multipleComparisonCorrectionScope": "variations", "sequentialTestingEnabled": true }, "mutableFieldsByStatus": { "not_started": { "property1": [ "string" ], "property2": [ "string" ] }, "running": { "property1": [ "string" ], "property2": [ "string" ] }, "stopped": { "property1": [ "string" ], "property2": [ "string" ] } } } ]}Get Holdout by Id GET
Previous Page
Patch holdout PATCH
Updates an existing holdout, and returns the updated holdout. Updating holdouts uses the semantic patch format. To make a semantic patch request, you must append `domain-model=launchdarkly.semanticpatch` to your `Content-Type` header. To learn more, read [Updates using semantic patch](https://launchdarkly.com/docs/api#updates-using-semantic-patch). ### Instructions Semantic patch requests support the following `kind` instructions for updating holdouts. <details> <summary>Click to expand instructions for <strong>updating holdouts</strong></summary> #### endHoldout Ends a holdout. ##### Parameters None. Here's an example: ```json { "comment": "Optional comment describing why the holdout is ending", "instructions": [{ "kind": "endHoldout" }] } ``` #### removeExperiment Removes an experiment from a holdout. ##### Parameters - `value`: The key of the experiment to remove Here's an example: ```json { "comment": "Optional comment describing the change", "instructions": [{ "kind": "removeExperiment", "value": "experiment-key" }] } ``` #### updateDescription Updates the description of the holdout. ##### Parameters - `value`: The new description. Here's an example: ```json { "comment": "Optional comment describing the update", "instructions": [{ "kind": "updateDescription", "value": "Updated holdout description" }] } ``` #### updateName Updates the name of the holdout. ##### Parameters - `value`: The new name. Here's an example: ```json { "comment": "Optional comment describing the update", "instructions": [{ "kind": "updateName", "value": "Updated holdout name" }] } ``` </details>