Result Retention Policy Agent

The Results Retention Policy Agent removes older Results and their associated Records from the DB. The policies apply to PipelineRun, top-level TaskRun, and top-level CustomRun results. Retention policies can be used to manage database size and performance, and the retention duration applies to the database records irrespective of their underlying Runs' age.

It is recommended that the Retention Policy Agent be used in conjunction with a cluster-resource pruning mechanism such as Tekton Results Wacher's resource deletion or Tekton Pruner, with a Results Retention Policy longer than the in-cluster retention period. This avoids the situation where a pruned Result record is re-created in the database because it still exists in the cluster.

For best results, the Retention Policy Agent should also be used in conjunction with the disable_storing_incomplete_runs setting.

Configuration

The Results Retention Policy Agent is configured via the tekton-results-config-results-retention-policy ConfigMap.

Changes to this ConfigMap are automatically reloaded and the pruning schedule is updated without restarting the agent pod. Cleanup runs according to that schedule.

Database settings are loaded separately from the tekton-results-api-config ConfigMap and database credentials from the tekton-results-postgres Secret. Both the API server and the Retention Policy Agent read these settings only at startup. After changing database settings or credentials, restart the pods for both components to apply the changes. See Applying database configuration changes.

The following fields are supported:

• runAt: Determines when to run the pruning job for the DB. It uses a cron schedule format. The default is "7 7 * * 7" (every Sunday at 7:07 AM).
• defaultRetention: The fallback retention period for how long to store Results and Records when no specific policy matches. This value does not override the retention period of a matching policy; it only applies when no policies match a given Result. This can be a number (e.g., 30), which is interpreted as days, or a duration string (e.g., 30d, 24h). The default is 30d.

⚠️ IMPORTANT - Migration from maxRetention to defaultRetention. DATA LOSS RISK

maxRetention is deprecated and will be removed in a future release. Please migrate to defaultRetention as soon as possible. If a user has maxRetention set higher than 30 days and does not migrate to defaultRetention, when maxRetention is removed the defaultRetention records older than the default defaultRetention may be deleted.

To migrate: modify the tekton-results-config-results-retention-policy ConfigMap to rename data.maxRetention to data.defaultRetention.

Backward Compatibility Behavior:

• If both maxRetention and defaultRetention are present, maxRetention takes priority to maintain backward compatibility.
• If only maxRetention is set, it will be used (with a deprecation warning in logs).
• If only defaultRetention is set, it will be used (recommended).

• policies: A list of fine-grained retention policies that allow for more specific control over data retention.

Fine-Grained Retention Policies

You can define a list of policies to control retention based on various criteria. The policies field in the ConfigMap accepts a YAML string containing a list of policy objects. Each policy has a name, a selector, and a retention period.

When the retention job runs, it evaluates a Result against the policies in the order they are defined. The first policy that matches the Result will be applied. If no policies match, the default defaultRetention period is used.

Policy Fields:

• name: A descriptive name for the policy.
• selector: Defines the criteria for matching Results. All conditions within a selector are combined with an AND logic—a Result must meet all specified criteria (matchNamespaces, matchLabels, matchAnnotations, matchStatuses) for the policy to apply. If a particular selector type (e.g., matchLabels) is omitted from a policy, it will match all Results for that criterion. For example, a policy without a matchNamespaces selector will match Results from any namespace.
  • matchNamespaces: A list of namespaces. A Result matches if its namespace is in this list (an OR logic is applied to the values in the list).
  • matchLabels: A map where the key is a label name and the value is a list of possible label values. A Result must have all the specified label keys, and for each key, its value must be in the provided list (an OR logic is applied to the values in the list).
  • matchAnnotations: A map where the key is an annotation name and the value is a list of possible annotation values. This works similarly to matchLabels.
  • matchStatuses: A list of final statuses. A Result matches if its final status is in this list (an OR logic is applied to the values in the list). The status is determined by the reason field of the primary Succeeded condition in the PipelineRun or TaskRun status. Common values include Succeeded, Failed, Cancelled, Running, and Pending. For a more comprehensive list of possible status reasons, refer to the Tekton documentation.
• retention: The retention period for Results matching this policy. This can be a number (e.g., 7), which is interpreted as days, or a duration string (e.g., 24h).

Example ConfigMap:

Here is an example of a ConfigMap that defines multiple, comprehensive retention policies:

apiVersion: v1
kind: ConfigMap
metadata:
  name: tekton-results-config-results-retention-policy
  namespace: tekton-pipelines
data:
  runAt: "0 2 * * *" # Run every day at 2:00 AM
  defaultRetention: "30d"
  policies: |
    - name: "retain-critical-failures-long-term"
      selector:
        matchNamespaces:
          - "production"
          - "prod-east"
        matchLabels:
          "criticality": ["high"]
        matchStatuses:
          - "Failed"
      retention: "180d"
    - name: "retain-annotated-for-debug"
      selector:
        matchAnnotations:
          "debug/retain": ["true"]
      retention: "14d"
    - name: "default-production-policy"
      selector:
        matchNamespaces:
          - "production"
          - "prod-east"
      retention: "60d"
    - name: "short-term-ci-retention"
      selector:
        matchNamespaces:
          - "ci"
      retention: "7d"

In this example:

1. A failed Result in the production or prod-east namespace with the label criticality: high will be kept in the database for 180 days.
2. Any Result with the annotation debug/retain: "true" will be kept for 14 days.
3. Any other Result in the production or prod-east namespace will be kept for 60 days.
4. Any Result in the ci namespace will be kept for 7 days.
5. All other Results that do not match any of these policies will be kept for the default defaultRetention period of 30 days.

Migrating from maxRetention to defaultRetention

In the tekton-results-config-results-retention-policy ConfigMap, rename data.maxRetention to data.defaultRetention.