Skip to content

Proposal: Agentic process evidence #600

Description

@carmithersh
  1. Use case
    Allow troubleshooting and gating through policy-as-code on agentic processes that ran for a specific, defined task through agentic session log collection and evidence creation.
    This suggestion mostly targets agentic processes being run in the software development lifecycle but is also relevant for other scenarios such as customer's support cases, business decisions and other use cases.
    The suggestion tries to solve the compliance gaps organizations have when using AI on high-risk processes.

  2. How to solve
    Collect agentic session logs related to an agentic process and persist them in a searchable, remote storage, once the process completes, create an in-toto evidence referencing these session logs with the significant data elements inside the predicate that will allow governance on the process.
    This suggestion is detailed in https://github.com/jfrog/agentic-process-evidence, which we would like to offer as a community standard and resource. We welcome contributions and suggestions.

  3. How to gate:

  • Check for allowed/prohibited providers, models, agents, tools and harnesses
  • Verify human accountability
  • Verify human oversight
  • Run post-process alignment checks
  • Verify organization context document usage (rules, guidelines, instructions)
  1. Spec
    While predicate type should be appropriate to the agentic process (e.g. code-development, pr-approvla, promotion, etc...) the predicate content must use a predefined standard format that will allow processing different kind of proceses an organization might handle through its agents.
    See evidence spec here: https://github.com/jfrog/agentic-process-evidence/blob/main/spec/agentic-process-evidence.md

example:

{
  "_type": "https://in-toto.io/Statement/v1",
  "subject": [
    {
      "uri": "https://github.com/octocat/hello-world/commit/bf02510c8ce0b804a099797510af...c19325acfb979538c5b521304c83cde63892",
      "digest": {
        "gitCommit": "bf02510c8ce0b804a099797510af...c19325acfb979538c5b521304c83cde63892"
      }
    }
  ],
  "predicateType": "https://octocat.com/evidence/<agentic-code-review|agentic-dev-process>/v1",
  "predicate": {
    "providers": [
      {
        "harness": {
          "name": "cursor",
          "version": "3.5.33"
        },
        "agent": {
          "id": "stable-agent-id",
          "name": "cursor-agent",
          "version": "1.0"
        },
        "languageModels": [
          {
            "inferenceProvider": "anthropic/claude-opus-4-8",
            "resolved": "anthropic/claude-opus-4-8-20260701"
          }
        ]
      }
    ],
    "traceId": "process id | ci_job_run_uri",
    "sessionsLogs": [
      {
        "uri": "session log url",
        "digest": {
          "sha256": "session log sha..."
        }
      }
    ],
    "tools": [{"name": "tool-name", "version": "tool version"}],
    "contextArtifacts": [
      {
        "tags": ["policy"],
        "uri": "link-to-development-guideline",
        "data": "The complete artifact document inline",
        "digest": {
          "sha256": "policy-sha256"
        }
      },
      {
        "tags": ["guideline"],
        "uri": "link-to-architecture-guidelines",
        "digest": {
          "sha256": "architecture-sha256"
        }
      }
    ],
    "custom": {
      "baseCommit": {
        "uri": "https://github.com/octocat/hello-world/commit/bf02510c8ce0b804a099797510af...c19325acfb979538c5b521304c83cde63892",
        "digest": {
          "gitCommit": "bf02510c8ce0b804a099797510af...c19325acfb979538c5b521304c83cde63892"
        }
      },
      "requirements": [
        {
          "issue": "https://octocat.atlassian.net/browse/FIN-3678",
          "title": "my requirements subject"
        }
      ]
    },
    "result": "COMPLETED",
    "intents": ["short description"],
    "processSummary": "long text",
    "owner": "login|email",
    "reviewers": ["login|email"],
    "startTimestamp": "ISO 8601 timestamp",
    "endTimestamp": "ISO 8601 timestamp"
  },
  "createdAt": "2026-04-08T08:55:17.242Z",
  "createdBy": "dev-auto-reviewer"
}

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions