For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Route OpenAI Decisions API requests through agentgateway and track token usage.

Route OpenAI Decisions API requests through agentgateway to get token usage tracking and observability.

About

The OpenAI Decisions API answers a list of questions about an input. Each question is a predicate, a choice, or a score. The response returns one answer per question.

The TypeSafe SystemOne API (/v1/systemone) uses a different format for the same question types. Agentgateway translates requests and replies between the two formats to match the client and provider.

The following settings apply to both request formats.

Supported providers

These providers accept Decisions and SystemOne requests. Other providers reject them.

If a custom provider declares both formats, each request uses its original format.

A custom provider must declare Decisions or SystemOne. Otherwise, it rejects both request formats with a 400 error. The error names the request format and provider:

failed to process LLM request: unsupported conversion: from Decisions to provider custom

Before you begin

1. Install and set up an agentgateway proxy.
2. Set up access to the OpenAI LLM provider.

Map the Decisions route type

Requests default to the Completions route type. On your OpenAI AgentgatewayBackend, map these paths:

• /v1/decisions to Decisions.
• /v1/systemone to SystemOne.

If you already set up multiple endpoints, add both paths to your existing AgentgatewayBackend.

1. Update the openai AgentgatewayBackend from the OpenAI setup guide. This command replaces that AgentgatewayBackend.

   kubectl apply -f- <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayBackend
   metadata:
     name: openai
     namespace: agentgateway-system
   spec:
     ai:
       provider:
         openai: {}
     policies:
       auth:
         secretRef:
           name: openai-secret
       ai:
         routes:
           "/v1/decisions": "Decisions"
           "/v1/systemone": "SystemOne"
           "/v1/chat/completions": "Completions"
           "*": "Passthrough"
   EOF

   Review the following table to understand this configuration. For more information, see the API docs.

   Setting	Description
   ai.provider.openai	The OpenAI provider. The example sets no default model, so each request must name its model.
   policies.ai.routes."/v1/decisions"	Processes requests on this path as Decisions requests. Without this entry, the "*": "Passthrough" route forwards Decisions requests without processing them. If you omit routes, agentgateway processes every request as a chat completion.
   policies.ai.routes."/v1/systemone"	Processes requests on this path as SystemOne requests. The OpenAI provider receives them translated to the Decisions format.
   policies.ai.routes."*"	Passes requests on all other paths through unchanged.

Send a Decisions request

Send a request with an input and a list of questions to the /v1/decisions path.

1. Send the request. Replace <decisions-model> with an OpenAI model that supports the Decisions API.

   Cloud Provider LoadBalancerPort-forward for local testing

   curl "http://$INGRESS_GW_ADDRESS/v1/decisions" \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "<decisions-model>",
       "input": [{"role": "user", "content": [{"type": "input_text", "text": "The package was left in the rain and the box is soaked through."}]}],
       "questions": [{"type": "predicate", "name": "visible_damage", "instructions": "Does the customer report visible damage to the package?"}]
     }' | jq

   curl "localhost:8080/v1/decisions" \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "<decisions-model>",
       "input": [{"role": "user", "content": [{"type": "input_text", "text": "The package was left in the rain and the box is soaked through."}]}],
       "questions": [{"type": "predicate", "name": "visible_damage", "instructions": "Does the customer report visible damage to the package?"}]
     }' | jq

   Example output:

   {
     "model": "<decisions-model>",
     "answers": [
       {"type": "predicate", "name": "visible_damage", "probability": 0.64}
     ],
     "usage": {"input_tokens": 156, "output_tokens": 0, "total_tokens": 156}
   }

2. Optional: Send the same question in SystemOne. Use state for the input and a map keyed by name for the questions.

   The noul question type returns the probability that a statement is true. It corresponds to predicate in the Decisions API.

   Cloud Provider LoadBalancerPort-forward for local testing

   curl "http://$INGRESS_GW_ADDRESS/v1/systemone" \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "<decisions-model>",
       "state": [{"role": "user", "content": "The package was left in the rain and the box is soaked through."}],
       "questions": {"visible_damage": {"type": "noul", "instructions": "Does the customer report visible damage to the package?"}}
     }' | jq

   curl "localhost:8080/v1/systemone" \
     -H 'Content-Type: application/json' \
     -d '{
       "model": "<decisions-model>",
       "state": [{"role": "user", "content": "The package was left in the rain and the box is soaked through."}],
       "questions": {"visible_damage": {"type": "noul", "instructions": "Does the customer report visible damage to the package?"}}
     }' | jq

   Example output:

   {
     "model": "<decisions-model>",
     "answers": {"visible_damage": {"type": "noul", "noul": 0.64}},
     "usage": {"input_tokens": 156, "output_tokens": 0}
   }

Verify token tracking

After the request completes, check that agentgateway recorded the token usage in its metrics.

1. Port-forward the agentgateway proxy on port 15020.

   kubectl port-forward deployment/agentgateway-proxy -n agentgateway-system 15020 &

2. Open the agentgateway metrics endpoint.

3. Look for the agentgateway_gen_ai_client_token_usage metric. The metric includes labels for the token type (input or output) and the model.

For more information about LLM metrics, see LLM cost tracking.

Cleanup

To restore the original AgentgatewayBackend, apply the openai AgentgatewayBackend from OpenAI again.

To keep the AgentgatewayBackend but stop processing Decisions and SystemOne requests, remove the "/v1/decisions": "Decisions" and "/v1/systemone": "SystemOne" entries from its routes.