EvaluatorGenerationJob interface

Package:
@azure/ai-projects

Evaluator Generation Job resource — a long-running job that generates rubric-based evaluator definitions from source materials. On success, the result is the persisted EvaluatorVersion.

Properties

created_at

The timestamp when the job was created, represented in Unix time (seconds since January 1, 1970).
error

Error details — populated only on failure.
evaluator_description

Optional human-friendly description for the resulting evaluator. Surfaced as EvaluatorVersion.description on the persisted evaluator. Typically collected from the UI alongside evaluator_display_name. The evaluator_ prefix disambiguates this from any other description fields on related models.
evaluator_display_name

Optional human-friendly display name for the resulting evaluator. Surfaced as EvaluatorVersion.display_name on the persisted evaluator. When omitted, the service uses evaluator_name as the display name. The evaluator_ prefix disambiguates this from the immutable evaluator_name identifier.
evaluator_name

The evaluator name (immutable identifier). 1-256 characters; allowed characters are ASCII letters, digits, underscore (_), period (.), tilde (~), and hyphen (-). The prefix builtin. is reserved for system-managed evaluators and is rejected by the service. If an evaluator with this name already exists in the project (and is rubric-subtype), the service creates a new version under the same name and uses the prior version's dimensions as context for incremental improvement (foundation of the post-//build adaptive loop). Old versions remain queryable via get_version(name, version). If the existing evaluator is not a rubric-subtype evaluator (built-in, prompt-based, code-based), the request is rejected with 400 Bad Request.
finished_at

The timestamp when the job finished, represented in Unix time (seconds since January 1, 1970).
id

Server-assigned unique identifier.
input_quality_warnings

Non-fatal input-quality advisories produced by the generation pipeline. Read-only; service-generated; populated only on terminal jobs when advisories fired. Omitted when generation was clean. Cleared when a subsequent PATCH to the paired EvaluatorVersion.definition invalidates the advisories.
model

The LLM model to use for rubric generation (e.g., 'gpt-4o'). Required — users must provide their own model rather than relying on service-owned capacity.
result

Result produced on success.
sources

Source materials for generation — agent descriptions, prompts, traces, or datasets. Each entry is an EvaluatorGenerationJobSource variant discriminated by type.
status

Current lifecycle status.
usage

Token consumption summary. Populated when the job reaches a terminal state.

Property Details

created_at

The timestamp when the job was created, represented in Unix time (seconds since January 1, 1970).

created_at?: Date

Property Value

Date

error

Error details — populated only on failure.

error?: ErrorModel

Property Value

ErrorModel

evaluator_description

Optional human-friendly description for the resulting evaluator. Surfaced as EvaluatorVersion.description on the persisted evaluator. Typically collected from the UI alongside evaluator_display_name. The evaluator_ prefix disambiguates this from any other description fields on related models.

evaluator_description?: string

Property Value

string

evaluator_display_name

Optional human-friendly display name for the resulting evaluator. Surfaced as EvaluatorVersion.display_name on the persisted evaluator. When omitted, the service uses evaluator_name as the display name. The evaluator_ prefix disambiguates this from the immutable evaluator_name identifier.

evaluator_display_name?: string

Property Value

string

evaluator_name

The evaluator name (immutable identifier). 1-256 characters; allowed characters are ASCII letters, digits, underscore (_), period (.), tilde (~), and hyphen (-). The prefix builtin. is reserved for system-managed evaluators and is rejected by the service. If an evaluator with this name already exists in the project (and is rubric-subtype), the service creates a new version under the same name and uses the prior version's dimensions as context for incremental improvement (foundation of the post-//build adaptive loop). Old versions remain queryable via get_version(name, version). If the existing evaluator is not a rubric-subtype evaluator (built-in, prompt-based, code-based), the request is rejected with 400 Bad Request.

evaluator_name: string

Property Value

string

finished_at

The timestamp when the job finished, represented in Unix time (seconds since January 1, 1970).

finished_at?: Date

Property Value

Date

id

Server-assigned unique identifier.

id?: string

Property Value

string

input_quality_warnings

Non-fatal input-quality advisories produced by the generation pipeline. Read-only; service-generated; populated only on terminal jobs when advisories fired. Omitted when generation was clean. Cleared when a subsequent PATCH to the paired EvaluatorVersion.definition invalidates the advisories.

input_quality_warnings?: RubricGenerationInputQualityWarning[]

Property Value

RubricGenerationInputQualityWarning[]

model

The LLM model to use for rubric generation (e.g., 'gpt-4o'). Required — users must provide their own model rather than relying on service-owned capacity.

model: string

Property Value

string

result

Result produced on success.

result?: EvaluatorVersion

Property Value

EvaluatorVersion

sources

Source materials for generation — agent descriptions, prompts, traces, or datasets. Each entry is an EvaluatorGenerationJobSource variant discriminated by type.

sources: EvaluatorGenerationJobSourceUnion[]

Property Value

EvaluatorGenerationJobSourceUnion[]

status

Current lifecycle status.

status?: JobStatus

Property Value

JobStatus

usage

Token consumption summary. Populated when the job reaches a terminal state.

usage?: EvaluatorGenerationTokenUsage

Property Value

EvaluatorGenerationTokenUsage