You are viewing documentation for Kubeflow 0.7

This is a static snapshot from the time of the Kubeflow 0.7 release.
For up-to-date information, see the latest version.

Metadata API Reference

Reference documentation for the Kubeflow Metadata API
api/service.proto

Kubeflow Metadata Service REST API

Version: v1alpha1

Default request content-types: application/json
Default response content-types: application/json
Schemes: http, https

Summary

Path Operation Description
/api/v1alpha1/artifact_types GET
POST
/api/v1alpha1/artifact_types/{name} DELETE
GET
/api/v1alpha1/artifact_types/{name}/artifacts GET
/api/v1alpha1/artifact_types/{name}/artifacts/{id} DELETE
GET
/api/v1alpha1/artifact_types/{parent}/artifacts POST

/api/v1alpha1/artifacts GET
/api/v1alpha1/events POST
/api/v1alpha1/events/artifacts/{name} GET

List events based on an artifact or execution id.
/api/v1alpha1/events/executions/{name} GET

List events based on an artifact or execution id.
/api/v1alpha1/execution_types GET
POST
/api/v1alpha1/execution_types/{name} DELETE
GET
/api/v1alpha1/execution_types/{name}/executions GET
/api/v1alpha1/execution_types/{name}/executions/{id} DELETE
GET
/api/v1alpha1/execution_types/{parent}/executions POST
/api/v1alpha1/executions GET

Paths

GET /api/v1alpha1/artifact_types

Tags: MetadataService

Uses default content-types: application/json

200 OK

A successful response.

apiListArtifactTypesResponse

POST /api/v1alpha1/artifact_types

Tags: MetadataService

Uses default content-types: application/json

ml_metadataArtifactType

Uses default content-types: application/json

200 OK

A successful response.

apiCreateArtifactTypeResponse

DELETE /api/v1alpha1/artifact_types/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}

Tags: MetadataService
name

ArtifactType names are of the form artifact_types/{namespace}/{name}.

 path string

Uses default content-types: application/json

200 OK

A successful response.

apiGetArtifactTypeResponse

GET /api/v1alpha1/artifact_types/{name}/artifacts

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

apiListArtifactsResponse

DELETE /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags: MetadataService
id path string
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags: MetadataService
id path string
name

Artifact name is like artifact_types/{namespace}/{typename}/artifact/{id}.

 path string

Uses default content-types: application/json

200 OK

A successful response.

apiGetArtifactResponse
NOTE: The order of the following RPC methods affects the order of matching a particular HTTP path. So put a more specific path pattern before a generic one. For example, GET /api/v1alpha1/artifact_types/{parent}/artifacts should appear before GET /api/v1alpha1/artifact_types/{name} to be possibly matched.

POST /api/v1alpha1/artifact_types/{parent}/artifacts

Tags: MetadataService

Uses default content-types: application/json

The Artifact to create. Note that Artifact.type_id is ignored.

ml_metadataArtifact
parent

Creates the specified artifact as an instance of ArtifactType with this fully qualified name. |parent| takes the form artifact_types/{namespace}/{name>}.

 path string

Uses default content-types: application/json

200 OK

A successful response.

apiCreateArtifactResponse

GET /api/v1alpha1/artifacts

Tags: MetadataService
name query string

Uses default content-types: application/json

200 OK

A successful response.

apiListArtifactsResponse

POST /api/v1alpha1/events

Tags: MetadataService

Uses default content-types: application/json

ml_metadataEvent

Uses default content-types: application/json

200 OK

A successful response.

List events based on an artifact or execution id.

GET /api/v1alpha1/events/artifacts/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

apiListEventsResponse
List events based on an artifact or execution id.

GET /api/v1alpha1/events/executions/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

apiListEventsResponse

GET /api/v1alpha1/execution_types

Tags: MetadataService

Uses default content-types: application/json

200 OK

A successful response.

apiListExecutionTypesResponse

POST /api/v1alpha1/execution_types

Tags: MetadataService

Uses default content-types: application/json

ml_metadataExecutionType

Uses default content-types: application/json

200 OK

A successful response.

apiCreateExecutionTypeResponse

DELETE /api/v1alpha1/execution_types/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

apiGetExecutionTypeResponse

GET /api/v1alpha1/execution_types/{name}/executions

Tags: MetadataService
name path string

Uses default content-types: application/json

200 OK

A successful response.

apiListExecutionsResponse

DELETE /api/v1alpha1/execution_types/{name}/executions/{id}

Tags: MetadataService
id path string
name path string

Uses default content-types: application/json

200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}/executions/{id}

Tags: MetadataService
id path string
name

Execution name is like execution_types/{namespace}/{typename}/execution/{id}.

 path string

Uses default content-types: application/json

200 OK

A successful response.

apiGetExecutionResponse

POST /api/v1alpha1/execution_types/{parent}/executions

Tags: MetadataService

Uses default content-types: application/json

The Execution to create. Note that Execution.type_id is ignored.

ml_metadataExecution
parent

Creates the specified artifact as an instance of ExecutionType with this fully qualified name. |parent| takes the form execution_types/{namespace}/{name>}.

 path string

Uses default content-types: application/json

200 OK

A successful response.

apiCreateExecutionResponse

GET /api/v1alpha1/executions

Tags: MetadataService
name query string

Uses default content-types: application/json

200 OK

A successful response.

apiListExecutionsResponse

Schema definitions

apiCreateArtifactResponse: object

artifact: ml_metadataArtifact

Newly created artifact with id.

apiCreateArtifactTypeResponse: object

artifact_type: ml_metadataArtifactType

Newly created artifact type with id.

apiCreateExecutionResponse: object

execution: ml_metadataExecution

Newly created execution with id.

apiCreateExecutionTypeResponse: object

execution_type: ml_metadataExecutionType

Newly created execution type with id.

apiGetArtifactResponse: object

artifact: ml_metadataArtifact

apiGetArtifactTypeResponse: object

artifact_type: ml_metadataArtifactType

apiGetExecutionResponse: object

execution: ml_metadataExecution

apiGetExecutionTypeResponse: object

execution_type: ml_metadataExecutionType

apiListArtifactsResponse: object

artifacts: object[]
ml_metadataArtifact

apiListArtifactTypesResponse: object

artifact_types: object[]
ml_metadataArtifactType

apiListEventsResponse: object

events: object[]
ml_metadataEvent
artifacts: object
ml_metadataArtifact
executions: object
ml_metadataExecution

apiListExecutionsResponse: object

executions: object[]
ml_metadataExecution

apiListExecutionTypesResponse: object

execution_types: object[]
ml_metadataExecutionType

EventPath: object

A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

steps: object[]

A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

PathStep

ml_metadataAnyArtifactStructType: object

Every ArtifactStruct is a member of this type.

ml_metadataArtifact: object

id: string (int64)

The id of the artifact.

type_id: string (int64)

The id of an ArtifactType. Type must be specified when an artifact is created, and it cannot be changed.

uri: string

The uniform resource identifier of the physical artifact. May be empty if there is no physical artifact.

properties: object

Properties of the artifact. Properties must be specified in the ArtifactType.

ml_metadataValue
custom_properties: object

User provided custom properties which are not defined by its type.

ml_metadataValue

ml_metadataArtifactStructType: object

The type of an ArtifactStruct. An artifact struct type represents an infinite set of artifact structs. It can specify the input or output type of an ExecutionType. See the more specific types referenced in the message for more details.

simple: ml_metadataArtifactType
union_type: ml_metadataUnionArtifactStructType
intersection: ml_metadataIntersectionArtifactStructType
list: ml_metadataListArtifactStructType
none: ml_metadataNoneArtifactStructType
any: ml_metadataAnyArtifactStructType
tuple: ml_metadataTupleArtifactStructType
dict: ml_metadataDictArtifactStructType

ml_metadataArtifactType: object

id: string (int64)

The id of the type. 1-1 relationship between type names and IDs.

name: string

The name of the type.

properties: object

The schema of the type. Properties are always optional in the artifact. Properties of an artifact type can be expanded but not contracted (i.e., you can add columns but not remove them).

ml_metadataPropertyType

ml_metadataDictArtifactStructType: object

properties: object

Underlying properties for the type.

ml_metadataArtifactStructType
none_type_not_required: boolean (boolean)

If true, then if properties["foo"] can be None, then that key is not required.

extra_properties_type: ml_metadataArtifactStructType

Extra keys are allowed that are not specified in properties. These keys must have the type specified below. If this is not specified, then extra properties are not allowed.

ml_metadataEvent: object

An event represents a relationship between an artifact ID and an execution. There are four kinds of events, relating to both input and output, as well as declared versus undeclared. For example, the DECLARED_INPUT events are part of the signature of an execution. For example, consider: my_execution({"data":[3,7],"schema":8}) Where 3, 7, and 8 are artifact_ids. Assuming execution_id is 12, this becomes: {artifact_id:3, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":0}]}} {artifact_id:7, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":1}]}} {artifact_id:8, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"schema"}]}} The INPUT is an artifact actually read by the execution. The OUTPUT is an artifact actually written by the execution. The DECLARED_OUTPUT are the artifacts that are the "official" output. For example, the trainer may output multiple caches of the parameters (as OUTPUT objects), but then finally write the SavedModel as a DECLARED_OUTPUT. TODO(martinz): add a type for Event, similar to ArtifactType.

artifact_id: string (int64)

The artifact id is required for an event, and should refer to an existing artifact.

execution_id: string (int64)

The execution_id is required for an event, and should refer to an existing execution.

path: EventPath

The path in an artifact struct, or the name of an artifact.

type: ml_metadataEventType

The type of an event.

milliseconds_since_epoch: string (int64)

ml_metadataEventType: string , x ∈ { UNKNOWN (default) , DECLARED_OUTPUT , DECLARED_INPUT , INPUT , OUTPUT }

Events distinguish between an artifact that is written by the execution (possibly as a cache), versus artifacts that are part of the declared output of the Execution. For more information on what DECLARED_ means, see the comment on the message.

ml_metadataExecution: object

id: string (int64)

The id of the execution.

type_id: string (int64)

The id of an ExecutionType. The ExecutionType must be specified and cannot be changed.

last_known_state: ml_metadataExecutionState

The last known state of an execution.

properties: object

Properties of the Execution. Properties must be specified in the ExecutionType.

ml_metadataValue
custom_properties: object

User provided custom properties which are not defined by its type.

ml_metadataValue

ml_metadataExecutionState: string , x ∈ { UNKNOWN (default) , NEW , RUNNING , COMPLETE , FAILED }

TODO(martinz): consider adding INVALIDATED as a state.

ml_metadataExecutionType: object

id: string (int64)

The id of the type. 1-1 relationship between type names and IDs.

name: string

The name of the type.

properties: object

The schema of the type. Properties are always optional in the execution.

ml_metadataPropertyType
input_type: ml_metadataArtifactStructType

The ArtifactStructType of the input. For example: { "dict":{ "properties":{ "schema":{ "union_type":{ "none":{}, "simple":{...schema type...} }, }, "data":{ "simple":{...data_type...} } } } } That would be an optional schema field with a required data field.

output_type: ml_metadataArtifactStructType

ml_metadataIntersectionArtifactStructType: object

A member of this type must satisfy all constraints. This primarily useful not as an end-user type, but something calculated as an intermediate type in the system.

For example, suppose you have a method: def infer_my_input_type(a): # try to infer the input type of this method. use_in_method_x(a) # with input type x_input use_in_method_y(a) # with input type y_input

Given this information, you know that infer_my_input_type has type {"intersection":{"constraints":[x_input, y_input]}}.

IntersectionArtifactStructType intersection_type = {"constraints":[ {"dict":{"properties":{"schema":{"any":{}}}, "extra_properties":{"any":{}}}}, {"dict":{"properties":{"data":{"any":{}}}, "extra_properties":{"any":{}}}}]} Since the first constraint requires the dictionary to have a schema property, and the second constraint requires it to have a data property, this is equivalent to: ArtifactStructType other_type = {"dict":{"properties":{"schema":{"any":{}},"data":{"any":{}}}}, "extra_properties":{"any":{}}}

constraints: object[]
ml_metadataArtifactStructType

ml_metadataListArtifactStructType: object

Represents an ArtifactStruct list type with homogeneous elements.

element: ml_metadataArtifactStructType

Every entry in the list must be of this type. Note: if this type is Any, then the list can have arbitrary elements.

ml_metadataNoneArtifactStructType: object

ml_metadataPropertyType: string , x ∈ { UNKNOWN (default) , INT , DOUBLE , STRING }

TODO(martinz): consider moving this inside some message, to avoid having literals directly in apo package.

ml_metadataTupleArtifactStructType: object

An ordered list of heterogenous artifact structs. The length of the list is fixed. Each position in the list can have a different type.

elements: object[]
ml_metadataArtifactStructType

ml_metadataUnionArtifactStructType: object

Represents a union of types.

candidates: object[]

An artifact struct matches this type if it matches any of the candidates. If candidates is empty, this is a bottom type (matches no artifacts).

ml_metadataArtifactStructType

ml_metadataValue: object

A value in properties.

int_value: string (int64)
double_value: number (double)
string_value: string

PathStep: object

index: string (int64)
key: string

Last modified 12.08.2019: Add metadata API reference doc (#1058) (2d9564cb)