GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1EvaluationJobConfig (google_api_data_labeling v0.4.1)
View SourceConfigures specific details of how a continuous evaluation job works. Provide this configuration when you create an EvaluationJob.
Attributes
-
bigqueryImportKeys
(type:map()
, default:nil
) - Required. Prediction keys that tell Data Labeling Service where to find the data for evaluation in your BigQuery table. When the service samples prediction input and output from your model version and saves it to BigQuery, the data gets stored as JSON strings in the BigQuery table. These keys tell Data Labeling Service how to parse the JSON. You can provide the following entries in this field:data_json_key
: the data key for prediction input. You must provide either this key orreference_json_key
.reference_json_key
: the data reference key for prediction input. You must provide either this key ordata_json_key
.label_json_key
: the label key for prediction output. Required.label_score_json_key
: the score key for prediction output. Required. *bounding_box_json_key
: the bounding box key for prediction output. Required if your model version perform image object detection. Learn how to configure prediction keys. -
boundingPolyConfig
(type:GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1BoundingPolyConfig.t
, default:nil
) - Specify this field if your model version performs image object detection (bounding box detection).annotationSpecSet
in this configuration must match EvaluationJob.annotationSpecSet. -
evaluationConfig
(type:GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1EvaluationConfig.t
, default:nil
) - Required. Details for calculating evaluation metrics and creating Evaulations. If your model version performs image object detection, you must specify theboundingBoxEvaluationOptions
field within this configuration. Otherwise, provide an empty object for this configuration. -
evaluationJobAlertConfig
(type:GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1EvaluationJobAlertConfig.t
, default:nil
) - Optional. Configuration details for evaluation job alerts. Specify this field if you want to receive email alerts if the evaluation job finds that your predictions have low mean average precision during a run. -
exampleCount
(type:integer()
, default:nil
) - Required. The maximum number of predictions to sample and save to BigQuery during each evaluation interval. This limit overridesexample_sample_percentage
: even if the service has not sampled enough predictions to fulfillexample_sample_perecentage
during an interval, it stops sampling predictions when it meets this limit. -
exampleSamplePercentage
(type:float()
, default:nil
) - Required. Fraction of predictions to sample and save to BigQuery during each evaluation interval. For example, 0.1 means 10% of predictions served by your model version get saved to BigQuery. -
humanAnnotationConfig
(type:GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1HumanAnnotationConfig.t
, default:nil
) - Optional. Details for human annotation of your data. If you set labelMissingGroundTruth totrue
for this evaluation job, then you must specify this field. If you plan to provide your own ground truth labels, then omit this field. Note that you must create an Instruction resource before you can specify this field. Provide the name of the instruction resource in theinstruction
field within this configuration. -
imageClassificationConfig
(type:GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1ImageClassificationConfig.t
, default:nil
) - Specify this field if your model version performs image classification or general classification.annotationSpecSet
in this configuration must match EvaluationJob.annotationSpecSet.allowMultiLabel
in this configuration must matchclassificationMetadata.isMultiLabel
in input_config. -
inputConfig
(type:GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1InputConfig.t
, default:nil
) - Rquired. Details for the sampled prediction input. Within this configuration, there are requirements for several fields:dataType
must be one ofIMAGE
,TEXT
, orGENERAL_DATA
.annotationType
must be one ofIMAGE_CLASSIFICATION_ANNOTATION
,TEXT_CLASSIFICATION_ANNOTATION
,GENERAL_CLASSIFICATION_ANNOTATION
, orIMAGE_BOUNDING_BOX_ANNOTATION
(image object detection). If your machine learning model performs classification, you must specifyclassificationMetadata.isMultiLabel
. You must specifybigquerySource
(notgcsSource
). -
textClassificationConfig
(type:GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1TextClassificationConfig.t
, default:nil
) - Specify this field if your model version performs text classification.annotationSpecSet
in this configuration must match EvaluationJob.annotationSpecSet.allowMultiLabel
in this configuration must matchclassificationMetadata.isMultiLabel
in input_config.
Summary
Functions
Unwrap a decoded JSON object into its complex fields.
Types
@type t() :: %GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1EvaluationJobConfig{ bigqueryImportKeys: map() | nil, boundingPolyConfig: GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1BoundingPolyConfig.t() | nil, evaluationConfig: GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1EvaluationConfig.t() | nil, evaluationJobAlertConfig: GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1EvaluationJobAlertConfig.t() | nil, exampleCount: integer() | nil, exampleSamplePercentage: float() | nil, humanAnnotationConfig: GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1HumanAnnotationConfig.t() | nil, imageClassificationConfig: GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1ImageClassificationConfig.t() | nil, inputConfig: GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1InputConfig.t() | nil, textClassificationConfig: GoogleApi.DataLabeling.V1beta1.Model.GoogleCloudDatalabelingV1beta1TextClassificationConfig.t() | nil }