google_api_dataflow v0.21.0 GoogleApi.Dataflow.V1b3.Model.Environment View Source
Describes the environment in which a Dataflow Job runs.
Attributes
clusterManagerApiService
(type:String.t
, default:nil
) - The type of cluster manager API to use. If unknown or unspecified, the service will attempt to choose a reasonable default. This should be in the form of the API service name, e.g. "compute.googleapis.com".dataset
(type:String.t
, default:nil
) - The dataset for the current project where various workflow related tables are stored.The supported resource type is:
Google BigQuery: bigquery.googleapis.com/{dataset}
experiments
(type:list(String.t)
, default:nil
) - The list of experiments to enable.flexResourceSchedulingGoal
(type:String.t
, default:nil
) - Which Flexible Resource Scheduling mode to run in.internalExperiments
(type:map()
, default:nil
) - Experimental settings.sdkPipelineOptions
(type:map()
, default:nil
) - The Cloud Dataflow SDK pipeline options specified by the user. These options are passed through the service and are used to recreate the SDK pipeline options on the worker in a language agnostic and platform independent way.serviceAccountEmail
(type:String.t
, default:nil
) - Identity to run virtual machines as. Defaults to the default account.serviceKmsKeyName
(type:String.t
, default:nil
) - If set, contains the Cloud KMS key identifier used to encrypt data at rest, AKA a Customer Managed Encryption Key (CMEK).Format: projects/PROJECT_ID/locations/LOCATION/keyRings/KEY_RING/cryptoKeys/KEY
tempStoragePrefix
(type:String.t
, default:nil
) - The prefix of the resources the system should use for temporary storage. The system will append the suffix "/temp-{JOBNAME} to this resource prefix, where {JOBNAME} is the value of the job_name field. The resulting bucket and object prefix is used as the prefix of the resources used to store temporary data needed during the job execution. NOTE: This will override the value in taskrunner_settings. The supported resource type is:Google Cloud Storage:
storage.googleapis.com/{bucket}/{object} bucket.storage.googleapis.com/{object}
userAgent
(type:map()
, default:nil
) - A description of the process that generated the request.version
(type:map()
, default:nil
) - A structure describing which components and their versions of the service are required in order to run the job.workerPools
(type:list(GoogleApi.Dataflow.V1b3.Model.WorkerPool.t)
, default:nil
) - The worker pools. At least one "harness" worker pool must be specified in order for the job to have workers.workerRegion
(type:String.t
, default:nil
) - The Compute Engine region (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1". Mutually exclusive with worker_zone. If neither worker_region nor worker_zone is specified, default to the control plane's region.workerZone
(type:String.t
, default:nil
) - The Compute Engine zone (https://cloud.google.com/compute/docs/regions-zones/regions-zones) in which worker processing should occur, e.g. "us-west1-a". Mutually exclusive with worker_region. If neither worker_region nor worker_zone is specified, a zone in the control plane's region is chosen based on available capacity.
Link to this section Summary
Functions
Unwrap a decoded JSON object into its complex fields.
Link to this section Types
t()
View Sourcet() :: %GoogleApi.Dataflow.V1b3.Model.Environment{ clusterManagerApiService: String.t(), dataset: String.t(), experiments: [String.t()], flexResourceSchedulingGoal: String.t(), internalExperiments: map(), sdkPipelineOptions: map(), serviceAccountEmail: String.t(), serviceKmsKeyName: String.t(), tempStoragePrefix: String.t(), userAgent: map(), version: map(), workerPools: [GoogleApi.Dataflow.V1b3.Model.WorkerPool.t()], workerRegion: String.t(), workerZone: String.t() }
Link to this section Functions
Unwrap a decoded JSON object into its complex fields.