Using the Diffo Provider Extension

Copy Markdown View Source
Run in Livebook 
Mix.install(
  [
    {:diffo, "~> 0.3.0"}
  ],
  config: [
    diffo: [ash_domains: [Diffo.Provider]]
  ],
  consolidate_protocols: false
)

Overview

Diffo is a Telecommunications Management Forum (TMF) Service and Resource Manager, built for autonomous networks.

It is implemented using the Ash Framework leveraging core and community extensions including some created and maintained by diffo-dev. As such it is highly customizable using Spark DSL and as necessary Elixir. If you are not already familiar with Ash then please explore Ash Get Started

First ensure you've explored the Diffo Livebook for an introduction to Diffo: Run in Livebook

In this livebook you will learn about:

  • TMF Services and Resources
  • Building your own Domain
  • Declaring Instance resources with the unified provider do DSL
  • Using the Assigner for partial resource allocation and assignment
  • Composing a Resource from partially assigned Resources
  • Declaring Party kinds with provider do
  • Declaring Place kinds with provider do

Installing Neo4j and Configuring Bolty

Diffo uses the Ash Neo4j DataLayer, which requires Neo4j to be installed.

While Neo4j community edition is open source and you can build from source it is likely that you'll use an installation.

AshNeo4j uses neo4j which must be installed and running. You can install latest major Neo4j versions from the community tab at Neo4j Deployment Center, or use the 5.26.8 direct link

When you install neo4j you'll typically have a default username and password. Take note of this and any other non-standard config.

Update the configuration below as necessary and evaluate.

config = [
  uri: "bolt://localhost:7687",
  auth: [username: "neo4j", password: "password"],
  user_agent: "diffoLivebook/1",
  pool_size: 15,
  max_overflow: 3,
  prefix: :default,
  name: Bolt,
  log: false,
  log_hex: false
]

Bolty needs a process in your supervision tree, this will start one with the config if not already running:

AshNeo4j.BoltyHelper.start(config)

Now you should be able to verify that Neo4j is running:

AshNeo4j.BoltyHelper.is_connected()

You can get all nodes related to other nodes the following query:

AshNeo4j.Cypher.run("MATCH (n1)-[r]->(n2) RETURN r, n1, n2 LIMIT 50")

It is helpful to have a Neo4j browser open locally, typically:

http://localhost:7474/browser/

Once you connect and issue a query like the one above you'll be able to explore the results interactively.

OPTIONAL If you want to clear your database you can evaluate:

AshNeo4j.Neo4jHelper.delete_all()

TMF Services and Resources

TMF Services are network services with industry standard structure and API that are operated for you by a Provider Entity. Ideally TMF Services are as abstract as possible, such that the Consumer specifies their intent (often by selecting a service from a catalog and providing minimal configuration of features and/or characteristics) allowing the provider to deliver the service as best it sees fit. This is powerful as it allows the service to perform advanced uses cases, like move, technology change, and allow the provider to optimise and even dynamically recompose the service. TMF Resources are generally a network resource that needs to be assigned to provide a service. They are generally too low level to have value on their own and where possible are entirely hidden from the product layer.

TMF Services are generally composed of services and/or resources. TMF Resources can also be composed of resources (but not services).

TMF Services and Resources are similar in that they each have a Specification, and are defined by Features and Characteristics. They also can have outgoing relationships with other services and resources, indeed this is fundamental to composition and in particular resource assignment.

Resources are generally created/managed/owned by a Provider, and assigned to a Consumer. Often the assignment is effectively a lease during which period the consumer has exclusive use of the resource under the provider's conditions, effectively 'owning' the resource.

When a Provider creates a pool of resources this is known as 'allocation'. For instance a VLAN pool may contain VLAN ID's 0..4095, and perhaps a new pool is inherently allocated with either a new interface, or the creation of a logical L2 VLAN domain.

When a Consumer is leased a resource this is assignment.

Assignment is effectively a request for a relationship from a Provider Resource 'back up' to a Consumer Service or Resource. There are different variants on this:

  • Specific Resource assignment - the specific resource requested by the Consumer is assigned
  • 'To specification' Resource assignment - an entire resource is assigned by the Provider, allocation may be 'just in time'
  • Partial Resource assignment - a partial resource is assigned by the Provider, the consumer is aware of the 'pool resource'.
  • Specific partial resource assignment - a partial resource requested by the Consumer is assigned

In all cases the assignment is only successful if the Provider allows the requested relationship to occur from it back to the Consumer.

Partial resource assignment uses a relationship characteristic to indicate which part of the resource is optionally requested and ultimately assigned.

Provider Extension

Diffo.Provider.BaseInstance is an Ash Resource Fragment for domain-specific Instance kinds (services and resources). It provides a rich set of base attributes — id, href, name, type, state and more — plus the unified Diffo.Provider.Extension DSL.

The extension provides a single provider do section containing everything needed to describe and wire an Instance kind. Declarations are baked into the module at compile time and introspectable at runtime via generated functions (specification/0, characteristics/0, features/0, parties/0, places/0) and Diffo.Provider.Extension.Info.

The provider do section contains:

specification do — the TMF Specification (id, name, type, version, description, category). The id is a stable UUID4, the same in every environment for this Instance kind.

characteristics do — typed value slots backed by Diffo.Provider.BaseCharacteristic-derived resources.

features do — optional capabilities with their own typed characteristic payload.

pools do — assignable pools for partial resource allocation. Each pool :name, :thing declaration creates an AssignableCharacteristic node during build and generates pools/0 / pool/1 on the module. Pool bounds (first, last, algorithm, assignable_type) are set in a :define action via Pool.update_pools/3. Assignment actions use Assigner.assign/3 — the thing name is looked up from the pool declaration.

parties do — party roles: party (singular), parties (plural), party_ref (reference, no direct edge).

places do — place roles: place (singular), places (plural), place_ref (reference).

behaviour do — declares which Ash create actions to wire for build lifecycle management. Declaring create :name injects :specified_by, :features, and :characteristics arguments automatically onto that action.

Each characteristic is a dedicated Ash resource using the Diffo.Provider.BaseCharacteristic fragment. It carries direct typed attributes and a :value calculation that builds a companion <Module>.Value TypedStruct for ordered JSON encoding. The TypedStruct uses AshJason.TypedStruct to control field order in the JSON output.

For partial resource allocation and assignment we've created Diffo.Provider.Assigner. The host resource declares a pools do section listing each assignable pool by name and the kind of thing being assigned. Pool bounds (first/last value, algorithm) are set via a :define action. Each assignment is stored as a Diffo.Provider.DefinedSimpleRelationship node carrying type: :assignedTo and a single NameValuePrimitive characteristic holding the thing name and assigned value. These are distinct from regular TMF Diffo.Provider.Relationship nodes and are accessible on an instance via instance.assignments.

Let's imagine a Compute domain which operates GPU and NPU resources. We want to expose a Cluster composite resource which can be dynamically composed of a number of GPU and NPU cores.

Each instance of Cluster could be created on Consumer demand as a 'container' for the GPU and NPU core partial resources.

Each of the GPU and NPU Resource instances is created and managed by the Provider and is effectively a resource pool for individually assignable cores.

We'll define all the resources first, then declare the Diffo.Compute domain once they are all compiled — Ash validates code_interface at domain compile time so all referenced resources must exist first.

Declaring a Composite Resource

We will start by declaring the Cluster Resource. It is going to be a composite resource, where it can be assigned individual GPU and NPU cores via resource relationships. It is an Ash.Resource incorporating the Diffo.Provider.BaseInstance fragment.

First we define the ClusterCharacteristic typed resource and its companion Value TypedStruct:

defmodule Diffo.Compute.ClusterCharacteristic do
  @moduledoc "Typed characteristic carrying cluster composition fields."
  use Ash.Resource,
    fragments: [Diffo.Provider.BaseCharacteristic],
    domain: Diffo.Compute

  resource do
    plural_name :cluster_characteristics
  end

  attributes do
    attribute :gpu_cores, :integer, public?: true, default: 0, constraints: [min: 0]
    attribute :npu_cores, :integer, public?: true, default: 0, constraints: [min: 0]
  end

  calculations do
    calculate :value, Diffo.Type.CharacteristicValue,
              Diffo.Provider.Calculations.CharacteristicValue do
      public? true
    end
  end

  actions do
    create :create do
      accept [:name, :gpu_cores, :npu_cores]
      argument :instance_id, :uuid
      argument :feature_id, :uuid
      change manage_relationship(:instance_id, :instance, type: :append)
      change manage_relationship(:feature_id, :feature, type: :append)
    end

    update :update do
      accept [:gpu_cores, :npu_cores]
    end
  end

  preparations do
    prepare build(load: [:value])
  end

  jason do
    pick [:name, :value]
    compact true
  end
end

defmodule Diffo.Compute.ClusterCharacteristic.Value do
  @moduledoc "Value struct for ClusterCharacteristic — controls JSON field order."
  use Ash.TypedStruct, extensions: [AshJason.TypedStruct]

  typed_struct do
    field :gpu_cores, :integer
    field :npu_cores, :integer
  end

  jason do
    pick [:gpu_cores, :npu_cores]
    compact true
  end
end

Now the Cluster resource itself. It declares ClusterCharacteristic as the :cluster characteristic — updates to it are made directly on the characteristic resource, so no update :define is needed here:

defmodule Diffo.Compute.Cluster do
  @moduledoc """
  Cluster Resource Instance
  """

  alias Diffo.Provider.BaseInstance
  alias Diffo.Provider.Instance.Relationship
  alias Diffo.Compute
  alias Diffo.Compute.ClusterCharacteristic
  alias Diffo.Compute.Tenant
  alias Diffo.Compute.Engineer

  use Ash.Resource,
    fragments: [BaseInstance],
    domain: Compute

  resource do
    description "An Ash Resource representing a Cluster"
    plural_name :Clusters
  end

  provider do
    specification do
      id "4bcfc4c9-e776-4878-a658-e8d81857bed7"
      name "cluster"
      type :resourceSpecification
      description "A Cluster Resource Instance"
      category "Network Resource"
    end

    characteristics do
      characteristic :cluster, ClusterCharacteristic
    end

    parties do
      party :operator, Tenant
      party :manager, Engineer
    end

    places do
      place :data_centre, Diffo.Compute.DataCentre
    end

    relationships do
      source :all
      target :all
    end

    behaviour do
      actions do
        create :build
      end
    end
  end

  actions do
    create :build do
      description "creates a new Cluster resource instance for build"
      accept [:id, :name, :type, :which]
      argument :relationships, {:array, :struct}
      argument :places, {:array, :struct}
      argument :parties, {:array, :struct}

      change set_attribute(:type, :resource)
      change load [:href]
      upsert? false
    end

    update :relate do
      description "relates the cluster with other instances"
      argument :relationships, {:array, :struct}

      change after_action(fn changeset, result, _context ->
               with {:ok, _cluster} <- Relationship.relate_instance(result, changeset),
                    {:ok, cluster} <- Compute.get_cluster_by_id(result.id),
                    do: {:ok, cluster}
             end)
    end
  end
end

Using the Assigner

We'll now define a GPU Resource which uses the Diffo.Provider.Assigner functionality.

First define the GpuCharacteristic typed resource and its Value TypedStruct:

defmodule Diffo.Compute.GpuCharacteristic do
  @moduledoc "Typed characteristic carrying GPU identity fields."
  use Ash.Resource,
    fragments: [Diffo.Provider.BaseCharacteristic],
    domain: Diffo.Compute

  resource do
    plural_name :gpu_characteristics
  end

  attributes do
    attribute :family, :atom, public?: true, description: "the GPU family name"
    attribute :model, :string, public?: true, description: "the GPU model name"
    attribute :technology, :atom, public?: true, description: "the GPU technology"
  end

  calculations do
    calculate :value, Diffo.Type.CharacteristicValue,
              Diffo.Provider.Calculations.CharacteristicValue do
      public? true
    end
  end

  actions do
    create :create do
      accept [:name, :family, :model, :technology]
      argument :instance_id, :uuid
      argument :feature_id, :uuid
      change manage_relationship(:instance_id, :instance, type: :append)
      change manage_relationship(:feature_id, :feature, type: :append)
    end

    update :update do
      accept [:family, :model, :technology]
    end
  end

  preparations do
    prepare build(load: [:value])
  end

  jason do
    pick [:name, :value]
    compact true
  end
end

defmodule Diffo.Compute.GpuCharacteristic.Value do
  @moduledoc "Value struct for GpuCharacteristic — controls JSON field order."
  use Ash.TypedStruct, extensions: [AshJason.TypedStruct]

  typed_struct do
    field :family, :atom
    field :model, :string
    field :technology, :atom
  end

  jason do
    pick [:family, :model, :technology]
    compact true
  end
end

The GPU resource declares GpuCharacteristic for the typed :gpu slot and uses pools do to declare the :cores assignable pool. The update :define action updates both the typed characteristic and the pool bounds. The update :assign_core action uses Assigner.assign/3 — the thing name (:core) is looked up from the pool declaration automatically:

defmodule Diffo.Compute.GPU do
  @moduledoc """
  GPU Resource Instance
  """

  alias Diffo.Provider.BaseInstance
  alias Diffo.Provider.Instance.Relationship
  alias Diffo.Provider.Extension.Characteristic
  alias Diffo.Provider.Extension.Pool
  alias Diffo.Provider.Assigner
  alias Diffo.Provider.Assignment
  alias Diffo.Compute
  alias Diffo.Compute.GpuCharacteristic

  use Ash.Resource,
    fragments: [BaseInstance],
    domain: Compute

  resource do
    description "An Ash Resource representing a GPU"
    plural_name :gpus
  end

  provider do
    specification do
      id "ad50073f-17e0-45cb-b9b1-aa4296876156"
      name "gpu"
      type :resourceSpecification
      description "A GPU Resource Instance"
      category "Network Resource"
    end

    characteristics do
      characteristic :gpu, GpuCharacteristic
    end

    pools do
      pool :cores, :core
    end

    relationships do
      source :all
      target :all
    end

    behaviour do
      actions do
        create :build
      end
    end
  end

  actions do
    create :build do
      description "creates a new GPU resource instance for build"
      accept [:id, :name, :type, :which]
      argument :relationships, {:array, :struct}
      argument :places, {:array, :struct}
      argument :parties, {:array, :struct}

      change set_attribute(:type, :resource)
      change load [:href]
      upsert? false
    end

    update :define do
      description "sets GPU identity and allocates the cores pool"
      argument :characteristic_value_updates, {:array, :term}

      change after_action(fn changeset, result, _context ->
               with {:ok, result} <-
                      Characteristic.update_all(result, changeset, characteristics()),
                    {:ok, result} <- Pool.update_pools(result, changeset, pools()),
                    {:ok, result} <- Compute.get_gpu_by_id(result.id),
                    do: {:ok, result}
             end)
    end

    update :relate do
      description "relates the GPU with other instances"
      argument :relationships, {:array, :struct}

      change after_action(fn changeset, result, _context ->
               with {:ok, result} <- Relationship.relate_instance(result, changeset),
                    {:ok, result} <- Compute.get_gpu_by_id(result.id),
                    do: {:ok, result}
             end)
    end

    update :assign_core do
      description "assigns a core from this GPU to another instance"
      argument :assignment, :struct, constraints: [instance_of: Assignment]

      change after_action(fn changeset, result, _context ->
               with {:ok, result} <- Assigner.assign(result, changeset, :cores),
                    {:ok, result} <- Compute.get_gpu_by_id(result.id),
                    do: {:ok, result}
             end)
    end
  end
end

Aliases, Inherited DSL, and Field Calculations

Aliases on assignment slots

Every AssignmentRelationship carries an optional :alias — an atom given to a slot by the consuming (target) side before or when the assignment is bound. Think of it as a stable name for the slot: the consumer says "I have a slot called :primary_gpu", and the producer assigns into it carrying alias: :primary_gpu. The alias never changes, even if the assignment is recreated.

Pass the alias via Assignment.alias when assigning:

# Assign a core from gpu_1 into cluster_1's :primary_gpu slot
assignment = %{
  assignment: %Assignment{
    assignee_id: cluster_1.id,
    operation: :auto_assign,
    alias: :primary_gpu
  }
}
gpu_1 = Compute.assign_gpu_core!(gpu_1, assignment)

The identity constraint [:target_id, :alias] on AssignmentRelationship guarantees at most one assignment per (cluster, alias) pair — the :primary_gpu slot can only hold one assignment at a time.

Inheriting a place from an assigned resource

A service or resource can declare that it inherits a place from the instance that assigned something to it — without creating its own PlaceRef edge. The inherited_place DSL entity in places do generates an Ash calculation that traverses the assignment graph at read time.

In our Compute example: if a GPU instance has a :data_centre place, and a Cluster wants to surface the data centre of its primary GPU, it can declare:

provider do
  places do
    # Traverses AssignmentRelationship where alias = :primary_gpu,
    # reads PlaceRef with role :data_centre from the source GPU instance.
    inherited_place :primary_data_centre, via: [:primary_gpu], source_role: :data_centre
  end
end

Load it like any other calculation:

cluster = Ash.load!(cluster_1, [:primary_data_centre], domain: Compute)
# cluster.primary_data_centre => [%DataCentre{...}]

inherited_party works identically for party inheritance:

# Cluster inherits the operating Tenant from the GPU it was assigned from
provider do
  parties do
    inherited_party :operator, via: [:primary_gpu], source_role: :operator
  end
end

Reading fields from the assignment graph

Three calculation modules handle common traversal patterns. All return lists.

FieldFromAssignment — reads a field directly from the AssignmentRelationship record. Use it for values that live on the relationship itself: :value, :pool, :thing, :alias.

# Core number assigned to this cluster under the :primary_gpu slot
calculate :primary_core, {:array, :integer},
  {Diffo.Provider.Calculations.FieldFromAssignment, [alias: :primary_gpu, field: :value]}

FieldViaAssignedRelationship — traverses assignment in reverse (cluster → GPU) and reads a field from the source instance. Use it for fields that live on the assigning resource, not the relationship.

# Name of the GPU holding the :primary_gpu slot on this cluster
calculate :primary_gpu_name, {:array, :string},
  {Diffo.Provider.Calculations.FieldViaAssignedRelationship,
   [via: [:primary_gpu], field: :name]}

FieldViaRelationship — traverses DefinedSimpleRelationship in the forward direction (source → target) filtered by alias: and/or type:. Use it when this instance is the source of a named forward relationship.

# Name of the downstream node this GPU provides to
calculate :downstream_name, {:array, :string},
  {Diffo.Provider.Calculations.FieldViaRelationship,
   [type: :assignedTo, alias: :downstream, field: :name]}
I want…Use
Value on the assignment record (:value, :pool)FieldFromAssignment
Field from the instance that assigned to meFieldViaAssignedRelationship
Field from an instance I have a forward relationship toFieldViaRelationship
Place/party inherited via assignmentinherited_place / inherited_party

Party Extension

Diffo.Provider.BaseParty is an Ash Resource Fragment for domain-specific Party kinds, mirroring BaseInstance. It provides common Party attributes — id, href, name, type, referred_type — and the unified Diffo.Provider.Extension DSL. Within provider do, a Party kind uses instances do, parties do, and places do sections to declare the roles it plays.

type defaults to :PartyRef and can be set to :Individual, :Organization, or :Entity. Domain party kinds typically set type in their build action. The id defaults to a generated uuid but can be set to any meaningful string (such as an ABN or a data centre identifier).

The Diffo.Provider.Extension DSL cheat sheet is at DSL-Diffo.Provider.Extension.

Defining Party kinds

We'll add two Party kinds to our Compute domain — Tenant for the operating company, and Engineer for the individuals who manage resources.

defmodule Diffo.Compute.Tenant do
  @moduledoc """
  Tenant in the Compute domain
  """

  alias Diffo.Provider.BaseParty
  alias Diffo.Compute

  use Ash.Resource,
    fragments: [BaseParty],
    domain: Compute

  resource do
    description "A Compute Tenant"
    plural_name :tenants
  end

  actions do
    create :build do
      accept [:id, :name]
      change set_attribute(:type, :Organization)
    end
  end

  provider do
    instances do
      role :operator, Diffo.Compute.Cluster
      role :operator, Diffo.Compute.GPU
    end
  end
end
defmodule Diffo.Compute.Engineer do
  @moduledoc """
  Engineer in the Compute domain
  """

  alias Diffo.Provider.BaseParty
  alias Diffo.Compute

  use Ash.Resource,
    fragments: [BaseParty],
    domain: Compute

  resource do
    description "A Compute Engineer"
    plural_name :engineers
  end

  actions do
    create :build do
      accept [:id, :name]
      change set_attribute(:type, :Individual)
    end
  end

  provider do
    instances do
      role :manager, Diffo.Compute.Cluster
    end
    parties do
      role :employer, Diffo.Compute.Tenant
    end
  end
end

Place Extension

Diffo.Provider.BasePlace is an Ash Resource Fragment for domain-specific Place kinds, mirroring BaseInstance and BaseParty. It provides common Place attributes — id, href, name, type, referred_type — and the unified Diffo.Provider.Extension DSL. Within provider do, a Place kind uses instances do, parties do, and places do sections to declare the roles it plays.

type defaults to :PlaceRef and is typically set in the build action to the concrete place type (:GeographicSite, :GeographicLocation, or :GeographicAddress). When referred_type is present, type must be :PlaceRef — meaning this Place is a reference rather than a physical location.

Defining Place kinds

We'll add a DataCentre Place kind to our Compute domain. Clusters are hosted at a data centre; the instances do block records that relationship from the DataCentre's perspective.

defmodule Diffo.Compute.DataCentre do
  @moduledoc """
  DataCentre in the Compute domain
  """

  alias Diffo.Provider.BasePlace
  alias Diffo.Compute

  use Ash.Resource,
    fragments: [BasePlace],
    domain: Compute

  resource do
    description "A Compute Data Centre"
    plural_name :data_centres
  end

  jason do
    pick [:id, :href, :name, :type]
    compact true
    rename type: "@type"
  end

  outstanding do
    expect [:id, :name, :type]
  end

  actions do
    create :build do
      accept [:id, :href, :name]
      change set_attribute(:type, :GeographicSite)
    end
  end

  provider do
    instances do
      role :data_centre, Diffo.Compute.Cluster
      role :data_centre, Diffo.Compute.GPU
    end
  end
end

Compute Domain

With all resources defined we can now declare the Diffo.Compute domain, which exposes a typed API for each resource:

defmodule Diffo.Compute do
  @moduledoc """
  Compute - example domain
  """
  use Ash.Domain,
    otp_app: :diffo,
    validate_config_inclusion?: false

  alias Diffo.Compute.GPU
  alias Diffo.Compute.GpuCharacteristic
  #alias Diffo.Compute.NPU
  alias Diffo.Compute.Cluster
  alias Diffo.Compute.ClusterCharacteristic
  alias Diffo.Compute.Tenant
  alias Diffo.Compute.Engineer
  alias Diffo.Compute.DataCentre

  resources do
    resource GPU do
      define :get_gpu_by_id, action: :read, get_by: :id
      define :build_gpu, action: :build
      define :define_gpu, action: :define
      define :relate_gpu, action: :relate
      define :assign_gpu_core, action: :assign_core
    end

    resource GpuCharacteristic do
      define :update_gpu_characteristic, action: :update
    end

    #resource NPU do
      #define :get_npu_by_id, action: :read, get_by: :id
      #define :build_npu, action: :build
      #define :define_npu, action: :define
      #define :relate_npu, action: :relate
      #define :assign_npu_core, action: :assign_core
    #end

    resource Cluster do
      define :get_cluster_by_id, action: :read, get_by: :id
      define :build_cluster, action: :build
      define :relate_cluster, action: :relate
    end

    resource ClusterCharacteristic do
      define :update_cluster_characteristic, action: :update
    end

    resource Tenant do
      define :create_tenant, action: :build
      define :get_tenant_by_id, action: :read, get_by: :id
      define :list_tenants, action: :read
    end

    resource Engineer do
      define :create_engineer, action: :build
      define :get_engineer_by_id, action: :read, get_by: :id
      define :list_engineers, action: :read
    end

    resource DataCentre do
      define :create_data_centre, action: :build
      define :get_data_centre_by_id, action: :read, get_by: :id
    end
  end
end

Creating Party instances

Clear any data from previous runs before starting (safe to re-evaluate):

AshNeo4j.Neo4jHelper.delete_all()

Now the domain is defined we'll create our Tenant and Engineer first — we'll need them when building Cluster instances. The id for the Tenant is set to a meaningful string — the company's ABN.

alias Diffo.Compute
alias Diffo.Provider.Instance.Party

{:ok, tenant} = Compute.create_tenant(%{
  id: "51824753556",
  name: "Acme Compute Pty Ltd"
})

{:ok, engineer} = Compute.create_engineer(%{
  name: "Alice Zhang"
})

Creating a Cluster

First we create the data centre — our DataCentre resource uses BasePlace, so it is managed via the Compute domain API like any other domain resource:

alias Diffo.Provider.Instance.Place

{:ok, dc} = Compute.create_data_centre(%{id: "NXTM2", name: "NextDC M2"})

Now build the cluster, passing the data centre as a place and our party members by id and role:

places = [%Place{id: dc.id, role: :data_centre}]
parties = [
  %Party{id: tenant.id, role: :operator},
  %Party{id: engineer.id, role: :manager}
]
cluster_1 = Diffo.Compute.build_cluster!(%{name: "cluster_1", places: places, parties: parties})
Jason.encode!(cluster_1, pretty: true) |> IO.puts

Using the Assigner

Now we'll create a couple of GPU instances:

gpu_1 = Compute.build_gpu!(%{name: "GPU 1"})
gpu_2 = Compute.build_gpu!(%{name: "GPU 2"})

We define each GPU: setting its typed :gpu characteristic fields and allocating the :cores pool bounds. Both are passed via characteristic_value_updates to the :define action — Characteristic.update_all handles the typed :gpu update and Pool.update_pools handles the :cores pool bounds:

gpu_attrs = [
  gpu: [family: :nvidia, model: "GeForce RTX5090", technology: :blackwell],
  cores: [first: 1, last: 680, assignable_type: "tensor"]
]

gpu_1 = Compute.define_gpu!(gpu_1, %{characteristic_value_updates: gpu_attrs})
gpu_2 = Compute.define_gpu!(gpu_2, %{characteristic_value_updates: gpu_attrs})

The :cores pool is backed by an AssignableCharacteristic node that records the range bounds and algorithm. Free cores are computed at assignment time from the count of existing AssignmentRelationship records — there is no stored free counter. We can render one as json:

Jason.encode!(gpu_1, pretty: true) |> IO.puts

Composing a Resource from partially assigned Resources

Now we can auto-assign GPU cores from each GPU to our cluster_1. We'll assign 3 cores from gpu_1, and one from gpu_2.

alias Diffo.Provider.Assignment

assignment = %{assignment: %Assignment{assignee_id: cluster_1.id, operation: :auto_assign}}
gpu_1 = Compute.assign_gpu_core!(gpu_1, assignment)
gpu_1 = Compute.assign_gpu_core!(gpu_1, assignment)
gpu_1 = Compute.assign_gpu_core!(gpu_1, assignment)
gpu_2 = Compute.assign_gpu_core!(gpu_2, assignment)

Now our cluster should have a core from each GPU. Check in the Neo4j browser for :DefinedSimpleRelationship nodes from gpu_1 and gpu_2 to cluster_1. There should be four — each has type: :assignedTo and a single characteristic carrying the thing name (:core) and the assigned integer value (e.g. 1, 2, 3 from gpu_1 and 1 from gpu_2).

The GPU's assignments hold each assignment, showing the assigned core number in the JSON encoding as a resourceRelationshipCharacteristic:

Jason.encode!(gpu_1, pretty: true) |> IO.puts

Make sure you have a look at it in the neo4j browser. There should be :DefinedSimpleRelationship nodes from each GPU resource instance to the cluster_1 resource instance, each carrying the assigned core number. There is no central assignment table — the DefinedSimpleRelationship nodes ARE the assignments. They are separate from the regular :Relationship nodes used for TMF service/resource relationships, and are accessible in Elixir via instance.assignments.

As an exercise, clone the GPU resource to create an NPU resource and assign some NPU cores from it to your cluster. Check that the assigned NPU cores are unique.

What happens when there are none left to assign? What happens when I request a specific assignment from an instance to which the partial resource is already assigned?

What Next?

In this tutorial you've used Diffo's unified provider do extension to define a Compute domain with:

  • A composite Cluster resource that receives GPU cores via Diffo.Provider.Assigner
  • A GPU resource using pools do to declare the :cores assignable pool — pool :cores, :core replaces the old characteristic :cores, AssignableValue pattern
  • Assignments stored as Diffo.Provider.DefinedSimpleRelationship records with type: :assignedTo (distinct from TMF Relationship nodes); accessible via instance.assignments
  • Tenant and Engineer Party kinds declared with provider do that express which instances they operate and manage
  • A DataCentre Place kind that declares the instances located at it

BaseParty and BasePlace follow the same provider do pattern as BaseInstance — domain-specific resources use them as fragments, write their own actions for domain-specific attributes, and declare their roles via the unified DSL sections.

The full DSL reference is at DSL-Diffo.Provider.Extension.

If you find Diffo useful please visit and star on github. Feel free to join discussions and raise issues to discuss PR's.