A spec in several files#

In this tutorial you build the specification of an optimisation model (a spec) for dispatch from files that each contain one part of the spec. You merge them into one spec, and then add a component without changing the files you already have. This tutorial takes about 30 minutes. Do your first spec first.

Each file here is one part that merge composes (a fragment).

The network#

Make a file network.yaml, with a constraint that balances every bus and an objective that minimises the total cost. The file lists the injection and the total cost as names used but not declared here (under given:). Each file you make next adds a part of a sum (a term) to the injection and to the total cost.

description: Every bus is balanced in every snapshot.

given:
  expressions:
    injection:
      dims: [snapshot, bus]
      description: what the components put into a bus, less what they take out
    total_cost:
      dims: []
      description: what running the system costs

dimensions:
  snapshot: { dtype: int, description: dispatch periods }
  bus: { description: network nodes }

constraints:
  balance:
    dims: [snapshot, bus]
    expression: injection == 0

objective:
  sense: minimize
  expression: total_cost

Check the file:

python -m mathspec check network.yaml

import mathspec as ms

for note in ms.advice('network.yaml'):
    print(note)

expression 'injection' is read here and declared elsewhere: the model this one is layered onto provides it. A tool refuses the spec where that model does not, over the same dims. merge() replaces this entry with the one another file declares, or builds it from the terms other files add.
expression 'total_cost' is read here and declared elsewhere: the model this one is layered onto provides it. A tool refuses the spec where that model does not, over the same dims. merge() replaces this entry with the one another file declares, or builds it from the terms other files add.

mathspec accepts the file. check prints one remark that is not a refusal (a line of advice) for each name the file lists under given:.

Keep this Python session open. Later steps use ms.

The generators#

Make a file generators.yaml. The file declares the shared dimensions again, with only the type of their labels (a dtype). The adds_to: keys of the file make generation a term of the injection and generation_cost a term of the total cost. gen_bus is a table of label pairs (a relation) from each generator to its bus. generation uses gen_bus to sum the dispatch by bus.

description: A generator fleet, each unit on one bus.

given:
  expressions:
    injection: { dims: [snapshot, bus] }
    total_cost: { dims: [] }

dimensions:
  snapshot: { dtype: int }
  bus: { dtype: str }
  generator: { description: generating units }

relations:
  gen_bus: { key: generator, values: bus, description: the bus a generator sits on }

parameters:
  capacity: { dims: [generator], description: installed capacity }
  cost: { dims: [generator], description: marginal cost }

variables:
  dispatch:
    description: output of a generator in a snapshot
    dims: [snapshot, generator]
    bounds: { lower: 0, upper: capacity }

expressions:
  generation:
    description: what the fleet puts into a bus
    expression: sum(dispatch, over=generator, by=gen_bus[bus])
    adds_to: injection
  generation_cost:
    description: what running the fleet costs
    expression: sum(dispatch * cost)
    adds_to: total_cost

Check the file:

python -m mathspec check generators.yaml

for note in ms.advice('generators.yaml'):
    print(note)

expression 'injection' is read here, and this file adds a term to it. merge() sums the term with the terms other files add under that name. Until then, the program reads it and does not build it.
expression 'total_cost' is read here, and this file adds a term to it. merge() sums the term with the terms other files add under that name. Until then, the program reads it and does not build it.

mathspec accepts the file, and prints one line of advice for each sum that the file adds a term to.

Use markdown or to_markdown to print the math of generators.yaml on its own:

python -m mathspec markdown --no-legend generators.yaml

print(ms.to_markdown('generators.yaml', legend=False))

Each term appears as its own definition. The sum in generation covers the generators on bus \(b\), the \(g\) where \(\mathrm{gen\_bus}(g) = b\). mathspec prints no objective, because the file sets none.

The loads#

Make a file loads.yaml. Its term, consumption, takes the demand out of each bus:

description: The demand at every bus.

given:
  expressions:
    injection: { dims: [snapshot, bus] }

dimensions:
  snapshot: { dtype: int }
  bus: { dtype: str }

parameters:
  demand: { dims: [snapshot, bus], description: demand to be met }

expressions:
  consumption:
    description: what the loads take out of a bus
    expression: -demand
    adds_to: injection

Check the file:

python -m mathspec check loads.yaml

for note in ms.advice('loads.yaml'):
    print(note)

expression 'injection' is read here, and this file adds a term to it. merge() sums the term with the terms other files add under that name. Until then, the program reads it and does not build it.

mathspec accepts the file, with one line of advice, for the injection.

Merging the files#

merge has no command, so you merge the files in Python. If you used the command line so far, start Python in the folder that contains the files. Merge the three files, and give them as a list:

import mathspec as ms

spec = ms.merge(['network.yaml', 'generators.yaml', 'loads.yaml'])
print(spec.expressions['injection'].expression)
print(spec.dimensions['snapshot'].description)

generation + consumption
dispatch periods

merge writes the injection as the sum of the two terms by name, in the order of the list. In the merged spec, each term stays a reusable expression with a name (a named expression). snapshot has the one description written for it, which is the description in network.yaml.

Print the math of the merged spec:

print(ms.to_markdown(spec, legend=False))

The merged spec defines injection and total_cost from their terms, and the objective and the balance constraint of the network use these two sums.

The imports#

Make a file imports.yaml. It adds a term, purchase, to the injection and a term, import_cost, to the total cost:

description: Power bought from outside the network, at a price.

given:
  expressions:
    injection: { dims: [snapshot, bus] }
    total_cost: { dims: [] }

dimensions:
  snapshot: { dtype: int }
  bus: { dtype: str }

parameters:
  import_limit: { dims: [bus], description: most a bus can import }
  import_price: { dims: [snapshot], description: price of imported power }

variables:
  imported:
    description: power a bus imports in a snapshot
    dims: [snapshot, bus]
    bounds: { lower: 0, upper: import_limit }

expressions:
  purchase:
    description: what the imports put into a bus
    expression: imported
    adds_to: injection
  import_cost:
    description: what the imports cost
    expression: sum(imported * import_price)
    adds_to: total_cost

Check the file:

python -m mathspec check imports.yaml

for note in ms.advice('imports.yaml'):
    print(note)

expression 'injection' is read here, and this file adds a term to it. merge() sums the term with the terms other files add under that name. Until then, the program reads it and does not build it.
expression 'total_cost' is read here, and this file adds a term to it. merge() sums the term with the terms other files add under that name. Until then, the program reads it and does not build it.

mathspec accepts the file, with the same advice as for the generators.

In Python, merge the four files:

spec = ms.merge(['network.yaml', 'generators.yaml', 'loads.yaml', 'imports.yaml'])
print(spec.expressions['injection'].expression)
print(spec.expressions['total_cost'].expression)

generation + consumption + purchase
generation_cost + import_cost

The injection has a third term at the end, and the total cost has a second term, although network.yaml is unchanged. The cost of the imports is a term, because merge takes the objective from one file only.

The emission cap#

Make a file emissions.yaml. Its constraint caps what the fleet emits, and the file lists dispatch under given: instead of declaring it:

description: A cap on what the fleet emits over the horizon.

given:
  variables:
    dispatch: { dims: [snapshot, generator] }

dimensions:
  snapshot: { dtype: int }
  generator: { dtype: str }

parameters:
  emission_rate: { dims: [generator], description: emissions per unit of output }
  emission_cap: { dims: [], description: most the fleet may emit }

constraints:
  emission_limit:
    dims: []
    expression: sum(dispatch * emission_rate) <= emission_cap

Check the file:

python -m mathspec check emissions.yaml

for note in ms.advice('emissions.yaml'):
    print(note)

variable 'dispatch' is read here and declared elsewhere: the model this one is layered onto provides it. A tool refuses the spec where that model does not, over the same dims. merge() replaces this entry with the one another file declares.

mathspec accepts the file, and advises that another file must declare dispatch.

In Python, merge all five files:

spec = ms.merge(['network.yaml', 'generators.yaml', 'loads.yaml', 'imports.yaml', 'emissions.yaml'])
print(sorted(spec.constraints))
print(bool(spec.program.given))

['balance', 'emission_limit']
False

The merged spec has both constraints. The cap now uses the dispatch that generators.yaml declares. spec.program is the merged spec, every name resolved (its program). spec.program.given is empty, so no name is left for a file outside these five files to provide.

Leaving the network out#

Merge the generators and the loads without the network:

ms.merge(['generators.yaml', 'loads.yaml'])

fragments 'generators.yaml' and 'loads.yaml' add a term to 'injection', and no other fragment reads it. Add the fragment that reads 'injection', or fix the spelling under 'given:'.

merge refuses the list, because a term needs a file that lists its sum under given:.

What you built#

You have built a dispatch spec from five files. The network uses two sums, three component files add terms to those sums, and one file uses a variable that another declares. You added the imports and the emission cap without changing network.yaml.

Where to next#

• Compose a spec from several files lists the steps for merge and for override, which lays a file of changes (a patch) over a spec.