Skip to content

Operators#

This page describes the four operators of the expression grammar, sum, at, shift and sum_back, and what mathspec refuses in each call.

An operator is a construct built into the language, such as sum (operator). The four operators below act on arithmetic expressions, and count, position and the comparisons act in a condition.

A reported expression may also call dual, which reads the dual of a constraint after the solver runs.

A file cannot add an operator, and only a change to mathspec adds one. A macro can combine the operators that exist.

Operator Result
sum(array) sum removes every dimension of array. The result is a scalar
sum(array, over=dim) sum removes dim
sum(array, over=[a, …]) sum removes each dimension in the list
sum(array, over=dim, by=relation[c]) sum removes dim, and groups the result by the column c of the relation. The call joins on the other key columns, so array and the result are indexed over them
sum(array, over=[dim, …], by=relation[c, …]) The call does the same with several dimensions and several columns of one relation, joins on them together, and groups by their product
at(array, by=relation[c]) The result is array read at the value of the relation's column c, once per key of the relation. The result gains the key
shift(array, along=dim, offset=n) The result is the value n positions earlier along dim, and the vacated edge is absent
shift(array, along=dim, offset=n, edge='wrap') The result is the value n positions earlier, counted cyclically, so nothing is vacated
shift(array, along=dim, offset=n, edge=v) The result is the value n positions earlier, with the number v at the vacated edge
shift(array, along=dim, offset=p, edge=…) The result is the value p positions earlier, with p an integer parameter read per entity
shift(array, along=dim, offset=n, within=relation[c]) The result is the value n positions earlier inside each group that the relation's column c makes. Neighbours, edges and a wrap all belong to that group
sum_back(array, along=dim, window=n) The result is the sum of the last n positions along dim, ending at the position being written
sum_back(array, along=dim, window=p) The result is the sum of the last p positions, with p an integer parameter read per entity
sum_back(array, along=dim, window=p, edge='wrap') Instead of stopping short at its start, the window reaches around the dimension
sum_back(array, along=dim, window=n, within=relation[c]) The window stays inside each group that the relation's column c makes

array is any expression with the right dimensions, such as a parameter or a variable. Every operator as math lists the typeset form of each row of the table.

sum#

sum adds the values of an expression along one or more dimensions, and removes those dimensions from the result. Each row of the result is the sum of the values that share its coordinates, and no value moves between rows. array must be indexed over each dimension in over=, and a list contains each dimension once. sum(x, over=[a, b]) is the same as sum(sum(x, over=a), over=b), and the typesetter prints it as one sum over both sets.

With by=, the sum adds up each group that a column of a relation makes, and the result is indexed over that column. The balance at each bus (a nodal balance) is one sum(by=) per kind of component:

dimensions:
  bus: { dtype: str }
  generator: { dtype: str }
  line: { dtype: str }
relations:
  gen_bus: { key: generator, values: bus }
  line_from: { key: line, values: bus }
  line_to: { key: line, values: bus }
parameters:
  load: { dims: [bus] }
  cost: { dims: [generator] }
variables:
  p: { dims: [generator], bounds: { lower: 0 } }
  f: { dims: [line] }
constraints:
  nodal_balance:
    dims: [bus]
    expression: >-
      sum(p, over=generator, by=gen_bus[bus])
      + sum(f, over=line, by=line_to[bus])
      - sum(f, over=line, by=line_from[bus])
      == load
objective: { sense: minimize, expression: "sum(p * cost, over=generator)" }
\[ \sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bus}(g) = b} p_{g} + \sum_{l \in \mathcal{L} \,:\, \mathrm{line\_to}(l) = b} f_{l} - \left( \sum_{l \in \mathcal{L} \,:\, \mathrm{line\_from}(l) = b} f_{l} \right) = \mathrm{load}_{b} \qquad \forall\, b \in \mathcal{B} \]
A nodal balance sums each kind of component by bus The relation gen_bus puts the generators wind and gas on bus north, and solar on bus south. The relations line_from and line_to give the line l1 from north to south. The constraint nodal_balance has one row per bus. At north it adds p of wind and p of gas, and subtracts f of l1. At south it adds p of solar and f of l1. gen_bus a relation generator bus windnorth gasnorth solarsouth line_from, line_to two relations line from to l1northsouth sum(…, by=…[bus]) one sum per bus nodal_balance a constraint bus left side northp[wind] + p[gas] − f[l1] southp[solar] + f[l1]

With the data in the figure, the sum through gen_bus adds p of wind and gas at bus north. The line l1 leaves north and enters south. Each sum removes generator or line and adds bus, so the constraint is indexed over bus alone.

At load, mathspec refuses sum(x) over an expression that has no dimension. With a parameter total: { dims: [] } added to the spec above, and the constraint r: { dims: [], expression: "sum(p) <= sum(total)" }, mathspec prints:

Constraint 'r': sum() has no over= or by=, and its operand has no dimensions to sum over. Remove the sum.

The scalar total has nothing to add up, so mathspec suggests the rewrite: no sum.

See also

  • How a relation is used, for the dimensions that a call through a relation joins on, and those its result has.

at#

at takes, for each key label of a relation, the value of an expression at the label that the relation maps it to. at adds nothing up (a lookup), and takes one row for each key label. Here at takes, for every line, the build decision of its bus:

dimensions:
  bus: { dtype: str }
  line: { dtype: str }
relations:
  line_bus: { key: line, values: bus }
parameters:
  cap: { dims: [line] }
variables:
  build: { dims: [bus], domain: binary }
  f: { dims: [line] }
constraints:
  flow_needs_build:
    dims: [line]
    expression: f <= cap * at(build, by=line_bus[bus])
objective: { sense: minimize, expression: "sum(build, over=bus)" }
\[ f_{l} \le \mathrm{cap}_{l} \cdot \mathit{build}_{\mathrm{line\_bus}(l)} \qquad \forall\, l \in \mathcal{L} \]
at reads one bus value once for each line at that bus The variable build has one row per bus, north and south. The relation line_bus puts lines l1 and l2 on bus north and line l3 on bus south. at of build by line_bus bus has one row per line: l1 and l2 both read build at north, and l3 reads build at south. build a variable bus north south line_bus a relation line bus l1 north l2 north l3 south at(build, by=line_bus[bus]) an expression, one row per line line value l1 build[north] l2 build[north] l3 build[south] read at bus one row per line

at removes the column in by=, bus, and adds the key, line, so build is read once per line.

shift#

shift takes the value of an expression a number of positions earlier along an ordered dimension. shift counts positions in the order of the dimension, not labels. Only the values move, and the result has the coordinates of the expression. edge= sets the value at a coordinate where nothing moved in. Here edge='wrap' makes a store cyclic, so the first snapshot takes the value of the last:

dimensions:
  snapshot: { dtype: int, ordered: true }
  storage: { dtype: str }
parameters:
  eta: { dims: [storage] }
variables:
  soc: { dims: [snapshot, storage], bounds: { lower: 0 } }
  charge: { dims: [snapshot, storage], bounds: { lower: 0 } }
  discharge: { dims: [snapshot, storage], bounds: { lower: 0 } }
constraints:
  storage_balance:
    dims: [snapshot, storage]
    expression: soc == shift(soc, along=snapshot, offset=1, edge='wrap') + charge * eta - discharge
objective: { sense: minimize, expression: sum(charge) }
\[ \mathit{soc}_{t,s} = \mathit{soc}_{t \ominus 1,s} + \mathit{charge}_{t,s} \cdot \mathrm{eta}_{s} - \mathit{discharge}_{t,s} \qquad \forall\, t \in \mathcal{T},\ s \in \mathcal{S} \]
A wrapped shift gives the first snapshot the value of the last The variable soc has three rows for the storage battery, at snapshots 0, 1 and 2. shift of soc along snapshot, with offset 1 and edge wrap, has the same three rows. At snapshot 0 it is soc at 2, at snapshot 1 it is soc at 0, and at snapshot 2 it is soc at 1. soc a variable snapshot storage 0battery 1battery 2battery edge='wrap' offset=1, counted cyclically shift(soc, …) an operator result snapshot storage value 0batterysoc[2, battery] 1batterysoc[0, battery] 2batterysoc[1, battery]

With three snapshots of data, snapshot 0 of the storage battery takes the value of the last snapshot. Each later snapshot takes the value of the one before it. The legend of the typeset document defines \(t \ominus k\) as \(t-k\), counted cyclically.

Without edge=, the vacated coordinate is absent, so a row that uses it is also absent.

Over a variable, the only numeric edge of a shift is 0. With the constraint r: soc >= shift(soc, along=snapshot, offset=1, edge=5) over [snapshot, storage] in the spec above, mathspec prints:

Constraint 'r': shift(edge=5) reads a variable, and over a variable only edge=0 is allowed. Write edge=0, and add the constant to the expression.

mathspec suggests the rewrite: edge=0, with the constant 5 in the expression.

At load, mathspec refuses a shift with no edge= over an expression with no variable. With a parameter inflow over [snapshot, storage] added to the spec above, and r: soc >= shift(inflow, along=snapshot, offset=1), mathspec prints three rewrites:

Constraint 'r': shift() over an expression with no variable leaves the vacated positions with no value. Write which you mean:
  shift(x, along=d, offset=n, edge='wrap')   the dimension really is cyclic
  shift(x, along=d, offset=n, edge=0)        the vacated positions contribute zero
  ...and a where: excluding them        the vacated rows should not exist at all
A where: alone does not remove this refusal, and edge=0 alone keeps a row whose bound is that zero.

Each rewrite is one meaning of the vacated positions: a cyclic dimension, a zero, or no row.

See also

within=#

within= takes a partition, such as one into seasons, and the neighbour of a coordinate is then the one before it in its own group:

dimensions:
  snapshot: { dtype: int, ordered: true }
  season: { dtype: str }
relations:
  season_of: { key: snapshot, values: season }
parameters:
  inflow: { dims: [snapshot] }
variables:
  soc: { dims: [snapshot], bounds: { lower: 0 } }
constraints:
  season_balance:
    dims: [snapshot]
    expression: soc == shift(soc, along=snapshot, offset=1, edge='wrap', within=season_of[season]) + inflow
objective: { sense: minimize, expression: sum(soc) }
\[ \mathit{soc}_{t} = \mathit{soc}_{t \ominus^{\mathrm{season\_of}(t)} 1} + \mathrm{inflow}_{t} \qquad \forall\, t \in \mathcal{T} \]
shift with within= wraps inside each season The variable soc has four rows, snapshots 1 to 4. The relation season_of puts snapshots 1 and 2 in winter and snapshots 3 and 4 in summer. shift of soc along snapshot, offset 1, edge wrap, within season_of season, has one row per snapshot. Snapshot 2 reads soc at 1 and snapshot 4 reads soc at 3. Snapshot 1, the first of winter, wraps to soc at 2, the last of winter, and snapshot 3, the first of summer, wraps to soc at 4. shift(soc, along=snapshot, offset=1, edge='wrap', within=season_of[season]) soc a variable snapshot 1 2 3 4 season_of a relation snapshot season 1 winter 2 winter 3 summer 4 summer the result an expression snapshot value 1 soc[2] 2 soc[1] 3 soc[4] 4 soc[3] one back group by season 1 and 3 wrap inside their season the group of season winter the group of season summer

The first snapshot of each season takes the value of the last snapshot of the same season. Every edge= setting applies one group at a time, and a coordinate in no group is absent under every edge=.

offset= from a parameter#

A parameter as offset states a construction lead time, a transit time, or any other delay that is a column of the data:

dimensions:
  technology: { dtype: str }
  month: { dtype: int, ordered: true }
parameters:
  lead: { dims: [technology], dtype: int }
  demand: { dims: [technology, month] }
variables:
  order:
    dims: [technology, month]
    bounds: { lower: 0 }
constraints:
  arrives_after_its_lead:
    dims: [technology, month]
    expression: shift(order, along=month, offset=lead, edge=0) >= demand
objective: { sense: minimize, expression: sum(order) }
\[ \mathit{order}_{m \boxminus_{0} \mathrm{lead},t} \ge \mathrm{demand}_{m,t} \qquad \forall\, m \in \mathcal{M},\ t \in \mathcal{T} \]
A parameter as offset shifts each technology by its own lead The variable order has six rows, months 1, 2 and 3 for technologies gas and wind. The parameter lead is 1 for wind and 2 for gas. shift of order along month, with offset lead and edge 0, has six rows. Wind reads one month back, so it is 0 at month 1, order at month 1 and wind at month 2, and order at month 2 and wind at month 3. Gas reads two months back, so it is 0 at months 1 and 2, and order at month 1 and gas at month 3. order a variable month technology 1gas 1wind 2gas 2wind 3gas 3wind shift(offset=lead, edge=0) lead: gas 2, wind 1 shift(order, …) an operator result month technology value 1gas0 1wind0 2gas0 2windorder[1, wind] 3gasorder[1, gas] 3windorder[2, wind]

For each technology, the shift takes order lead months back, and \(m \boxminus_{0} k\) is \(m-k\) with 0 past the edge.

mathspec refuses the file at load for any of four faults of the parameter. With lead declared dtype: float, mathspec prints:

Constraint 'arrives_after_its_lead': shift(offset=lead) counts positions, but 'lead' is declared dtype: float. Declare it dtype: int.

An offset is a count of positions, so lead must be dtype: int.

The offset may not vary along the dimension that the shift moves along. With lead declared over [technology, month], mathspec prints:

Constraint 'arrives_after_its_lead': shift(offset=lead) steps along 'month', and 'lead' is declared over ['month', 'technology']. Declare 'lead' over dims without 'month'.

lead then varies along month, the dimension that the shift moves along.

mathspec reads the offset over the dimensions of the shifted expression, and over the dimension that a within= column groups into. At load, mathspec refuses an offset over any other dimension. With a dimension period, a relation period_of: { key: month, values: period }, and lead declared over [period], the spec loads under within=period_of[period], with one offset per period. Without the within= on the shift, mathspec refuses the file and prints:

Constraint 'arrives_after_its_lead': shift(offset=lead): 'lead' varies over ['period'], which the expression does not carry (dims ['month', 'technology']). Declare 'lead' over dims the expression carries, or group by a relation into one of ['period'].

period is neither a dimension of order nor a group of a within=.

An offset may not negate the parameter, and with offset=-lead mathspec prints:

Constraint 'arrives_after_its_lead': shift(offset=-lead) negates a named offset. Negate the values of the parameter instead.

mathspec suggests the rewrite: the values of lead, negated in the data.

sum_back#

sum_back sums the last window positions along an ordered dimension, ending at the position being written. sum_back states the hours a started unit stays on (a minimum up time), a rolling budget or a delivery horizon. The dimension stays in the result, and with a window of 1 the result is array.

dimensions:
  unit: { dtype: str }
  hour: { dtype: int, ordered: true }
parameters:
  min_up: { dims: [unit], dtype: int }
variables:
  started: { dims: [unit, hour], domain: binary }
  on: { dims: [unit, hour], domain: binary }
constraints:
  stays_up_its_own_time:
    dims: [unit, hour]
    expression: sum_back(started, along=hour, window=min_up) <= on
objective: { sense: minimize, expression: sum(on) }
\[ \sum_{h' \in \mathcal{H} \,:\, 0 \le h - h' < \mathrm{min\_up}} \mathit{started}_{h',u} \le \mathit{on}_{h,u} \qquad \forall\, h \in \mathcal{H},\ u \in \mathcal{U} \]
sum_back with a window per unit, short at the start or wrapped The variable started has eight rows, hours 1 to 4 for units u1 and u2. The parameter min_up is 1 for u1 and 2 for u2. With window min_up, each row of u1 is started at that hour alone. Each row of u2 sums started at that hour and the hour before, except hour 1, whose window is short: it holds started of u2 at hour 1 alone, and no row is lost. With edge wrap, hour 1 of u2 reaches around to hour 4, so it holds started of u2 at hour 4 plus started of u2 at hour 1. sum_back(started, along=hour, window=min_up, …) started a variable hour unit 1 u1 1 u2 2 u1 2 u2 3 u1 3 u2 4 u1 4 u2 min_up a parameter unit value u1 1 u2 2 window=min_up an expression, no edge= hour unit value 1 u1 started[1,u1] 1 u2 started[1,u2] 2 u1 started[2,u1] 2 u2 started[1,u2] + started[2,u2] 3 u1 started[3,u1] 3 u2 started[2,u2] + started[3,u2] 4 u1 started[4,u1] 4 u2 started[3,u2] + started[4,u2] hour 1 of u2: a short window, no row lost window=min_up, edge='wrap' an expression, the window reaches around hour unit value 1 u1 started[1,u1] 1 u2 started[4,u2] + started[1,u2] 2 u1 started[2,u1] 2 u2 started[1,u2] + started[2,u2] 3 u1 started[3,u1] 3 u2 started[2,u2] + started[3,u2] 4 u1 started[4,u1] 4 u2 started[3,u2] + started[4,u2] hour 1 of u2 reaches around to hour 4

A unit that started in any of the last min_up hours must be on, and each unit has its own window length.

A window given as a parameter is dtype: int, and does not vary along the dimension being summed. With min_up declared over [unit, hour], mathspec prints:

Constraint 'stays_up_its_own_time': sum_back(window=min_up) steps along 'hour', and 'min_up' is declared over ['hour', 'unit']. Declare 'min_up' over dims without 'hour'.

min_up then varies along hour, the dimension that the window runs along.

Without edge=, a window that reaches past the start of the dimension is short, and no row is lost. edge= takes 'wrap' or nothing, and 'wrap' makes the window reach around to the end. With edge=0, mathspec prints:

Constraint 'stays_up_its_own_time': sum_back(edge=...) takes 'wrap' or nothing. To add a constant, write it in the expression.

mathspec suggests the rewrite: the constant, written in the expression.

As for shift, within= takes a partition, and the window then stays inside each group.

Every operator as math#

The typesetter prints each form from the smallest spec that uses it.

Operator Renders as
sum(array) \(\sum_{g \in \mathcal{G},\ t \in \mathcal{T}} p_{g,t} \le \mathrm{budget}\)
sum(array, over=dim) \(\sum_{g \in \mathcal{G}} p_{g,t} \le \mathrm{limit}_{t} \qquad \forall\, t \in \mathcal{T}\)
sum(array, over=[a, …]) \(\sum_{g \in \mathcal{G},\ t \in \mathcal{T}} p_{c,g,t} \le \mathrm{limit}_{c} \qquad \forall\, c \in \mathcal{C}\)
sum(array, over=dim, by=relation[c]) \(\sum_{g \in \mathcal{G} \,:\, \mathrm{gen\_bus}(g) = b} p_{g,t} \le \mathrm{limit}_{b,t} \qquad \forall\, b \in \mathcal{B},\ t \in \mathcal{T}\)
sum(array, over=dim, by=relation[c]), joining on the rest of the key \(\sum_{g \in \mathcal{G} \,:\, \mathrm{zone\_of}(g,\ e) = z} p_{g,e} \ge \mathrm{demand}_{e,z} \qquad \forall\, e \in \mathcal{E},\ z \in \mathcal{Z}\)
sum(array, over=[dim, …], by=relation[c, …]) \(\sum_{g \in \mathcal{G},\ e \in \mathcal{E} \,:\, \mathrm{slot\_of.bus}(g,\ e) = b \wedge \mathrm{slot\_of.technology}(g,\ e) = t} p_{g,e} \le \mathrm{cap}_{b,t} \qquad \forall\, b \in \mathcal{B},\ t \in \mathcal{T}\)
at(array, by=relation[c]) \(p_{t} \le \mathrm{cap}_{\mathrm{period\_of}(t)} \qquad \forall\, t \in \mathcal{T}\)
at(array, by=relation[c]), two columns over one dimension \(f_{l} \le \mathrm{cap}_{\mathrm{ends.bus0}(l)} \qquad \forall\, l \in \mathcal{L}\)
shift(array, along=dim, offset=n) \(p_{t} \le p_{t - 1} \qquad \forall\, t \in \mathcal{T}\)
shift(array, along=dim, offset=n, edge='wrap') \(p_{t} \le p_{t \ominus 1} \qquad \forall\, t \in \mathcal{T}\)
shift(array, along=dim, offset=n, edge=v) \(p_{t} \le p_{t \boxminus_{0} 1} \qquad \forall\, t \in \mathcal{T}\)
shift(array, along=dim, offset=p, edge=…) \(\mathit{order}_{m \boxminus_{0} \mathrm{lead},t} \ge \mathrm{demand}_{m,t} \qquad \forall\, m \in \mathcal{M},\ t \in \mathcal{T}\)
shift(array, along=dim, offset=n, within=relation[c]) \(p_{t} \le p_{t \ominus^{\mathrm{season\_of}(t)} 1} \qquad \forall\, t \in \mathcal{T}\)
sum_back(array, along=dim, window=n) \(\sum_{h' \in \mathcal{H} \,:\, 0 \le h - h' < 3} \mathit{started}_{h',u} \le \mathit{on}_{h,u} \qquad \forall\, h \in \mathcal{H},\ u \in \mathcal{U}\)
sum_back(array, along=dim, window=p) \(\sum_{h' \in \mathcal{H} \,:\, 0 \le h - h' < \mathrm{min\_up}} \mathit{started}_{h',u} \le \mathit{on}_{h,u} \qquad \forall\, h \in \mathcal{H},\ u \in \mathcal{U}\)
sum_back(array, along=dim, window=p, edge='wrap') \(\sum_{h' \in \mathcal{H} \,:\, 0 \le h \ominus h' < \mathrm{min\_up}} \mathit{started}_{h',u} \le \mathit{on}_{h,u} \qquad \forall\, h \in \mathcal{H},\ u \in \mathcal{U}\)
sum_back(array, along=dim, window=n, within=relation[c]) \(\sum_{h' \in \mathcal{H} \,:\, 0 \le h -^{\mathrm{day\_of}(h)} h' < 3} \mathit{started}_{h',u} \le \mathit{on}_{h,u} \qquad \forall\, h \in \mathcal{H},\ u \in \mathcal{U}\)
dual(constraint) \(\mathit{price}_{t} = \lambda_{\mathrm{balance},t} \qquad \forall\, t \in \mathcal{T}\)

See also