Validation¶
Model users and model developers are usually distinct persons. Therefore, users may not be aware of model limitations. Moreover, only a few model parameters may be meaningful to them.
To address these issues, CoSApp allows model developers to specify the validity range and visibility scope of all variables. This section focuses on the validation feature.
The concept¶
A validity range and a limit range can be defined on all variables in CoSApp.
The validity range defines an interval within which the model can be used with confidence. The yellow areas between validity range and limits define values for which validation by an expert is required. Finally, values beyond the limits should be avoided as the reliability of the model is unknown.
These ranges are only provided as information, and are not enforced during the execution of a model. Therefore, validation should be performed a posteriori. This behavior has been chosen to limit the constraints on mathematical systems within a model.
Defining validation criteria¶
CoSApp variables are defined in Port or in System instances. Validity ranges can be specified in the setup of these classes.
For example in a Port:
[2]:
from cosapp.base import System, Port
class MyPort(Port):
def setup(self):
self.add_variable(
'v', 22.,
valid_range = (-2, 5),
invalid_comment = "Design rule abc forbids `v` outside [-2, 5]",
limits = (-10, None),
out_of_limits_comment = "The model has not been tested for v < -10",
)
And in a System:
[3]:
class MySystem(System):
def setup(self):
# Definition of validation criteria on a inward variable 'd'
self.add_inward(
'd', 7.,
valid_range = (-2, 5),
invalid_comment = "Design rule abc forbid `d` outside [-2, 5]",
limits = (None, 10),
out_of_limits_comment = "The model has not been tested for d > 10",
)
# Overwrite the default validation criteria on the variable 'v' of port 'port_in'
self.add_input(
MyPort,
'port_in', {
'v': dict(
valid_range = (0, 3),
invalid_comment = "Design rule blah-blah recommends `v` to be within [0, 3]",
limits = (None, 10),
out_of_limits_comment = "The model has not been tested for v > 10"
)
}
)
If no range are provided, the variable is always considered as valid. In case the value is not valid but falls within the limits, the invalid_comment will be shown to inform the user on the reason behind the validity range. If the value is out of limits, the out_of_limits_comment will be displayed.
If one end of the range is unbounded, users may specify a None value. For example, a non-negative variable will correspond to limits = (0, None).
Displaying validation criteria¶
You can get relevant information by displaying the documentation of the Port or System object.
[4]:
from cosapp.tools import display_doc
display_doc(MyPort)
[5]:
display_doc(MySystem)
Testing validation criteria¶
As mentioned earlier, validity ranges are not enforced during simulations. To check the validity of the results after execution, you may add special driver ValidityCheck to the master System, which will produce a post-run data validity report.
Values falling outside the prescribed limits will be gathered in the ERROR section; those between the valid range and the limits will appear in the WARNING section. Valid values will simply be omitted.
[6]:
from cosapp.drivers import ValidityCheck
s = MySystem('master_system')
s.add_driver(ValidityCheck('validation'))
s.run_drivers()
inwards.d = 7 not in [-2, 5] - Design rule abc forbid `d` outside [-2, 5]
port_in.v = 22 not in [-inf, 10] - The model has not been tested for v > 10
Example¶
We will demonstrate this concept on the circuit example.
The physical problem consists in determining the two node voltages which satisfy electric current conservation.
[7]:
from cosapp.base import System, Port, Scope
from cosapp.drivers import NonLinearSolver, ValidityCheck
import numpy as np
Set validity criteria¶
By default, the current intensity will be defined as positive.
In Resistor components, we add a validity range on current intensity, pretending a non-constant behaviour of the resistance value for high currents.
We can look at the documentation to check that the validation criteria are taken into account
[10]:
from cosapp.tools import display_doc
display_doc(Resistor)
Building the circuit¶
[11]:
from __future__ import annotations
class Diode(Dipole):
"""Regularized diode model
The current intensity flowing through the diode is calculated based on
$ I = I_s \\exp \\left( \\dfrac{V_{in} - V_{out}}{V_t} - 1 \\right) $
"""
tags = ['cosapp', 'developer']
def setup(self):
super().setup()
self.add_inward('Is', 1e-15, desc='Saturation current in Amps')
self.add_inward('Vt', 0.025875, scope=Scope.PROTECTED, desc='Thermal voltage in Volts')
def compute_I(self):
"""Regularized diode model"""
self.I.I = self.Is * np.exp(self.deltaV / self.Vt - 1.)
class Node(System):
"""Electric node model with `n_in` incoming and `n_out` outgoing currents.
"""
def setup(self, n_in=1, n_out=1):
self.add_property('n_in', max(1, int(n_in)))
self.add_property('n_out', max(1, int(n_out)))
incoming = tuple(
self.add_input(Intensity, f"I_in{i}")
for i in range(self.n_in)
)
outgoing = tuple(
self.add_input(Intensity, f"I_out{i}")
for i in range(self.n_out)
)
self.add_property('incoming_currents', incoming)
self.add_property('outgoing_currents', outgoing)
self.add_inward('V', 1.0, unit='V')
self.add_outward('sum_I_in', 0., unit='A', desc='Sum of all incoming currents')
self.add_outward('sum_I_out', 0., unit='A', desc='Sum of all outgoing currents')
self.add_unknown('V')
self.add_equation('sum_I_in == sum_I_out', name='current balance')
def compute(self):
self.sum_I_in = sum(current.I for current in self.incoming_currents)
self.sum_I_out = sum(current.I for current in self.outgoing_currents)
@classmethod
def make(
cls,
name: str,
parent: System,
incoming: list[Dipole]=[],
outgoing: list[Dipole]=[],
pulling=None,
) -> Node:
"""Factory creating new node within `parent`, with
appropriate connections with incoming and outgoing dipoles.
"""
node = cls(name, n_in=len(incoming), n_out=len(outgoing))
parent.add_child(node, pulling=pulling)
for dipole, current in zip(incoming, node.incoming_currents):
parent.connect(dipole.I, current)
parent.connect(dipole.V_out, node.inwards, 'V')
for dipole, current in zip(outgoing, node.outgoing_currents):
parent.connect(dipole.I, current)
parent.connect(dipole.V_in, node.inwards, 'V')
return node
class Source(System):
def setup(self, I=0.1):
self.add_inward('I', I)
self.add_output(Intensity, 'I_out', {'I': I})
def compute(self):
self.I_out.I = self.I
class Ground(System):
def setup(self, V=0.):
self.add_inward('V', V)
self.add_output(Voltage, 'V_out', {'V': V})
def compute(self):
self.V_out.V = self.V
class Circuit(System):
def setup(self):
R1 = self.add_child(Resistor('R1', R=100.), pulling={'V_out': 'Vg'})
R2 = self.add_child(Resistor('R2', R=10000.))
D1 = self.add_child(Diode('D1'), pulling={'V_out': 'Vg'})
# Define nodes
Node.make('n1',
parent=self,
outgoing=[R1, R2],
pulling={'I_in0': 'I_in'},
)
Node.make('n2',
parent=self,
incoming=[R2],
outgoing=[D1],
)
Create a function generating the system of interest¶
[12]:
def make_circuit(name='model', I=1.0, add_drivers=True) -> System:
"""Factory creating a circuit with initial source intensity `I`.
If `add_drivers` is True, a numerical solver and validation driver
are added to the head system.
"""
s = System(name)
s.add_child(Source('source', I=I))
s.add_child(Ground('ground', V=0.0))
s.add_child(Circuit('circuit'))
s.connect(s.source.I_out, s.circuit.I_in)
s.connect(s.ground.V_out, s.circuit.Vg)
if add_drivers:
# Add numerical solver and validation driver
s.add_driver(NonLinearSolver('solver'))
s.add_driver(ValidityCheck('validation'))
return s
Solving the valid problem¶
First we will solve the initial problem with a intensity source of 0.1 A and check that the results are valid.
[13]:
p = make_circuit(I=0.1)
# Execute the problem
p.run_drivers()
print(f"Sanity check: {p.circuit.R1.I.I + p.circuit.D1.I.I = } ({p.source.I} expected)")
Sanity check: p.circuit.R1.I.I + p.circuit.D1.I.I = 0.1 (0.1 expected)
Solving invalid problems¶
Then we will solve the problem with a high intensity in order to trigger the validation criteria on the resistors.
[14]:
p = make_circuit(I=50.)
# Execute the problem
p.run_drivers()
print(f"Sanity check: {p.circuit.R1.I.I + p.circuit.D1.I.I = } ({p.source.I} expected)")
circuit.R1.I.I = 49.505 not in [-25, 25] - Resistance may not be constant for currents exceeding 25 A
Sanity check: p.circuit.R1.I.I + p.circuit.D1.I.I = 50.00000000000001 (50.0 expected)
Finally we will solve the problem with a intensity source of -0.1 A (regarded as invalid, here).
[15]:
p = make_circuit(I=-0.1)
# Execute the problem
p.run_drivers()
print(f"Sanity check: {p.circuit.R1.I.I + p.circuit.D1.I.I = } ({p.source.I} expected)")
source.I_out.I = -0.1 not in [0.0, inf] - Current can only flow in one direction.
circuit.n1.I_in0.I = -0.1 not in [0.0, inf] - Current can only flow in one direction.
circuit.n1.I_out0.I = -0.1 not in [0.0, inf] - Current can only flow in one direction.
circuit.I_in.I = -0.1 not in [0.0, inf] - Current can only flow in one direction.
Sanity check: p.circuit.R1.I.I + p.circuit.D1.I.I = -0.1 (-0.1 expected)