Noise Models
bbobax.noise
Black-box Optimization Benchmarking Noise Models.
The three BBOB noise models (Gaussian, uniform, Cauchy) follow Hansen, Finck,
Ros, Auger, "Real-Parameter Black-Box Optimization Benchmarking 2009: Noisy
Functions Definitions" (INRIA RR-6869) and are verified against the official
bbobbenchmarks.py formulas in tests/test_noise.py.
A noise model has the same shape of contract as a problem -- sample draws the
instance's settings, apply disturbs one value -- and a problem holds one,
statically. Like the 24 functions, a model is chosen by holding the object
rather than by switching on an index, so nothing evaluates the branches it did
not want. Mixture is the one exception, and pays for itself in the open.
Two things to know:
- Noise applies to the raw function value only; the boundary penalty and
f_opt are added outside it (
BBOBProblem.evaluate), as the paper prescribes. - Stabilization is part of the three official models, not an option. Each
of
fGauss,fUniformandfCauchyends by adding1.01 * 1e-8and returning the undisturbed value below that tolerance, unconditionally, so noise can never stop an algorithm reaching the target precision.Noiselesshas none, as official's noise-free functions do not.
Severity is continuous here, and that is a deviation. The paper defines two
discrete severities per model (moderate/severe) and the bbob-noisy suite pins each
function to one of them; bbobax samples the settings continuously across that
span instead, so difficulty varies per instance for free and a fixed model is
not a fixed difficulty. That is what meta-learning wants and what published
noisy-BBOB numbers are not: nothing here is comparable to a published
f101-f130 result unless the severity is pinned. The paper's two points are
therefore first-class -- Gaussian.severe(), Cauchy.moderate() -- and pin
the settings exactly.
Only the model family stays fixed on a problem. When one batch genuinely has to
mix families, Mixture restores that and states its cost.
Cauchy
Additive Cauchy outliers, fired with probability p.
Official fCauchy (paper 3.3):
f + alpha * max(0, 1e3 + 1{U < p} * N / |N|). Moderate is
alpha = 0.01, p = 0.05; severe alpha = 1, p = 0.2.
Source code in src/bbobax/noise.py
__init__(alpha_range=(0.01, 1.0), p_range=(0.05, 0.2))
Initialize the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
alpha_range
|
tuple[float, float]
|
Range alpha is drawn from, moderate to severe. |
(0.01, 1.0)
|
p_range
|
tuple[float, float]
|
Range the outlier probability is drawn from. |
(0.05, 0.2)
|
Source code in src/bbobax/noise.py
apply(key, value, params)
Disturb one value.
Source code in src/bbobax/noise.py
moderate()
classmethod
sample(key, num_dims)
Sample alpha and p.
Source code in src/bbobax/noise.py
CauchyParams
Gaussian
Multiplicative log-normal noise: f * exp(beta * N(0, 1)).
Official fGauss (paper 3.1). Moderate is beta = 0.01, severe beta = 1.
Source code in src/bbobax/noise.py
__init__(beta_range=(0.01, 1.0))
Initialize the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
beta_range
|
tuple[float, float]
|
Range beta is drawn from, moderate to severe. |
(0.01, 1.0)
|
apply(key, value, params)
Disturb one value.
moderate()
classmethod
sample(key, num_dims)
GaussianParams
Mixture
Draw a noise model per instance, alongside the instance itself.
Every other model here is chosen by holding it, so nothing dispatches. That is the right default, but it makes the model family a property of the problem rather than of the instance -- and meta-learning sometimes wants a batch of instances that disagree about which noise they carry. This restores exactly that, and is the only place in bbobax that pays for it:
models = Mixture(Gaussian(), Uniform(), Cauchy())
problem = Sphere(num_dims=10, noise_model=models)
The cost is real and worth stating. sample draws every model's settings
and apply selects with lax.switch, which under vmap over a varying
model_id evaluates every branch for every solution. With three models
that is three noise models computed per evaluation. Prefer looping over
models in Python -- as one loops over functions and dimensions -- and reach
for this only when one batch genuinely has to mix them.
Note that severity already varies continuously per instance inside a single
model (Gaussian draws its beta across the paper's moderate-to-severe
span), so a fixed model is not a fixed difficulty. Only the family is
fixed, and that is what this changes.
Source code in src/bbobax/noise.py
__init__(*models)
Initialize the mixture.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
*models
|
NoiseModel
|
The models to draw among, uniformly. |
()
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If no model is given. |
Source code in src/bbobax/noise.py
apply(key, value, params)
Disturb one value with the model this instance drew.
Source code in src/bbobax/noise.py
sample(key, num_dims)
Draw which model this instance carries, and every model's settings.
Source code in src/bbobax/noise.py
MixtureParams
NoiseModel
Bases: Protocol
How a raw function value is disturbed.
A protocol rather than a base class, exactly like Descriptor: the models
share no implementation, only a contract, and each owns its own parameter
type. sample draws the instance's settings and apply disturbs one value.
Source code in src/bbobax/noise.py
apply(key, value, params)
Disturb one raw function value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
Array
|
JAX random key. |
required |
value
|
Array
|
The raw function value. |
required |
params
|
Any
|
The instance's noise parameters. |
required |
Returns:
| Type | Description |
|---|---|
Array
|
The disturbed value. |
Source code in src/bbobax/noise.py
sample(key, num_dims)
Sample this model's settings for one instance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
Array
|
JAX random key. |
required |
num_dims
|
int
|
Dimensionality of the problem -- the uniform model's alpha is dimension-dependent. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
The instance's noise parameters. |
Source code in src/bbobax/noise.py
Noiseless
No noise: the value is returned untouched.
Official BBOB's noise-free functions do exactly this, with no stabilization -- so the noiseless suite is bit-for-bit the raw functions.
Source code in src/bbobax/noise.py
apply(key, value, params)
NoiselessParams
Uniform
Uniform-powered noise with a heavy low-value tail.
Official fUniform (paper 3.2):
U^beta * f * max(1, (1e9 / (f + eps)) ** (alpha * U)), with two
independent uniform draws.
Alpha is dimension-dependent -- the paper's 0.01 (0.49 + 1/D) moderate
and (0.49 + 1/D) severe -- so what is drawn here is the multiplier on
(0.49 + 1/D), which puts the paper's two severities at the endpoints of
the range at every dimension.
Source code in src/bbobax/noise.py
__init__(alpha_range=(0.01, 1.0), beta_range=(0.01, 1.0))
Initialize the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
alpha_range
|
tuple[float, float]
|
Range the multiplier on |
(0.01, 1.0)
|
beta_range
|
tuple[float, float]
|
Range beta is drawn from, moderate to severe. |
(0.01, 1.0)
|
Source code in src/bbobax/noise.py
apply(key, value, params)
Disturb one value.
Source code in src/bbobax/noise.py
moderate()
classmethod
Pin the paper's moderate severity: alpha = 0.01(0.49 + 1/D), beta = 0.01.
sample(key, num_dims)
Sample alpha and beta.
Source code in src/bbobax/noise.py
severe()
classmethod
UniformParams
stabilize(value, noisy)
Leave a value alone once it is within the target precision.
The official models spell this as fval += 1.01 * tol followed by
fval[ftrue < tol] = ftrue[...]; this is the same thing branchlessly.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
value
|
Array
|
The raw function value. |
required |
noisy
|
Array
|
The disturbed value. |
required |
Returns:
| Type | Description |
|---|---|
Array
|
The stabilized value. |