Ornstein-Uhlenbeck noise¶
Noise generated by a stochastic process that is characterized by its mean-reverting behavior.
Usage¶
The noise usage is defined in each agent’s configuration dictionary. A noise instance is set under the "noise"
sub-key. The following examples show how to set the noise for an agent:
from skrl.resources.noises.torch import OrnsteinUhlenbeckNoise
cfg = DEFAULT_CONFIG.copy()
cfg["exploration"]["noise"] = OrnsteinUhlenbeckNoise(theta=0.15, sigma=0.2, base_scale=1.0, device="cuda:0")
from skrl.resources.noises.jax import OrnsteinUhlenbeckNoise
cfg = DEFAULT_CONFIG.copy()
cfg["exploration"]["noise"] = OrnsteinUhlenbeckNoise(theta=0.15, sigma=0.2, base_scale=1.0)
API (PyTorch)¶
- class skrl.resources.noises.torch.ornstein_uhlenbeck.OrnsteinUhlenbeckNoise(theta: float, sigma: float, base_scale: float, mean: float = 0, std: float = 1, device: str | torch.device | None = None)¶
Bases:
Noise
- __init__(theta: float, sigma: float, base_scale: float, mean: float = 0, std: float = 1, device: str | torch.device | None = None) None ¶
Class representing an Ornstein-Uhlenbeck noise
- Parameters:
theta (float) – Factor to apply to current internal state
sigma (float) – Factor to apply to the normal distribution
base_scale (float) – Factor to apply to returned noise
mean (float, optional) – Mean of the normal distribution (default:
0.0
)std (float, optional) – Standard deviation of the normal distribution (default:
1.0
)device (str or torch.device, optional) – Device on which a tensor/array is or will be allocated (default:
None
). If None, the device will be either"cuda"
if available or"cpu"
Example:
>>> noise = OrnsteinUhlenbeckNoise(theta=0.1, sigma=0.2, base_scale=0.5)
- sample(size: Tuple[int] | torch.Size) torch.Tensor ¶
Sample an Ornstein-Uhlenbeck noise
- Parameters:
size (tuple or list of int, or torch.Size) – Shape of the sampled tensor
- Returns:
Sampled noise
- Return type:
Example:
>>> noise.sample((3, 2)) tensor([[-0.0452, 0.0162], [ 0.0649, -0.0708], [-0.0211, 0.0066]], device='cuda:0') >>> x = torch.rand(3, 2, device="cuda:0") >>> noise.sample(x.shape) tensor([[-0.0540, 0.0461], [ 0.1117, -0.1157], [-0.0074, 0.0420]], device='cuda:0')
- sample_like(tensor: torch.Tensor) torch.Tensor ¶
Sample a noise with the same size (shape) as the input tensor
This method will call the sampling method as follows
.sample(tensor.shape)
- Parameters:
tensor (torch.Tensor) – Input tensor used to determine output tensor size (shape)
- Returns:
Sampled noise
- Return type:
Example:
>>> x = torch.rand(3, 2, device="cuda:0") >>> noise.sample_like(x) tensor([[-0.0423, -0.1325], [-0.0639, -0.0957], [-0.1367, 0.1031]], device='cuda:0')
API (JAX)¶
- class skrl.resources.noises.jax.ornstein_uhlenbeck.OrnsteinUhlenbeckNoise(theta: float, sigma: float, base_scale: float, mean: float = 0, std: float = 1, device: str | jax.Device | None = None)¶
Bases:
Noise
- __init__(theta: float, sigma: float, base_scale: float, mean: float = 0, std: float = 1, device: str | jax.Device | None = None) None ¶
Class representing an Ornstein-Uhlenbeck noise
- Parameters:
theta (float) – Factor to apply to current internal state
sigma (float) – Factor to apply to the normal distribution
base_scale (float) – Factor to apply to returned noise
mean (float, optional) – Mean of the normal distribution (default:
0.0
)std (float, optional) – Standard deviation of the normal distribution (default:
1.0
)device (str or jax.Device, optional) – Device on which a tensor/array is or will be allocated (default:
None
). If None, the device will be either"cuda"
if available or"cpu"
Example:
>>> noise = OrnsteinUhlenbeckNoise(theta=0.1, sigma=0.2, base_scale=0.5)
- sample(size: Tuple[int]) ndarray | jax.Array ¶
Sample an Ornstein-Uhlenbeck noise
- Parameters:
- Returns:
Sampled noise
- Return type:
np.ndarray or jax.Array
Example:
>>> noise.sample((3, 2)) Array([[ 0.01878439, -0.12833427], [ 0.06494182, 0.12490594], [ 0.024447 , -0.01174496]], dtype=float32) >>> x = jax.random.uniform(jax.random.PRNGKey(0), (3, 2)) >>> noise.sample(x.shape) Array([[ 0.17988093, -1.2289404 ], [ 0.6218886 , 1.1961104 ], [ 0.23410667, -0.11247082]], dtype=float32)
- sample_like(tensor: ndarray | jax.Array) ndarray | jax.Array ¶
Sample a noise with the same size (shape) as the input tensor
This method will call the sampling method as follows
.sample(tensor.shape)
- Parameters:
tensor (np.ndarray or jax.Array) – Input tensor used to determine output tensor size (shape)
- Returns:
Sampled noise
- Return type:
np.ndarray or jax.Array
Example:
>>> x = jax.random.uniform(jax.random.PRNGKey(0), (3, 2)) >>> noise.sample_like(x) Array([[0.57450044, 0.09968603], [0.7419659 , 0.8941783 ], [0.59656656, 0.45325184]], dtype=float32)