blueprints.camera module#

Cameras provide a rendered image for each time step. A Camera can be attached to a blueprints.body.Body or to the blueprints.world.World directly. If a Camera is attached to a blueprints.body.Body in a Agent its rendered images are contained in the Agents observations. All cameras can be viewed when blueprints.world.World.view() by setting the camera in the Rendering section.

Camera Orientation#

Similarly to blueprints.light.Light Cameras can either be fixed in orientation locally/globally or tracking another Body by setting the Camera.mode and optionally Camera.target. For a detailed description see Camera.mode.

Local orientation#
>>> body = blue.Body(cameras=blue.Camera(mode='fixed'))
Global orientation#
>>> body = blue.Body(cameras=blue.Camera(mode='track'))
Tracking Body#
>>> one  = blue.Body(name='one', cameras=blue.Camera(mode='targetbody'))
>>> two  = blue.Body(name='two')
>>> root = blue.Body(name='root', bodies=[one, two])
>>> # Version one:
>>> root.bodies['one'].cameras.target = root.bodies['two']
>>> # Version two:
>>> root.bodies['two'].targeting_cameras = root.bodies['one'].cameras
XML example#
<body name="root">
    <body name="one">
        <camera mode="targetbody" name="unnamed_camera" target="two" />
    </body>
    <body name="two" />
</body>

Using the target attribute for a Camera results in continous tracking of the target Thing. If instead a Camera should be initially oriented to center an object, without tracking it during the simulation, the Camera.looking() method can be used to align the camera to a position or direction.

Looking in a Direction#
>>> camera = camera.looking(direction=[1, 1, 1])
Looking in a Direction#
<camera euler="2.3561945 -0.6154797 0.5235988" />

To center a position either a Thing or a global position can be passed.

Looking at Positions/Things#
>>> body = blue.Body(pos=[-1, 0, 1])
# VERSION 1
>>> camera = camera.looking(pos=body)
# VERSION 2
>>> camera = camera.looking(pos=[-1, 0, 1])
Looking at Positions/Things#
<camera euler="1.5707964 2.3561945 1.5707964" />

Recording#

To record videos with a Camera you can use the methods World.start_recording, World.step and World.stop_recording. For more details refer to start_recording.

Recording Videos#
>>> world.start_recording(cam_1, cam_2, cam_3, filename='my.gif', loop=0)
>>> world.step(seconds=5)
>>> world.stop_recording()
class blueprints.camera.Camera[source]#

Bases: CameraType, CyclicalThing, MoveableThing

This class is available through the shortcut blueprints.Camera.

Most attribute descriptions are partially taken from Mujoco.

fieldofview#

The angle of field of view in radians.

Type:

int | float, optional

ipd#

Inter-pupilary distance. This attribute only has an effect during stereoscopic rendering. It specifies the distance between the left and right viewpoints. Each viewpoint is shifted by +/- half of the distance specified here, along the X axis of the camera frame.

Type:

float

target#

The target attribute specifies a Body that is tracked by the camera. Compatible mode values are (‘targetbody’, ‘targetbodycom’).

Type:

blue.Body | None

mode#
The possible values are:

  • 'fixed': Position and orientation will change w.r.t. the parents frame of reference.

  • 'track': Position is at a constant offset from the parent in world coordinates, while the camera orientation is constant in world coordinates.

  • 'trackcom': is similar to 'track' but the constant spatial offset is defined relative to the center of mass of the kinematic subtree starting at the parent Body. This can be used to keep an entire mechanism in view. Note that the subtree center of mass for the World is the center of mass of the entire model.

  • 'targetbody': the Camera position is fixed in the parent Body, while the Camera orientation is adjusted so that it always points towards the targeted Body (which is specified with the target attribute)

  • 'targetbodycom': is the same as 'targetbody' but the Camera is oriented towards the center of mass of the subtree starting at the target Cody.

Type:

str

__init__(pos=[0.0, 0.0, 0.0], alpha=1.5707963267948966, beta=0.0, gamma=0.0, mode='fixed', resolution=[480, 480], fieldofview=0.7853981633974483, ipd=0.068, fps=30, name=None, x=None, y=None, z=None, quat=None, **kwargs)[source]#
Parameters:

  • pos (np.ndarray | list[int | float], optional) – Represents the position of the object. Changing this attribute also changes the properties x, y and z.

  • alpha (int | float, optional) – (Improper) euler angle of rotation around the x-axis in radian. Changing this value also changes the euler property.

  • beta (int | float, optional) – (Improper) euler angle of rotation around the y-axis in radian. Changing this value also changes the euler property.

  • gamma (int | float, optional) – (Improper) euler angle of rotation around the z-axis in radian. Changing this value also changes the euler property.

  • mode (str, optional) – The mode defines who the position and orientation is changed over time. See mode for a detailed description.

  • fieldofview (int | float, optional) – The angle of field of view in radians.

  • ipd (int | float, optional) – Inter-pupilary distance. This attribute only has an effect during stereoscopic rendering. It specifies the distance between the left and right viewpoints. Each viewpoint is shifted by +/- half of the distance specified here, along the X axis of the camera frame.

  • name (str | None, optional) – The user specified name of the Camera. It might change in the case of a naming conflict.

  • x (int | float |np.int32 | np.int64 | np.float32 | np.float64 | None, optional) – If pos is not specified, this argument sets the X position coordinate.

  • y (int | float |np.int32 | np.int64 | np.float32 | np.float64 | None, optional) – If pos is not specified, this argument sets the Y position coordinate.

  • z (int | float |np.int32 | np.int64 | np.float32 | np.float64 | None, optional) – If pos is not specified, this argument sets the Z position coordinate.

  • quat (list [ int | float ] | np.ndarray | None, optional) – If set, the quaternion orientation overwrites the euler angles alpha, beta and gamma.

  • **kwargs – Keyword arguments are passed to super().__init__.

  • resolution (ndarray | list[int])

  • fps (int | float | None)

Return type:

None

property observation#

The Camera observation is retrieved as the image rendered by the mujoco Renderer using OpenGL.

Returns:

RGB image using the specified camera settings.

Return type:

np.ndarray

start_recording(filename, fps=60, **kwargs)[source]#

This shorthand method starts the recording of a video. It can only be used if the camera is already attached within a World.

For details read World.start_recording.

Parameters:

  • filename (str) – The name (with a proper file extension indicating the format) of the video. If multiple cameras are passed, multiple videos will be created with the name of each camera prefixed to the filename.

  • fps (int) – The frames per second

  • **kwargs (dict) – Additional keyword arguments that are passed to imageio to write the video.

stop_recording()[source]#

This shorthand method stops the recording of a video. It can only be used if the camera is already attached within a World.

For details read World.start_recording.

looking(pos=None, direction=None, **kwargs)[source]#

The method creates a new camera that is focussed on pos. The Camera orientation is adjusted to center the pos argument in the image. Alternatively a direction in which the camera should look can be specified. This only constraints two degrees of freedom, so the thrid degree is chosen such that the camera records parallel to the ground.

Parameters:

  • pos (list [ int | float] | np.ndarray | blue.MoveableThingType | None) – If set, it can either be a (global) position or a MoveableThing that will be centered

  • direction (list [ int | float] | np.ndarray | None) – If set, the returned camera will be oriented parallel to the direction vector.

Returns:

A new Camera looking into the specified direction or onto the specified position/Thing

Return type:

blue.CameraType

property target: BodyType | None#

The target attribute specifies a Body that is tracked by the camera. Compatible mode values are (‘targetbody’, ‘targetbodycom’).

Return type:

blue.BodyType | None

property resolution: ndarray#

The resolution is used for observation rendering and recording. Rendering in World.view uses OpenGL defaults.

Returns:

The resolution of the camera.

Return type:

np.ndarray

property ipd: float#

This attribute only has an effect during stereoscopic rendering. It specifies the distance between the left and right viewpoints. Each viewpoint is shifted by \(\pm\) half of the distance specified here, along the \(X\) axis of the camera frame.

Returns:

Inter-pupilary distance

Return type:

float

property fieldofview: float#

Vertical field-of-view of the camera.

Returns:

Field of view

Return type:

float

property fps: float | None#

Frames per second

Return type:

float

property fovy: float#

Vertical field-of-view of the camera.

Returns:

field of view in degrees

Return type:

float

property frameinterval: float#

The delay between each rendering in seconds

Return type:

float