isaacsim-mcp-server

Bind a material to an object. Args: material_path: Prim path of the material. target_prim_path: Prim path of the object to apply the material to.

Capture an RGB image from a camera sensor. Args: prim_path: Prim path of the camera. output_path: File path to save the image. Returns metadata only if not set.

Remove all prims from the scene. Also empties any environment loaded by load_environment, so a later create_physics_scene does not stack a second ground under the first — it always creates one. The stage's defaultLight is always kept — a stage with no light renders black, which looks like a broken camera. Args: keep_physics: If True, keep physics scene prims. keep_environment: If True, keep the loaded environment. Reloading one costs seconds, so pass this when clearing objects between attempts.

Duplicate an existing object to a new prim path. Args: source_path: Prim path of the object to clone. target_path: Prim path for the cloned object. position: [x, y, z] position for the clone. Keeps original position if not set.

Create and wire an OmniGraph Action Graph. Builds a complete Action Graph with nodes, connections and attribute values using og.Controller.edit(). This is the programmatic equivalent of creating an Action Graph in the visual editor. Args: graph_path: USD prim path for the graph (default "/World/ActionGraph"). nodes: List of node definitions. Each dict has: - "path": Node path relative to graph (e.g. "OnPlaybackTick") - "type": OmniGraph node type (e.g. "omni.graph.action.OnPlaybackTick") connections: List of [source_attr, target_attr] pairs for wiring nodes. Each attr is "NodePath.outputs:attrName" or "NodePath.inputs:attrName". values: List of attribute value overrides. Each dict has: - "attr": Full attribute path (e.g. "ScriptNode.inputs:script") - "value": The value to set evaluator: Graph evaluator type (default "execution", what Action Graphs use). "push" evaluates every application update regardless of the timeline, so an OnPlaybackTick-driven ScriptNode would keep running even while the simulation is stopped. script_file: Convenience shortcut — path to a local Python script file. When provided, automatically creates OnPlaybackTick → ScriptNode nodes, wires them, and attaches the script file (sets usePath + scriptPath). The nodes and connections parameters are ignored when script_file is set. RECOMMENDED for anything you will iterate on — edit the file and reload_script "just works", with the better reload story. inline_script: Convenience shortcut — inline Python (must define setup(db)/compute(db)). Auto-creates OnPlaybackTick → ScriptNode, wires them, and sets the script inline (usePath=False). For small, static graphs. For anything you will iterate on, prefer script_file — it has the better reload story (edit the file + reload_script "just works"; inline edits need edit_action_graph). Example (inline script — one-step): create_action_graph( inline_script="def setup(db): pass\ndef compute(db): return True" ) Example (script file — one-step, recommended for iteration): create_action_graph( script_file="/path/to/controller.py" )

Add a camera sensor to the scene. Prefer target= over rotation= for aiming: cameras look down their local -Z and carry a built-in orientation, so hand-computed euler angles are easy to get wrong and give you a picture of the sky. The response echoes the rotation that was applied under "rotation" and the point under "aimed_at". Args: prim_path: Prim path for the camera. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. Ignored if target is given. resolution: [width, height] image resolution. Default 1280x720. target: [x, y, z] world point to look at, using +Z as up. Needs a position — either passed here or already on the prim.

Add a lidar sensor to the scene. Args: prim_path: Prim path for the lidar. position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. config: Lidar configuration name (e.g. "Example_Rotary").

Create a light in the scene. Args: light_type: Type of light — DistantLight, DomeLight, SphereLight, RectLight, DiskLight, or CylinderLight. position: [x, y, z] world position. intensity: Light intensity. color: [r, g, b] light color (0-1). rotation: [rx, ry, rz] rotation in degrees. prim_path: Custom prim path. Auto-generated if not provided.

Create a PBR or physics material. Args: material_type: "pbr" for visual material or "physics" for physics material. prim_path: Prim path for the material. Auto-generated if not set. material_path: Alias for prim_path. apply_material names this argument material_path, and an unknown argument is dropped silently rather than rejected — so asking for material_path="/World/Looks/Red" used to succeed while creating the material somewhere else entirely, and the follow-up apply_material then failed on a path that was never used. color: [r, g, b] diffuse color (0-1). PBR only. roughness: Surface roughness (0-1). PBR only. metallic: Metallic value (0-1). PBR only.

Create a primitive object (Cube, Sphere, Cylinder, Cone, Capsule, Plane). Prefer `size` for absolute sizing: `size` is the target in METERS (default 1.0), so `size=0.3` gives a 0.3 m object regardless of type. `scale` is a RAW MULTIPLIER of the primitive's NATIVE size, not meters. Native sizes: Cube/Sphere/Cylinder/Cone/Capsule = 2 m, Plane = 1 m. So `scale=0.5` on a Cube -> 1 m, and `scale=[0.4,0.4,0.3]` -> a 0.8 x 0.8 x 0.6 m box (0.4 * 2 m), which surprises callers who expect 0.4 m. Use `scale` only for deliberate non-uniform shaping; otherwise use `size`. If both are given, `scale` wins and `size` is ignored. For the geometric prims (Cube, Sphere, Cylinder, Cone, Capsule) this returns prim_path, actual_size [x, y, z] in meters, and bounding_box (min/max corners in world coordinates) so you can accurately place other objects relative to this one. A Plane has no such extent and returns prim_path only. Args: object_type: Type of primitive — Cube, Sphere, Cylinder, Cone, Capsule, or Plane (case-insensitive; "cube" is normalized to "Cube"). position: [x, y, z] world position. rotation: [rx, ry, rz] rotation in degrees. scale: [sx, sy, sz] RAW multiplier of the native size (2 m for most prims, 1 m for Plane). NOT meters. Overrides `size`. size: Target size in METERS (default 1.0). Absolute; independent of the primitive's native size. Ignored if `scale` is provided. color: [r, g, b] color values (0-1). physics_enabled: Enable physics on this object. prim_path: Custom prim path. Auto-generated if not provided.

Create a physics scene with ground plane. Call get_scene_info first to verify connection. Args: gravity: Gravity vector [x, y, z]. Default is standard gravity. scene_name: Name for the physics scene prim.

Create a robot in the scene from the Isaac Sim asset library. Supports fuzzy matching — e.g. "franka", "spot", "g1", "go1". Call list_available_robots first to see all available robots. Call create_physics_scene before creating robots. Returns prim_path, robot_key, joint_names, and num_dof so you can immediately use set_joint_positions without a follow-up get_robot_info call. Args: robot_type: Robot name or search term. Fuzzy matched against available robots. position: [x, y, z] world position. name: Custom name for the robot prim. prim_path: Exact USD prim path (e.g. "/World/Franka"). Overrides name-based path.

Delete an object from the scene. Args: prim_path: The prim path of the object to delete.

Edit an existing OmniGraph Action Graph: set attribute values or add connections. Use this to update ScriptNode scripts (inline or file path), change attribute values, or add new connections on an already-created graph. For ScriptNode with a local file script, set both usePath and scriptPath: values=[ {"attr": "ScriptNode.inputs:usePath", "value": true}, {"attr": "ScriptNode.inputs:scriptPath", "value": "/path/to/script.py"} ] For ScriptNode with inline script: values=[ {"attr": "ScriptNode.inputs:usePath", "value": false}, {"attr": "ScriptNode.inputs:script", "value": "def compute(db): ..."} ] Args: graph_path: USD prim path of the existing graph (default "/World/ActionGraph"). values: List of attribute value overrides. Each dict has: - "attr": Attribute path relative to graph (e.g. "ScriptNode.inputs:script") - "value": The value to set connections: List of [source_attr, target_attr] pairs to add.

Escape hatch: execute arbitrary Python code in Isaac Sim. PREFER named tools over this for: reading/setting joints (set_joint_positions, get_joint_positions), inspecting state (get_prim_info, get_physics_state, get_joint_config), stepping simulation (step_simulation), and checking logs (get_isaac_logs). USE this for: operations no named tool covers, such as creating Action Graphs, computing IK, setting up physics callbacks, or configuring advanced USD properties. CAUTION: touching an articulation controlled by a running ScriptNode / Action Graph can silently break its control path (no error is raised). While a graph is running, read-only diagnostics (get_prim_info, get_physics_state, get_joint_positions, get_isaac_logs) are safe, but stop_simulation before using execute_script or named write tools on the same articulation. For persistent controllers (>20 lines), write a .py file and load it with reload_script instead of pasting code here. Args: code: Python code to execute in the Isaac Sim context. cwd: Optional working directory to add to sys.path before execution.

Generate a 3D model from text or image using Beaver3D, then load it into the scene. Args: text_prompt: Text description for 3D generation. image_url: URL of an image for 3D generation. position: [x, y, z] world position for the generated model. scale: [sx, sy, sz] scale factors.

Diagnostic tool: recent WARN/ERROR logs plus captured print() output. Captures carb.log_*/omni.log WARN+ERROR and stdout from execute_script / reload_script (tagged [PRINT]). Plain print() outside those captured contexts may not appear. Defaults are agent-friendly: non-destructive (clear=False) and scoped to the current run (since_last_play=True) so you see logs from what you just did, not stale entries from previous runs. Args: clear: If True, empty the buffer after reading. Default False. count: Maximum number of log entries to return. since_last_play: If True (default), return only entries since the last timeline Play. Set False for the full buffer.

Diagnostic tool: get joint drive configuration for a robot articulation. Returns stiffness, damping, limits, target vs actual positions, and position error for each joint. Call this when: - Joint drives are not tracking targets (check position_error) - Joints are oscillating or unstable (check stiffness/damping ratio) - Joints hit limits unexpectedly (check lower_limit/upper_limit) Args: prim_path: USD path to the robot articulation root.

Read current joint positions from a robot. Units: radians for revolute joints, meters for prismatic joints. Joint order matches the joint_names from get_robot_info. For a combined step-and-read, prefer step_simulation with observe_joints. Args: prim_path: The prim path of the robot.

Get point cloud data from a lidar sensor. Requires the timeline to be playing — RTX lidar data is produced by Replicator while the sim runs, and a sweep only completes on some frames, so an empty read means "not this frame", not "saw nothing". By default returns a summary rather than the raw cloud: point_count, bounds, and the nearest hit. A full sweep is tens of thousands of points and megabytes of JSON, which is rarely what you want in a response. Args: prim_path: Prim path of the lidar sensor. max_points: Include this many points in the response, sampled at an even stride across the sweep. Omit for summary only. output_path: Write the complete cloud to this .npy file and return its path; numpy.load() reads it back as an (N, 3) array.

Diagnostic tool: get physics state for a prim. Returns rigid body status, velocities, kinematic flag, and collision info. `mass` is included only when the prim carries a UsdPhysics MassAPI — objects created by create_object do not, and take their mass from the collider's density. Velocity units: linear_velocity in m/s, angular_velocity in rad/s. Velocities are only non-zero once the simulation has advanced — step the simulation (or play) before reading them. Call this when: - Objects fall through the ground (check collision enabled) - Objects don't move when expected (check is_kinematic, mass) - Grasping fails (check collision on gripper fingers and target object) Args: prim_path: USD path to the prim to inspect.

Get detailed information about a specific prim. Returns type, children, and a transform block holding position, rotation [rx, ry, rz] in degrees (XYZ order, the same convention transform_object accepts), and scale. For geometric prims (Cube, Sphere, Cylinder, Cone, Capsule), also returns actual_size [x, y, z] in meters accounting for scale and default primitive dimensions. Args: prim_path: The USD prim path to inspect.

Get robot joint information including names, DOF count, joint types, and limits. Call this after create_robot to understand the robot's kinematic structure. Returns joint names ordered by DOF index, joint types (revolute/prismatic), and joint limits (radians for revolute, meters for prismatic — each entry carries its own `units`). Args: prim_path: The prim path of the robot.

Ping the Isaac Sim extension server and return scene information including stage path, assets root, and prim count.

Get the current simulation state: timeline status (playing/stopped/paused), simulation time, and physics dt. step_simulation does NOT require a running timeline — do not play just to step.

Import a robot from a URDF file into the scene. Args: urdf_path: Path to the URDF file. prim_path: Prim path for the imported robot. position: [x, y, z] world position.

List all available robots discovered from the Isaac Sim asset server. Returns robot keys, descriptions, manufacturers, and asset paths. The list is auto-discovered at startup and reflects the actual assets available in your Isaac Sim version.

List all available environments discovered from the Isaac Sim asset server. Includes warehouses, offices, outdoor scenes, and more.

List the prims directly under root_path, optionally filtered by type. One level deep by default, so `list_prims("/")` names /World and /Environment rather than everything inside them — a robot alone is hundreds of prims. The response echoes `recursive` so a shallow answer is never mistaken for a complete one. Pass recursive=True to walk the whole subtree. That is the one you want when checking whether something was really deleted, or when hunting a prim nested under a robot: a Camera at /World/Arm/EyeCam does not appear in a shallow listing of /World. Args: root_path: Root path to start listing from. prim_type: Filter by prim type (e.g. "Mesh", "Xform"). With recursive=True, non-matching prims are still descended into, so a Camera under an Xform is found. recursive: Walk the entire subtree instead of one level.

Load a pre-built environment into the scene. Supports fuzzy matching. Call list_environments first to see available options. Many shipped environments are authored Y-up and/or in centimeters; those are rotated and rescaled to match the stage, and the response reports what was applied under "corrections". It also returns "bounds" with the environment's extent and floor_height, so objects can be placed on the ground without a second query. Read prim_path from the response rather than assuming it — it defaults to a named child of /Environment. Args: environment: Environment name or search term (e.g. "warehouse", "hospital", "office"). prim_path: Prim path for the loaded environment. Defaults to /Environment/<name>, which keeps it separate from the stage's default lighting and lets clear_scene remove it.

Load a USD asset from a URL or file path into the scene. Args: usd_url: URL or local path to the USD file. prim_path: Prim path for the loaded asset. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.

Modify properties of an existing light. Args: prim_path: The prim path of the light to modify. intensity: New intensity value. color: [r, g, b] new light color (0-1).

Pause the physics simulation.

Start the physics simulation.

Force re-scan the asset server for available robots. Use this if new robot assets were added.

Reload a Python controller from a file on disk. Two modes, chosen automatically: - If any Action-Graph ScriptNode references this file (inputs:scriptPath), those ScriptNodes are force-recompiled so your on-disk edits take effect on the running graph. This is how you iterate on a ScriptNode controller. - Otherwise the file is (re-)executed as a standalone controller, the way you would use execute_script for code longer than ~20 lines. Workflow: 1. Write the controller as a .py file (attach via create_action_graph script_file=... for ScriptNode use) 2. reload_script to load / recompile it 3. step_simulation to debug (frozen timeline) or play for a ScriptNode demo 4. Edit the file and reload_script again to iterate The file's directory is auto-added to sys.path. Args: file_path: Path to the Python file on disk. script_file: Alias for file_path. create_action_graph names this argument script_file, and an unknown argument is dropped silently rather than rejected, so both spellings are accepted here instead of one of them quietly doing nothing. module_name: Optional module name to reload (e.g. 'my_controller').

Search the NVIDIA USD asset library by text description, then load the best match. Args: text_prompt: Text description of the 3D asset to search for. target_path: Prim path for the loaded result. position: [x, y, z] world position. scale: [sx, sy, sz] scale factors.

Set target joint positions on a robot via ArticulationAction. Units: radians for revolute joints, meters for prismatic joints (e.g. gripper fingers). Use get_robot_info to discover joint names, types, and limits first. After calling this, use step_simulation to advance and observe the result — do not use play_simulation + sleep. Args: prim_path: The prim path of the robot. joint_positions: List of target joint position values. joint_indices: Optional list of joint indices to set. Sets all joints if not provided.

Configure physics engine parameters. Args: gravity: Gravity vector [x, y, z]. time_step: Physics time step in seconds. gpu_enabled: Enable GPU-accelerated physics.

Advance the simulation by exactly N physics frames on a FROZEN timeline. step is self-contained: it initialises physics on first call and operates on a paused/stopped timeline, so N is always exact and observations correlate to a known frame count. Do NOT call play_simulation before or during the debug loop; step is for a frozen timeline. If the timeline is already playing, step returns an error (a free run cannot be counted frame-by-frame). Use play_simulation ONLY for a final continuous run / ScriptNode-driven demo, never for debugging. Typical debug loop (no play): 1. set_joint_positions to command the robot 2. step_simulation with observe_prims and observe_joints 3. get_joint_config if drives are not tracking correctly 4. get_physics_state if objects are not behaving as expected 5. Adjust and repeat Args: num_steps: Number of simulation frames to step. observe_prims: List of prim paths to observe (returns position + velocity). observe_joints: List of articulation prim paths to observe (returns joint positions).

Stop the physics simulation and reset to spawn state. Resets articulations and rigid bodies to their spawn pose (the state captured at first Play), like the Isaac UI Stop button. Call this to return the scene to a clean starting point before another run.

Set the transform (position, rotation, scale) of an existing object. Args: prim_path: The prim path of the object to transform. position: [x, y, z] new world position. rotation: [rx, ry, rz] new rotation in degrees. scale: [sx, sy, sz] new scale factors.