Chapter 3: Sensors and Actuators

Introduction

Physical artificial intelligence (AI) systems—particularly humanoid robots—rely fundamentally on their ability to perceive and interact with the physical world. Unlike purely software-based AI agents that operate in digital environments, physical AI systems must acquire real-time data from their surroundings and execute actions that produce measurable physical consequences. This dual capacity is enabled through two core hardware subsystems: sensors and actuators. Sensors convert environmental stimuli into quantifiable signals, while actuators transform control commands into mechanical motion. The design, selection, and integration of these components are critical determinants of a robot’s operational fidelity, responsiveness, and adaptability.

Role of Sensors in Physical AI Systems

Sensors serve as the perceptual interface between a robot and its environment. In physical AI, perception is not merely data collection but the basis for state estimation, decision-making, and adaptive behavior. A humanoid robot navigating a cluttered space, for instance, must continuously sense its surroundings to avoid collisions, maintain balance, and execute tasks like grasping or manipulation. The quality, diversity, and synchronization of sensor data directly influence the robot's ability to construct an accurate internal model of the world and act upon it reliably.

In contrast to traditional automation systems that operate in highly structured settings with fixed sensing modalities, physical AI systems face dynamic, partially observable, and often unpredictable environments. Therefore, robust multimodal sensing—where data from heterogeneous sources are fused to enhance situational awareness—is essential. This requirement underpins the sensor architectures found in modern humanoid platforms.

Types of Sensors

Vision Sensors

Vision systems are among the most widely deployed sensors in humanoid robotics. These include monocular and stereo cameras, depth cameras (e.g., RGB-D sensors like Microsoft Kinect or Intel RealSense), and event-based cameras. Vision enables object recognition, scene segmentation, motion tracking, and visual servoing. In humanoid robots such as Boston Dynamics’ Atlas or SoftBank’s NAO, vision provides the primary means of environmental mapping and task localization. However, vision systems are susceptible to lighting variations, occlusions, and motion blur, necessitating complementary sensing modalities.

Force and Torque Sensors

Force and torque sensors, typically situated at joints or within end-effectors, measure interaction forces between the robot and its environment. These sensors are indispensable for tasks requiring physical contact, such as grasping, pushing, or compliant manipulation. Six-axis force/torque (F/T) sensors provide three-dimensional force and moment measurements, enabling precise control of contact dynamics. For example, in humanoid robots like Honda’s ASIMO or Toyota’s Human Support Robot, F/T sensors facilitate stable walking by monitoring ground reaction forces at the feet.

Tactile Sensors

Tactile sensors provide localized information about contact pressure, texture, and shape. They are commonly integrated into robotic hands, fingertips, or skins. Tactile feedback allows fine manipulation—such as adjusting grip strength based on object fragility—and enhances safety during human–robot interaction. Capacitive, resistive, piezoelectric, and optical tactile sensors each offer trade-offs in spatial resolution, sensitivity, and durability. The development of artificial skin for humanoid platforms, as seen in projects like the University of Bologna’s iCub or the Shadow Robot Hand, exemplifies efforts to replicate human-like touch.

Inertial Measurement Units (IMUs)

IMUs combine accelerometers, gyroscopes, and sometimes magnetometers to estimate a robot’s orientation, angular velocity, and linear acceleration. They are essential for balance control, especially in bipedal locomotion where maintaining upright posture is inherently unstable. IMUs operate at high frequencies and are relatively low-cost, but they suffer from drift over time due to integration of noisy signals. In humanoid robots like Boston Dynamics’ Atlas or Agility Robotics’ Digit, IMUs provide real-time feedback for dynamic stabilization during walking, running, or recovery from disturbances.

Proprioceptive Sensors

Proprioception refers to a robot’s awareness of its own body configuration. Proprioceptive sensors include joint encoders (measuring angular position), current sensors (inferring torque via motor current), and strain gauges. These sensors do not rely on external stimuli but instead monitor internal states, forming the basis for kinematic and dynamic models used in motion planning and control. Accurate proprioception is crucial for coordinated multi-joint movements, such as those required for humanoid gestures or stair climbing.

Actuators and Motion Generation

Actuators are the mechanical counterparts to sensors, responsible for generating motion and applying forces. In humanoid robotics, actuators must balance power density, precision, bandwidth, and safety. Common actuator types include electric motors (DC, brushless DC, stepper), hydraulic and pneumatic systems, and emerging compliant actuators such as series elastic actuators (SEAs) or variable impedance actuators (VIAs).

Electric motors are prevalent due to their controllability and efficiency. However, their high gear ratios can reduce backdrivability, limiting compliance during unexpected interactions. Hydraulic systems, used in high-performance platforms like Atlas, offer high force-to-weight ratios but introduce complexity in fluid management and sealing. Compliant actuators, which incorporate mechanical compliance (e.g., springs) into the drive train, improve energy efficiency, shock absorption, and force control—critical for safe human–robot interaction.

Motion generation involves translating high-level task goals into low-level actuator commands. This process typically proceeds through hierarchical control layers: trajectory planning defines desired joint paths; inverse dynamics computes required torques; and low-level motor controllers execute the commands while compensating for actuator dynamics and disturbances.

Sensor–Actuator Integration

Effective physical AI requires tight coupling between sensing and actuation—a principle known as sensorimotor integration. This integration enables closed-loop control, where sensor feedback continuously adjusts actuator output to achieve desired behavior. For instance, during walking, foot-mounted force sensors detect ground contact, while IMUs monitor body tilt; this information is used in real time to modulate leg joint torques to maintain balance.

Modern humanoid systems employ centralized or distributed architectures to manage sensor–actuator data flow. Centralized systems simplify coordination but may introduce bottlenecks; distributed systems offer modularity and fault tolerance but require robust inter-module communication. Regardless of architecture, synchronization is paramount: misaligned timestamps between vision frames and joint encoder readings, for example, can degrade state estimation and control performance.

Sensor fusion algorithms—such as Kalman filters, particle filters, or deep learning–based estimators—are commonly used to integrate multimodal data into a coherent state representation. This fused state then informs actuator commands, closing the perception–action loop.

Challenges in Sensing and Actuation

Despite advances, several persistent challenges limit the performance of physical AI systems:

Noise

All physical sensors exhibit measurement noise due to electronic interference, mechanical vibration, or environmental factors. Force sensors may register spurious readings from cable strain; IMUs accumulate drift; vision systems misclassify objects under poor lighting. Robust filtering and outlier rejection are necessary but add computational overhead.

Latency

End-to-end latency—the delay between sensing an event and actuating a response—can critically undermine system stability. In dynamic tasks like running or catching, latencies exceeding tens of milliseconds may lead to failure. Minimizing latency requires co-design of hardware (e.g., high-bandwidth communication buses) and software (e.g., real-time operating systems and streamlined control pipelines).

Calibration

Accurate operation demands precise calibration of both individual sensors and their spatial and temporal relationships. For example, extrinsic calibration aligns camera and IMU coordinate frames; temporal calibration synchronizes data streams across sensors with different sampling rates. Calibration is nontrivial in field-deployed robots, where temperature changes, mechanical wear, or impacts can alter parameters over time. Automated or online calibration methods are an active area of research.

Trade-offs in Actuator Design

Actuators face inherent trade-offs among strength, speed, precision, energy efficiency, and safety. High-torque actuators often sacrifice bandwidth; compliant systems improve safety but reduce positioning accuracy. Selecting or designing actuators appropriate for a given task domain remains a key engineering challenge in humanoid robotics.

Conclusion

Sensors and actuators constitute the foundational hardware layer upon which physical AI systems operate. Their characteristics dictate the perceptual richness and behavioral repertoire of humanoid robots. As the field progresses, advances in sensor miniaturization, actuator compliance, and real-time sensorimotor integration will continue to expand the capabilities of physical AI agents. However, overcoming persistent challenges related to noise, latency, and calibration remains essential for deploying reliable and adaptive robotic systems in real-world environments.