Table of Contents

Headset Screen Recording: Documenting Immersive AR/MR Experiences

When developing, testing, or providing user support with augmented reality (AR) or mixed reality (MR) headsets, screen recording is an important means of reproducing and analyzing issues. However, unlike mobile devices, headset screen recording involves special technologies such as Passthrough and Foveated Rendering. The recorded screen usually does not fully equal the complete stereo view directly seen by the user's eyes. Therefore, when interpreting the recorded content, special attention should be paid to the differences between it and the actual human eye experience.

Why Do Visual Differences Exist in Headset Screen Recordings?

Users must pay special attention: The recorded screen of the headset device does not completely equal the actual picture you see behind the lenses. If you find any abnormalities in the recording, please describe them in combination with your actual feelings to avoid misleading developers.

The following are the main differences:

  1. Resolution and Clarity Differences
  • Recording: Usually a video stream with a resolution of 1080p or lower for a single eye (e.g., the left eye).
  • Human Eye: Depending on the application, the single - eye resolution of modern headsets can reach 2K or even higher.
  • Impact: The blurry text and pictures you see in the recording may not exist inside the headset; conversely, the tiny edge jaggedness you see inside the headset may not be visible at all in the recording.
  1. Field of View (FOV) Differences
  • Recording: The recorded picture is usually square. Especially in devices with a wide field of view, the edge content is often cropped.
  • Human Eye: The picture seen by the human eye is displayed binocularly, with a larger field of view.
  • Impact: If your problem occurs at the edge of the field of view, the recording may not accurately reflect it.
  1. Impact of Foveated Rendering
  • Recording: All content in the full - screen field of view will be included, including the low - resolution edge area.
  • Human Eye: The center of the human eye's gaze is high - resolution, while the non - central edge is low - resolution.
  • Impact: You actually feel that the picture is very clear, but when you review the recording, you find that some areas are blurry. This is usually a normal phenomenon, not a bug. Therefore, when recording, you must focus on the part you want to show, which will ensure that this area is clear.
  1. Impact of Frame Rate and Refresh Rate
  • Recording: The frame rate of the recorded video is usually 30FPS or 60FPS.
  • Human Eye: The actual screen refresh rate of the experience may reach 90Hz or 120Hz.
  • Impact: A very slight drop in frame rate may not be detectable in the recording, but the human eye will clearly feel "stuttering" or "dizziness." If the recording looks smooth but the actual experience is not, this is usually a performance issue, and you must explain it after recording.

Special Instructions for OST Headsets

Special Attention: If you are using an OST (Optical See - Through) headset, such as AR glasses like Rokid and Xreal, using the screen recording function cannot record the real - world content seen by the human eye.

What Are OST Devices?

OST devices project light directly into your eyes through semi - transparent optical lenses, and virtual content is superimposed on the field of view through micro - projection or waveguides, allowing you to see both the real world and the superimposed virtual images. The camera on the device that captures the outside world is only used for spatial positioning, not for display. Due to this mechanism, the device itself cannot "record" the real scene seen by the human eye.

Limitations of OST Device Screen Recording

  • Unable to Record Real Scenes: The screen recording function of OST devices only records the virtual content layer (GUI, 3D models, UI pointers, etc.).
  • Black Background: When you play back this video on your computer or phone, you will see virtual objects on a pure black background, completely losing the context of the real environment.
  • Loss of Occlusion Relationship: If a virtual object should be occluded by a real object (such as your hand or a table), in the recording, they will float on the black screen, looking like an incorrect hierarchical relationship.

Visual Error Issues with Virtual Content Superposition

Even in an ideal environment, there is inevitably a certain degree of visual error in the alignment between the virtual content presented by OST headsets and the real world. Such errors are not always caused by software bugs or tracking failures, but often stem from the fundamental differences between the physical characteristics of the device and the human visual system. Some errors cannot be completely eliminated under the current technical conditions.

When submitting feedback, pay attention to distinguishing the following situations:

  • Optical Alignment Error:
    • Phenomenon: When you move your head, virtual objects seem to "float" or have a slight deviation from the position of real objects.
    • Cause: The user looks directly at the real world through the optical lenses, while the rendering coordinate system of virtual content is usually built based on the head - tracking and environmental understanding system (such as SLAM). Due to the physical offset between the optical display light path and the camera/IMU used for spatial positioning, and the difficulty of achieving sub - pixel joint calibration, the alignment of virtual and real objects is particularly sensitive in the edge field of view or close - range scenarios.
  • Limitations of Human Eye Calibration:
    • Phenomenon: Virtual objects look like they have ghosting, blurred edges, or you feel that the image is not projected correctly in space.
    • Cause: OST devices rely extremely on accurate inter - pupil distance (IPD) and eyeball position calibration. If the device is not accurately calibrated for your eyes, or if the position of your helmet has shifted, the light projected by the optical engine cannot accurately enter your pupils. However, even if the device has calibration, the calibration process is based on limited sampling points, making it difficult to precisely match the eyeball geometry, corneal curvature, and visual perception habits of each user. Small calibration deviations will cause systematic offsets of virtual objects in the depth or horizontal position.
  • Individual Visual Differences:
    • Phenomenon: Different users may have different subjective judgments on the position of virtual content.
    • Cause: The vision (such as myopia, astigmatism) and dynamic focusing ability of different users will affect the subjective judgment of observing the superposition effect of virtual content. OST devices may lack real - time eye - tracking and personalized optical correction capabilities, so they cannot dynamically compensate for these differences for each user.
  • Environmental Factor Interference:
    • Phenomenon: In a bright environment, the virtual content becomes faint, unclear, or even invisible; in a dark environment, the virtual content is over - exposed or even has ghosting.
    • Cause: OST devices directly superimpose the virtual light from micro - projection or waveguides with the ambient light of the real world. If the real environment is too bright, it will override the weak virtual light; vice versa. This optical artifact is an inherent optical characteristic of OST devices.
Note

Such errors are inherent characteristics of OST AR glasses technology, not functional failures. When troubleshooting problems, please distinguish between "repairable software/tracking problems" and "experience boundaries limited by hardware and physiology." If user feedback involves slight misalignment, edge distortion, or inconsistent depth perception, it is recommended to first confirm whether it is within the device's nominal usage conditions (such as working distance, illumination range, calibration status).

How to Handle Feedback for OST Devices?

If you encounter visual problems when using an OST device, after ruling out the factors caused by the above device limitations, you can record the screen for feedback. However, simply submitting the recorded content is usually not enough:

  1. Accompany with Photos: Please use your phone to take a photo of the actual picture you see through the lenses (called a "first - person view photo"). This can show the relative position of virtual objects and the real environment.
  2. Describe the Environment: Describe in detail the environment you are in (lighting conditions, background color, whether there is movement such as pedestrians, etc.), because the display effect of OST is extremely dependent on ambient light.

Screen Recording Methods for Common Headset Devices

Note

During the screen recording process, the device may reduce the rendering frame rate or resolution, affecting the judgment of smoothness and tracking stability. It is recommended to record as soon as the problem is reproduced to avoid the impact of long - term operation on the results.

Apple Vision Pro

  1. Look up at the top of the screen until you see the control center point. Focus on this point and click.
  2. Gaze and click on Control Center > the "Record My View" button to start recording.
  3. To stop recording, open the control center and tap "Record My View" again.
  4. Your view recording will be stored in the "Photos" App .

PICO 4 Ultra

  1. Open Control Center > Settings > General > Screen Mirroring, Recording, and Screenshots. Select the recording mode as "Space" in "Screen Recording and Screenshots."
  2. Press the Home button on the controller briefly. Select the "Record" button in the pop - up menu on the screen and pull the trigger to start recording.
  3. At this time, the system will give a countdown prompt to start. Immediately return to the interface of the application to be recorded, and the recording will start automatically after the countdown ends.
  4. Press the Home button on the controller briefly again, and select the "Record" button again to stop recording.
  5. Your recorded files are saved in the "Internal Storage/DCIM/ScreenRecord" directory of the device by default.

Rokid Max Pro

  1. Enter the desktop, find the bottom status bar, and click the "Quick Settings" area next to the user avatar.
  2. Click the "Record" button in the pop - up window and immediately return to the interface of the application to be recorded. The system will start recording automatically.
  3. Click the X button on the Station Pro to return to the desktop. Click the "Record" button in the "Quick Settings" area again to stop recording.
  4. Your recorded files are saved in the "Internal Shared Storage/ScreenRecorder" directory of the device by default.

XREAL Air2 Ultra

  1. Swipe down on the Beam Pro screen to call up the control center of the glasses and click the "Record" button at the top.
  2. At this time, the system will give a countdown prompt. Immediately return to the interface of the application to be recorded.
  3. Click the red "Stop" button floating at the top of the screen to stop recording.
  4. Your recorded files are saved in the "Internal Shared Storage/Movies/Record" directory of the device by default.

Best Practice Recommendations

When using headset screen recording to feedback problems, pay attention to the following points:

  • Clearly indicate the device model, screen recording method, and environmental factors when submitting.
  • If the problem involves depth, occlusion, or tracking jitter, supplement with a written description of the actual phenomenon observed by the human eye, because the screen recording may not accurately present it.
  • For key scenarios, you can use an external camera to record the real - world operation of the user wearing the headset to provide more comprehensive context.

Although headset screen recording is convenient, it is always an approximate restoration of the experience. The most reliable diagnosis still needs to combine logs, sensor data, and user subjective feedback. For information on log collection methods, please refer to:

  • [Android Log Recording](log - android.md)
  • [iOS Log Recording](log - ios.md)
  • [Unity Log Recording](log - windows.md)