In one embodiment, a computing system may determine a device pose of a handheld device in a real-world space based on one or more sensors associated with the handheld device. The system may determine a head pose of a first user associated with the handheld device based on face-tracking data of the handheld device, the head pose being with respect to the handheld device in the real-world space. The system may render a first avatar associated with the first user in a virtual space based on the head pose of the first user in the real-world space. The system may render one or more virtual objects in the virtual space to the first user based on the head pose of the first user in the real-world space.