Sunday, July 21, 2024
AndroidSmart Phones

I just experienced next-level leg tracking and mixed reality with Pico 4

The Pico 4 from TikTok star ByteDance will probably receive at least three exciting new features before 2023 is over. NextPit was able to test out two of them at MWC 2023: new leg trackers and an improved RGB passthrough mode. Here are our first impressions.

Leg tracking for the Pico 4

Using the controllers and the quartet of 6DoF inside-out cameras, the Pico 4 tracks your hands among other things, which is nothing new so far. But the eternal question remains: What about the legs? We were able to check out pre-production units of Pico’s new Leg Trackers at MWC.

Neither the market launch nor the name of the leg trackers has been determined yet, but the design has largely been finalized, apart from the large stickers, of course. At least that is what the manufacturer informed us.

Pico 4 Tracker
This is what the small and featherweight Leg Tracker for the Pico 4 looks like. / © NextPit.

The principle is simple: You strap a tracker around your right and left leg, and voilà! Your digital self has legs. This is not only great for avatars but also offers exciting avenues for new applications. At MWC, we tried a fitness app where you had to chop blocks in “Beat Saber” style using one hand, as well as jump onto approaching blocks using your legs.

Pico 4 Tracker
“Very Swift”: Leg trackers for the Pico 4 are set up in a matter of seconds. / © NextPit.

Thus, the game not only requires a new dimension of body control but also becomes really sweaty after a while. VR or MR definitely offers enormous potential when it comes to fitness applications. According to a Pico spokesperson, the leg trackers could also be attached to the arms, for instance, to further improve accuracy.

Improved RGB passthrough for MR applications

The Pico 4 can also expect a revised RGB passthrough mode within the next three to six months. Here, the VR goggles will mix the image captured via the integrated RGB camera with virtual content. During the MWC demo, the demo room was mostly visible. While this was previously only possible in “2D” and at a poor quality at that, a noticeable revision took place in our test unit. A wall was virtually “broken open” and monsters and dragons came flying into the room.

The biggest challenge for the Pico 4 here is that it only has one 16 MP RGB camera available on the front for the passthrough mode, which sits right between the user’s eyes. This solitary camera only ‘sees’ in 2D. So, the Pico 4 relies on algorithms to calculate depth into the image information and seamlessly combine the camera image with virtual content.

Pico 4 MR with Passthrough
Given the low price, the Pico 4 offers a decent passthrough feature – at least soon. / © NextPit

For things that are located further away, this actually works very well. However, spatial perception is noticeably limited when it comes to close objects. This is noticeable in such instances when you want to grab things from reality based on the passthrough image. In such an environment, the hands tend to grab nothing as it takes some getting used to.

This worked much better with the (and admittedly, three times more expensive) HTC Vive XR Elite in our first hands-on. Not only is the depth perception excellent thanks to the built-in dedicated depth camera, but the image quality is also much better. Printed text is still difficult to read via the Pico 4’s passthrough feature.

With the HTC Vive XR Elite in particular, I could easily grab my phone from the table and write as well as read WhatsApp messages. Do take note that the revised MR feature upgraded the Pico 4 enormously in my eyes, which is a real plus point.

What about the third new feature for the Pico 4?

In the coming months, the Pico 4 will receive another feature that is particularly exciting for content creators. With the help of the smartphone app, it will be possible to determine the precise positional relationship between the smartphone and the VR user. This will, in turn, enable a mixed reality capture function. If you film yourself with your smartphone, the app will then transport you to the virtual world you are currently roaming in the video.

This is supposed to work best with a green screen (chroma keying), although the Pico spokesperson stressed in an interview with NextPit that the green screen is optional. The previously mentioned algorithms for distinguishing foreground and background are supposed to work precisely enough to detach the player from his gaming environment.

What do you think about the new features for the Pico 4? Do you think the inclusion of the passthrough function is long overdue? Or are you not into MR applications at all? Do you imagine doing a fitness workout in virtual reality? I look forward to your opinion in the comments!


This website uses cookies. By continuing to use this site, you accept our use of cookies.