EXPERIMENTAL: Virtual Reality

Is it something like this?

Is it something like this?

 

It's kind of like those examples you gave, but Reality Mixer is different and real-time passthrough.

In the 2 videos you gave, they are using a camera mounted near their chest and recording their greenscreened steering wheel in real life, and then in post processing software like Vegas or Photoshop they are manually chromakeying the video and laying it over in the game, and actually that is not real-time mixed reality at all.

Over the years, I have seen many iterations of mixed reality steering wheels, like when people get a greenscreen behind their wheel or literally painted their wheel green or something But Reality Mixer + Quest 3 passthrough quality is something different. I'd say Quest 3 has the best color passthrough since Meta's software for recreating stereoscopic vision is the most accurate out of anything I've tried.

This is the best video example of Reality Mixer I could find.

My main point is that you do not need an extra camera and physical greenscreen in real life to do this.

 

Overall I get it) it wouldn't be bad if you could film it the way it looks for you)

 

just to give my 2 cents as i've been working on getting hand tracking to work in beam VR aswell..in addition to the post above about being able to see your hands in mixed reality- its possible to get hand tracking working(no need for AR/passthrough) on the quest 1 and 2(and i'd assume 3) with openXR toolkit(no need for paid virtual desktop) with some fine tuning of text files basically.. managed to get it to work with assetto corsa on my quest 2 successfully once so far and controller tracking works already in Beam along with them noting some headsets might also already have working hand tracking.. so it's a matter of figuring out what needs to be modified in the files concerning openXR- unless the devs update it themselves at a later point anyway*

*assetto was confusing enough but with all the interactive stuff in beam it would be really useful.

 

There is a video in their post showing how it looks.

 

It's weird, but it's not showing up on my screen... well, that's fine.

 

O thats dumb lol Sorry. Here's a normal link you can paste yourself (remove the spaces): https:// youtu.be /7_X1wReZHgc?si=ZZYXKYXBzxaMvfsV

https:// streamable .com/ketwu9

 

https://www.theregister.com/2023/12/27/windows_mixed_reality_is_dead/

I take it there's not much point in buying a vr with a direct connection?

 

I was able to get VR working on Linux through Proton by editing some of the code of Proton. I have written more detail about the process and I have provided the version of Proton that I compiled here.

 

I made a longer video to demonstrate Beam.NG combined with Reality Mixer and a Quest 3 with color passthrough. (More details in youtube description. Srry for bad recording quality, I actually can run the game with 'Ultra' graphics resolution on Virtual Desktop)

This mixed reality setup takes the best of both worlds from VR sims and triple screen setups; you get the immersive 6DOF viewing experience from your VR headset, and the practicality of seeing your own hands and controls like from a triples setup...

 

Fun) but doesn't it affect the immersion of the game a lot? Also wanted to ask, do you feel the input lag on wifi? Gameplay is not too viscous, gelatinous? I'll be joining VR Beamng soon too) Ordered myself a quest 3)
What kind of Fow are you using, how many degrees?

 

well you'd still benefit from a wired connection mainly due to no interference and less latency.

 

On cable too the latency is quite high they say due to encoding decoding...hope I don't notice it

 

I think it improves the immersion, because without it you just see an empty steering wheel and no hands or arms. The immersion could be improved by better shaping of the passthrough barriers, currently I am only able to do cubes and not circles or curved shapes.

The latency is very playable with wireless Virtual Desktop, you just need your own router to be close to you in your room. My router is behind my monitor connected to my pc via ethernet.
The beamng devs also said something about wireless working better than wired for some reason. I love using a wireless connection and to not have to wires coming off my hmd. Back in the day, you had to buy 3rd party modules for hundreds of dollars just to get wireless capabilities and now it's built into the headset... it's a great upgrade to not be tethered by a cord. But for sim racing you are sitting 99% of the time so a cord would be logically optimal if positioned correctly on the chair headrest to the point where its unnoticeable and you would get the lowest possible latency, but only when beamng devs confirm it's as good as the wireless codec.

The fov is ever so slightly higher than the Quest 2, and the screen door effect is reduced slightly. It's not that noticeable of an upgrade over the 2. The killer features of the Quest 3 is the more accurate IPD adjustment knob, and the color passthrough for simulation applications.

 

It works flawlessly* with a Quest 3 and the Steam VR app from the Meta App Store.

(*only when you’re driving with no AI traffic, the UI starts freaking out when it’s enabled)

 

Whenever I go into driver cam, the car is see-through within a certain radius around me, sometimes I can't see the steering wheel. Changing seat pos, or camera hasn't helped.

 

Dear VR dev(s),

Please implement use of (or fix whatever issue is blocking their loading, e.g. fully respecting xrGetVulkanDeviceExtensionsKHR output):

VK_KHR_external_memory_fd
VK_KHR_external_semaphore_fd
VK_KHR_external_fence_fd

Or better yet, use XR_KHR_vulkan_enable2 (all of this per a HMD driver dev).

Pretty please?

Respectfully,
1 of the 4 linux nerds who also want to run native VR

 

That would be awesome. For reference, I'm using Monado:
https://monado.pages.freedesktop.org/monado/vulkan-extensions.html

It appears those 3 are the only linux-specific called out as required. Another user in the linux thread (@pranays123 ) also noted this and resorted to hand-patching Proton to get it running in Steam. So apparently this would benefit all, not just those going for native capability.