▲ 1915 points
back
698 comments
Foveated streaming! That's a great idea. Foveated rendering is complicated to implement with current rendering APIs in a way that actually improves performance, but foveated streaming seems like a much easier win that applies to all content automatically. And the dedicated 6 GHz dongle should do a much better job at streaming than typical wifi routers.
> Just like any SteamOS device, install your own apps, open a browser, do what you want: It's your PC.
It's an ARM Linux PC that presumably gives you root access, in addition to being a VR headset. And it has an SD card slot for storage expansion. Very cool, should be very hackable. Very unlike every other standalone VR headset.
> 2160 x 2160 LCD (per eye) 72-144Hz refresh rate
Roughly equivalent resolution to Quest 3 and less than Vision Pro. This won't be suitable as a monitor replacement for general desktop use. But the price is hopefully low. I'd love to see a high-end option with higher resolution displays in the future, good enough for monitor replacement.
> Monochrome passthrough
So AR is not a focus here, which makes sense. However:
> User accessible front expansion port w/ Dual high speed camera interface (8 lanes @ 2.5Gbps MIPI) / PCIe Gen 4 interface (1-lane)
Full color AR could be done as an optional expansion pack. And I can imagine people might come up with other fun things to put in there. Mouth tracking?
One thing I don't see here is optional tracking pucks for tracking objects or full body tracking. That's something the SteamVR Lighthouse tracking ecosystem had, and the Pico standalone headset also has it.
More detail from the LTT video: Apparently it can run Android APKs too? Quest compatibility layer maybe? There's an optional accessory kit that adds a top strap (I'm surprised it isn't standard) and palm straps that enable using the controllers in the style of the Valve Index's "knuckles" controllers.
> Foveated streaming! That's a great idea.
Back when I was in Uni, so late 80s or early 90s, my dad was Project Manager on an Air Force project for a new F-111 flight simulator, when Australia upgraded the avionics on their F-111 fighter/bombers.
The sim cockpit had a spherical dome screen and a pair of Silicon Graphics Reality Engines. One of them projected an image across the entire screen at a relatively low resolution. The other projector was on a turret that pan/tilted with the pilot's helmet, and projected a high resolution image but only in a perhaps 1.5m circle directly in from of where the helmet was aimed.
It was super fun being the project manager's kid, and getting to "play with it" on weekends sometimes. You could see what was happening while wearing the helmet and sitting in the seat if you tried - mostly ny intentionally pointing your eyes in a different direction to your head - but when you were "flying around" it was totally believable, and it _looked_ like everything was high resolution. It was also fun watching other people fly it, and being able to see where they were looking, and where they weren't looking and the enemy was speaking up on them.
> 2160 x 2160 LCD (per eye) 72-144Hz refresh rate
I question that we could not create a special purpose video codec that handles this without trickery. The "per eye" part sounds spooky at first, but how much information is typically different between these frames? The mutual information is probably 90%+ in most VR games.
If we were to enhance something like x264 to encode the 2nd display as a residual of the 1st display, this could become much more feasible from a channel capacity standpoint. Video codecs already employ a lot of tricks to make adjacent frames that are nearly identical occupy negligible space.
This seems very similar (identical?) to the problem of efficiently encoding a 3d movie:
Foveated streaming is cool. FWIW the Vision Pro does that for their Mac virtual display as well, and it works really well to pump a lot more pixels through.
Foveated streaming is wild to me. Saccades are commonly as low as 20-30ms when reading text, so guaranteeing that latency over 2.4Ghz seems Sisyphean.
I wonder if they have an ML model doing partial upscaling until the eyetracking state is propagated and the full resolution image under the new fovea position is available. It also makes me wonder if there's some way to do neural compression of the peripheral vision optimized for a nice balance between peripheral vision and hints in the embedding to allow for nicer upscaling.
> Roughly equivalent resolution to Quest 3 and less than Vision Pro. This won't be suitable as a monitor replacement for general desktop use. But the price is hopefully low.
Question, what is the criteria for deciding this to be the case? Could you not just move your face closer to the virtual screen to see finer details?
And foveated streaming has a 1-2ms wireless latency on modern GPUs according to LTT. Insane.
> Roughly equivalent resolution to Quest 3 and less than Vision Pro. This won't be suitable as a monitor replacement for general desktop use.
The real limiting factor is more likely to be having a large headset on your face for an extended period of time, combined with a battery that isn't meant for all-day use. The resolution is fine. We went decades with low resolution monitors. Just zoom in or bring it closer.
Why hasn't Meta tried this given the huge amount of R&D they've put into VR and they had literally John Carmack on the team in the past?
I use a 1920x1080 headset as a monitor replacement. It's absolutely fine. 2160x2160 will be more than workable as long as the tracking is on point.
This is the first standalone headset with an open ecosystem. That's a big deal.
Meta Quests & Apple Visions require developer verification to run your own software, and provide no root access, which slowed down innovation significantly.
Frame is obviously the main headline here, but they've also launching a new SteamOS mini-PC and a new controller.
https://store.steampowered.com/sale/steammachine
https://store.steampowered.com/sale/steamcontroller
No prices listed for any of them yet, as far as I can tell.
This is going to be an instant buy for me, and my first VR device ever. I've used the previous Steam VR headset over at a friends' place many times, but never bit the bullet to get one myself.
The fact that this can run standalone, doesn't have a bunch of wires dangling from it, and is pretty much a fully working Linux box makes this am almost on-brainer for me.
I do _hope_ the price is reasonable though, if it ends up being like Apple VR I might not buy into it immediately, but I'm hoping for a reasonable $1000 max price.
Such a miss not having good full-color AR included. I’m a VR enthusiast with a Meta Quest 3, and it’s a shame that this headset is better than the Quest in every way except the most important one.
In my opinion, VR gaming never becomes more than a gimmick. It adds a questionable improvement in graphics and immersion at the incredibly high cost of excluding yourself from the real world. Right now it’s not worth it, and I don’t think it ever will be, no matter how good the graphics get. That’s assuming they even solve the motion sickness problem, which doesn’t seem solvable to me at this point.
The motion controls in VR will also always be severely limited by the fact that you can’t see your surroundings. You can’t meaningfully move around or swing your arms fast in any realistic home environment when you’re in full VR. You’re constantly at risk of punching something or breaking something, or both. So the controls have to become really stiff and avoid requiring wide movement, at which point you might as well just push buttons on a gamepad.
But AR is a completely different thing. No motion sickness, no risk in any movement, you can move around without silly threadmills, and no exclusion from the world. It’s truly amazing. The AR boxing, pickleball, ping pong and golf are so much closer to real thing then to a videogame adaptation, even the shitty Quest graphics don't ruin the magic. Those AR experiences don't work on videogame rules and really deserve their own name and category - they're as different from gaming as books are from movies. If VR headsets don’t die out, AR is going to be the thing that brings them to the mainstream. I just wish it had more attention, more apps, and more non-Meta mainstream platforms. Not this time, sadly.
The whole "foveated streaming" sounds absolutely fascinating. If they can actually pull off doing it accurately in real time, that would be incredible. I can't even imagine the technical work behind the scenes to make it all work.
I'd really like to know what the experience is like of using it, both for games and something like video.
The dedicated communication dongle between the PC and the headset sounds like a real game changer.
Right now getting fast enough and reliable wireless connection means either tweaking to death one's setup or spending car money on the entire setup. In particular normal people usually don't realize how crappy their wi-fi and assume it's all the same, which would end in blaming the poor perf on the headset.
This is fantastic!
A while ago I bought the Quest 3 and set it up with WiFi 6 for streaming games. It's a decent setup, but I only bought it cause I was tired of waiting for the "rumored new headset by Valve".
And it seems everything on my wishlist is here:
- foveated rendering based on eye tracking - this is excellent, and was I think only available in the Quest Pro until now
- a dedicated wireless streaming dongle, with multiple radios on the headset - awesome, tuning WiFi 6 got me to a good-enough state, but I'm looking forward to a dedicated out-of-the-box solution
- pancake lenses
- inside-out tracking
In general, having had the Valve Index previously, and then using the Quest 3, it's a night-and-day difference to play something like Alyx wireless. Much better clarity with pancake lenses, too.
Main surprise here is their usage of a Snapdragon chip and not AMD, didn't expect this. I thought it would effectively be a steam deck hardware wise. Curious to see how well that works, esp. for standalone gaming. In practice though you'll likely want to be streaming any "pc-first" titles anyway.
This being a whole system that will allow you to put whatever software you want onto it makes me think that it might actually succeed at being what the Vision Pro wanted to be.
Now that they've ported Steam to Android with FEX + Proton [0] (what this is running), the question is will they release it for the rest of Android devices? There is a ton of Android gaming handhelds and people are already experimenting with things like Winlator [1] but having well supported way could be awesome.
It really cannot be understated how big of a deal this is. The tech here is impressive AND they're not trying to lock you in. Practically speaking it's the only VR headset that holds real-world appeal for the vast majority of gamers, because it plays the games they're already playing.
At last! I really enjoyed my time with the Oculus Quest 2, but could not stomach having Meta in my house/on my network. I sold it and resolved to either wait until I could get a good deal on an Index or Valve came around with something new, and now I can look forward to VR again!
Whoo - first party support - including a graphics stack on ARM!
I hope this means the GPU and drivers is advanced enough to run fully featured modern video games.
Windows for ARM was kinda sunk by the fact that the GPU wasn't compatible enough due to the crappy drivers and outdated GPU uArch optimized for mobile games.
I'm still kinda on the fence about VR, but I hope ARM + Linux succeeds in a big way and this'll make a truly handheld Steam Deck possible.
Brilliant, I'm anxiously awaiting Australian pricing details (and release dates...) but could definitely seeing myself getting one of these as my first VR device, and the controller looks great too.
Being able to run games on device (and on ARM) is very cool, but I wonder if there is a cheaper/lighter/longer-battery-life version of this that is stream only? That's probably a better fit for me personally, I can't imagine not having a streaming device nearby when I would be using it.
Also hate to be picky, but looks like the frame controllers pair directly to the headset so maybe can't be used on their own? Would be nice to use them standalone too.
One of the most interesting parts of this to me that not many people are talking about is what their approach will be to making Steam games work on ARM…
I assume they will have put a lot of work into an emulation layer (maybe an existing one like FEX) to make it usable similar to what they did with Proton? This could be really good for the Linux ARM ecosystem in general
Steam Frame is running SteamOS on ARM, and is capable of playing games standalone, which implies ARM support in Steam. Through granted, it could be in a limited form.
I've actually just recently started using my Quest 3 again through virtual desktop. I found the sharpening feature to actually make text look good enough that I'm able to work in it for hours uninterrupted, mostly writing papers and coding.
I'm super excited for this launch and for all the crazy open source builds, mods, and fun that are going to come from an open VR system (or at least that's my hope).
Nice to finally have a hackable headset on the market. I’ve been using the dji goggles for fpv flying because of vendor lock in. I wonder I could use these with OpenIPC to fly fpv with a fully open video stack.
I'm surprised no one is talking about the fact that the headset is ARM and that valve has been heavily contributing to the translation layer FEX.
I love my steam deck, but lately find myself reaching for emulation handhelds like the Retroid Pocket 5 more due to smaller size, especially when I'm leaving the house. There's already projects like GameNative that try to hack steam onto these devices, but if valve offers an official client on Android and other arm devices that would be incredible.
Edit: Some interesting insights in the FEX FAQ about why it's not a great fit for Android right now [0]. Interested to see if this ARM version of steamos is installable on other devices though. RP5 can already run alternatives like Rocknix
> Steam Frame Controllers: Powered by AA batteries (rated 40 hours)
Yes, I want to see standard batteries being used more. Too bad they didn't go with this on the Steam Controller.
It felt like the headset hardware had reached a sweet spot for the moment, but with this announcement the software situation is now also going to be reaching a similar place. Now we have a continuity of freedom with a PC OS on the headset away from home as well as the full mobility of a untethered headset display with the full power of a stationary PC at home. Can't wait to see what dev's do with this.
This feels like a really great headset. It was going to be an instant buy for me until I realised it has LCDs instead of OLEDs. I'll wait for reviews. I don't really want to buy these if they release another OLED version down the line.
Also, if this is arm and it has steam in it, that means we can finally run steam on arm, which means we can finally install steam "natively" on android Linux, specially now that we have the terminal app on android 16.
Can't wait for graphical acceleration to be fully merged to the terminal app and we have Linux running on android with near native performance and steam.
That mini pc... one more nail in the coffin of the xbox hardware business. Ouch.
Deckard at last! https://isvalvedeckardout.com/
The only really incredible VR experience I have had so far was Half-life Alyx. Is there anything that tier or even better these days?
That's gonna be one expensive BeatSaber machine, but after having owned an Index (and really feeling how nice that headset was when I got my current Quest 2), there's no way I'm not gonna get one.
Kind of weird using AA batteries, I'd imagine something else would be better suited for this?
This looks fantastic. The only negative I see so far it is only monochrome passthrough. The full color passthrough on the Quest 3 is pretty killer, it sucks to go backwards on that.
A resolution of 2160@110° is kind of low normally, effectively being around 20 pixels per degree. I wonder if they could pull off something like what the engineers behind the Simula VR headset did, where they shift the outer resolution to the centre of the lens, up to 35.5 PPD, higher than the Apple Vision Pro. Last I checked, it was difficult to find off-the-shelf VR displays suitable for pancake lenses with a resolution above 2160, so this might be the best option.
This explains why alyssa rosenzweig (from asahi linux) was paid for so long by Valve. She worked on FEX.
Will the eye tracking data be directly available to developers?
Also, can I hack the OS? Specifically interested in direct VR rendering (other headsets don't allow to bypass compositor).
Wonder if/when prescription lenses will be available for it. I had to get some for my index since my glasses were too big to fit inside the headset.
kudos to them for using AA batteries for the controller.
will help the hardware last longer. cz non-removable lithium batteries suck.
>2160 x 2160 LCD (per eye)
Here's hoping it will be like the Deck and we get Frame OLED in a year or so.
It is unfortunate that it doesn't have a strap that goes over the top of your head. It is extremely irritating that almost every headset does this, then charges extra for an actually usable strap that takes the pressure off of your face.
It's interesting to see them putting more attention on playing traditional games with this. I have long thought that the most broadly viable use for VR headsets in gaming is giving users a big screen to play their regular games. There just doesn't actually appear to be much market for true VR games considering all the complicating factors like motion sickness or requiring a big play area. It reminds me of the Nintendo Wii. Taking turns playing Beat Saber with a few of your friends is fun just like Wii Sports was, but in the end people are going to spend a lot more time sitting on their couch to play something more traditional like Mario Kart.
If it all launches with Half-Life 3 that works seamlessly across all 3 Steam platforms then we're living in the best timeline...
YouTube just happened to show me Gamers Nexus Lossless Scaling & Lossless Frame Generation video just now. [1]
Which, I can't help but wonder. If this ran on the Frame... To aid in streaming from your desktop... In conjunction with Foveated Streaming...
What's even more bonkers to me is the idea that this could really be baked in to the streaming, if you think about it.
Server generates 100 beautiful frames in one second.
The streaming hardware reminds the server that with the current network conditions, that it can only handle X bits per second.
The Frame says, "Hey, here's where the eyes were looking, here's where they are looking, here's where they're projected to look."
The Server knows how many actual frames it can send in a second.
But the Server can run the same Lossless Scaling & Lossless Frame Generation that the client will run.
The Server can then compare the predicted frames against the actual frames, and it can send the delta needed to correct the predictions. Especially in areas close to where the eyes are looking.
The Client consumes the stream, predicts frames, applies the delta, and ends up with better framerate and quality...
Can you do this all low latency, too? I suspect so.
Add it all up, and this is pretty bonkers. I mean, it's kind of obvious that that's how streaming "should work," but I think I hadn't really put it all together yet in my mind. My mental model was too "off the shelf." It didn't include how much effort the client could do... And that the server could then also do, to help the client... It's really a beautiful pairing, when you think about it.
What's really bonkers is that Steam could learn what final images are supposed to look like, for a given game, and train AI specifically for that game. Update the server and the client, and bam, everyone gets better quality. With zero developer interaction.
I'm unreasonably excited on all things Steam nowadays. I still like my PS5. And the PSVR2 is quite amazing for the games it has. But Steam has been amazing in getting back into games for me in ways that I did not anticipate.
2160x2160 in each eye for the headset