AR, VR, And A Model For 3D In HTML

Republished By Plato

Followers: 0

Tucked down somewhere in the Safari Technology Preview 161 release notes is a seemingly innocous line about support for a new HTML element and attribute:

Added support for <model src> and honor <source type> attributes ([email protected])

Anytime I see mention of some element I don’t recognize, my mind goes straight to Huh! New to me, but probably old news for everyone else. It’s poor posture, I know, as it could just as easily be:

Hmm, looks like some propriatary experiment.
Wow, a truly new thing!

Truth is, it’s sorta all three.

It’s an evolving concept

As in, the first somewhat official-sounding thing I found on <model> wasn’t in the W3C spec but in WebKit’s repo for explainers. All that’s in the README is a giant note from 2021 that “The <model> element has moved to the Immersive Web CG.” I was about to hop over but my eye caught the HistoryAndEvolution.md file which has a nice rundown of early context on the <model> concept:

The <model> element was born out of a desire to take the next step and improve the experience of Safari’s integration with iOS’s AR Quick Look feature.

I had to look at Apple’s splash page for AR Quick Look. You know the new feature that some stores have where you can transpose a 3D rendering of a product in your own home using your phone camera? That’s the sort of stuff we’re talking about, and Apple links up a nice case study from the Metropolitan Museum of Art.

As I understand it from this limited context:

Drop a <model> element in the document.
Add an external source file, e.g. <model src="assets/example.usdz">.

The original proposal is from the Immersive Web Committee Group

That’s the team looking make Virtual Reality (VR) and Augmented Reality (AR) part of the web. Apple linked up their repo, so I made the jump and went straight to the explainer. This isn’t the spec or anything, but the original proposal. A much better definition of the element!

HTML allows the display of many media types through elements such as <img>, <picture>, or <video>, but it does not provide a declarative manner to directly display 3D content. Embedding 3D content within a page is comparatively cumbersome and relies on scripting the <canvas> element. We believe it is time to put 3D models on equal footing with other, already supported, media types.

[…]

The HTML <model> element aims to allow a website to embed interactive 3D models as conveniently as any other visual media. Models are expected to be created by 3D authoring tools or generated dynamically, but served as a standalone resource by the server.

The basic example pulls this together. It really does feel like the <video> or <picture> elements:

<model style="width: 400px; height: 300px"> <source src="assets/example.usdz" type="model/vnd.usdz+zip"> <source src="assets/example.glb" type="model/gltf-binary">
</model>

.usdz? .glb? Not the type of files that typically cross my desk. Guess I’ll need to brush up on those and any other file types that <model> might support. Again, all of this is merely the original proposal.

The draft proposal isn’t stubbed out quite yet

But it does provide a nice outline of where things could possibly go:

Adding a model to a document
Enabling interactivity
Supporting multiple formats
Providing fallback content
Making it accessible

There’s a lot to figure out. Most of what’s there are documented issues that need addressing. It does, however, shed more light on <model> like proposed attributes that make it feel even more like <video> such as autoplay, controls, loop, muted, poster, etc.

It goes back even further

The very earliest mention of 3D modeling I found was Keith Clark’s 2018 post in which he prototypes a custom element called <x-model>. He describes it as “a placeholder that provides access to the DOM and CSSOM” where the loading and rendering is done in three.js.

Keith’s idea is followed by the <model-viewer> component Joe Medley shared in 2020 (and a subsequent update to it). There’s even a homepage for it and it’s fun to drag Neil Armstrong around in space.

It’s possibly just an experiment?

I mean, the draft spec hasn’t been fleshed out. Apple seems willing to play ball thanks to the Safari TP 161 announcement. That makes total sense given how bullish Apple is on AR as a whole. (Apple Glasses, anyone?)

Google seems to have its foot in the door, albeit on the Web Components side of things. It’s easy to see how there may be a conflict of interest between what Apple and Google want from AR on the web.

These are all just my notes from trying to grok everything. There’s gotta be a lot more nuance to it than what little I know about it so far. I’m sure someone smarter can tie neater bow around <model> in the comments. 😉

And while we’re talking Safari Technology Preview, 162 just released the other day and it enables CSS nesting and the CSS relative color syntax.