SoftspaceAR | Prototype03
Introducing the ordinospatial layout.
Softspace harnesses spatial computing to help thinkers and makers better see, understand, and develop the ideas that form the heart of their creative projects.
This is the third in a series of prototypes we’re releasing as a part of our open development process to build the augmented reality version of Softspace.
Help us invent a powerful new kind of tool for thought by 🧑🏽🔬 trying these prototypes, 🐦 following us on Twitter, and 👯♀️ joining the Softspace Discord.
SoftspaceAR Prototype03 just dropped! You can download it here and install it with SideQuest or the Oculus Developer Hub desktop app.
The Short Version
Prototype03 is about making Softspace deeply interoperable with your other tools.
Having a spatial canvas to organize your ideas on is amazing, but spatial layouts can be hard or impossible to translate to the other formats and devices in your workflow. And if the work you do in an app gets stuck there, you’re (justifiably) not going to use it very much.
With Prototype03, I wanted to figure out how Softspace might overcome this problem without simply becoming a VR version of an existing 2D app. Could we keep that immersive spatial magic, while reading from and writing to a file format that’s highly portable?
Like markdown?
Making Softspace workspaces interpretable as markdown files—and vice versa—would open up super exciting ways to fit Softspace into knowledge workflows. For example:
You could throw research notes and images into a Dropbox folder, use Softspace to synthesize them into a project proposal, and then directly email that proposal to your team.
You could work on your Roam/Notion/Obsidian/LogSeq/Athens/etc. personal knowledge repository in Softspace.
You could start brainstorming for a Substack post in Softspace, then continue editing the post on your phone while riding the subway 😉.
All this sounds great—but there’s a catch.
Markdown, like all text formats, is inherently ordinal. Softspace, running in augmented reality, is inherently spatial. Ordinality requires that each element have a definite place in a linear sequence, but highly spatial layouts want to break free of such constraints.
Prototype03 reconciles these demands with a 3D layout system that is strictly ordinal, while letting you move a text block in any of the three spatial dimensions.
Words can only go so far in conveying how any 3D UI works, so I encourage you to get the prototype and let me know what you think (via Twitter, Discord, or email).
Or, you can do that after reading this deeper dive 💪🏽🧠👇🏼.
Layout Systems
Softspace is a tool for working with the ideas and knowledge that you already use in your projects. This knowledge takes the form of text, images, websites, PDFs, videos, and other content. To position all this content in 3D space, we need to use a layout system.
Ideally, this layout system places things according to how they’re related to each other in the underlying data structure. This way, you can understand those relations just by looking at where things are, and you can modify them by moving things around.
For example: the tasks in your favorite to-do app are laid out in the order that they’re saved in the underlying database. If you drag in item into another position, the database is updated to match. Order == priority, so this interaction lets you reprioritize upcoming tasks.
Across knowledge management tools, notes are a core format. Therefore, to make Softspace highly interoperable with existing knowledge workflows, it should have good support for the data structure that notes come in: text documents.
A defining characteristic of elements in a text file is their order in the document. We want Softspace’s layout system to express that ordinality. At the same time, we don’t want to give up all the benefits of spatiality by simply rendering a 2D document floating in front of you.
We need to invent a layout system that is strictly ordinal, yet meaningfully spatial.
Ordinality
The biggest advantage of ordinality is portability. Imagine an email you draft with speech-to-text, then finish and send on your laptop, before it’s read on someone’s phone. Its fixed sequence of information can adapt to very different contexts without losing much, if any, of its meaning.
The ordinality of HTML allows for websites that respond consistently to different screen sizes. The ordinality of natural language lets us turn books into audiobooks and back. (The non-ordinality of spreadsheets makes them a huge pain to work with on a phone.)
There’s another, deeper benefit to ordinality. Ordinal content maps directly onto the temporal structure of human experience. That gives its authors a high degree of control over when ideas are encountered, and how they build on each other.
On the other hand, ordinal formats struggle as working environments for generating knowledge, because at larger scales, knowledge is not ordinal (it’s nested and networked).
As internet pioneer Ted Nelson puts it:
With all due respect to Ted: I actually think that writers do important and difficult work by charting navigable paths through that “tapestry of interconnection”. But he’s also right. We lose something crucial if our tools and methods obscure the true shape of that tapestry.
Spatiality
Softspace uses the emerging medium of spatial computing to let you work with that tapestry outside the limits of small, flat screens. We also believe that spatial layouts can unlock your innate superpowers of spatial perception, reasoning, and memory.
A maximally spatial layout system would let you place things freely in a 2D or 3D canvas. There might be fancy mechanics like collision detection or snapping, but ultimately, every element’s position would be determined by an (x, y, z) coordinate. This is how most whiteboarding apps, like Miro, work.
Within this Cartesian playground, you get the full benefits of spatiality. You can construct richly meaningful configurations of information that express a lot of nuance and complexity. If you work in one for a while, you can really recall and explore these layouts in your mind’s eye afterward.
The problem with freely spatial layouts is that you can’t reliably put their elements into a coherent order. And without this ordinality, spatial layouts are hard to port to less-spatial contexts.
For example, a lot of meaning is lost when you export a Miro board to a linear document. Also, the process uses an “across-then-down” heuristic that creates unpleasant surprises in content order.
When you add a third dimension, the problem gets much worse. 3D Cartesian layouts, like those in previous versions of Softspace, are essentially impossible to export cleanly.
Ordinospatial Layouts
So ordinal data format are high portable, but obscure how information ties together. Spatial layouts reveal those interconnections, but aren’t very portable. Could we have the best of both worlds?
This prototype implements an experimental layout system that enforces a strict order to elements, but lets you position them relatively freely all three spatial dimensions.
Content (which are text blocks only, for now) flows down columns; columns spread horizontally across a plane; and planes stack front-to-back.
This system lets you move things along the x-, y-, and z-axes, but always results in a spatial configuration that is unambiguously interpretable—both by machines and people—as a definite sequence of elements.
There’s some “give” in the positioning of elements in the x- and z-axes, which lets you place things closer together or further apart. Physical distance is an important way of expressing relations between ideas in spatial layouts.
And of course: as you build up the workspace in Prototype03, it automatically writes your work to a markdown file. You can use SideQuest to retrieve it at: sdcard / Android / data / com.softspace.ar.prototype03 / Prototype03-Text.md.
Conclusions
The criterion of success for this prototype was to develop a 3D layout system that was 1) strictly ordinal, yet 2) meaningfully spatial.
✅ Prototype03’s layout system is strictly ordinal. This allows it to write out a markdown file representing any possible workspace you create in it.
❓ Spatiality is a spectrum. The layout system is less spatial than a purely Cartesian one, but it still allows for relatively free placement of elements. Does this mediated spatiality convey enough of the benefits of the medium to justify working in a headset? I would say yes, but ultimately, only users can judge if this criterion has been met.
Future Ideas
Mapping breaks in the x, y, and z axes to H3, H2, and H1 headings in markdown.
Prototype03 only writes out a very simple markdown file, with paragraph blocks only. Future builds would need to support all the major block types.
Instead of pointing to local headset storage, point to a Dropbox folder.
Image file support
In-headset browser support
Tag / wikilink support
UX Improvements
Grip interactions are disabled when a hand is not within a certain zone of activation in front of your head, to prevent accidentally moving the workspace while typing. Thanks to Brian Eppert for this suggestion.
Fatigue from having to hold your hand in the air to interact with elements is real. I'm working on ways to mitigate this, both by reducing the number of interactions that require raising your hands (e.g. making text boxes navigable using the arrow keys), and increasing the speed and ease of hand interactions (e.g. making positioning of objects snappier and more responsive). Thanks to Andy Matuschak for pointing out this issue.
That’s a wrap. As always—thanks so much for reading, and please don’t hesitate to reach out with your thoughts, comments, and questions!
Until next time,
Yiliu
Looks amazing! When you're ready to share widely, let me know, I'll mention this for my readers. Keep up the great work!