cover image from https://www.moshimoshi-nippon.jp/219615/スカイランタン祭り-sky-lantan-festival-東京-神戸-kobe-tokyo-img_184265_1-min-2

My earlier work automated the fusion of technical drawing in-situ within models. Since 2012 this has been implemented in 9 different softwares but remains largely siloed in each, and limited by flaws (mine) in the original design.

My current work proposes to address those flaws (of intelligibility and portability) through better design, of an evolution in technical drawing’s form of expression, given it’s instantiation in-situ within models both mental and digital. The proposed evolution in form is both directly supportive of the existing traditional form of technical drawing, honoring it with amplification of the force of its effect, and at the same time, a meaningful evolution in its form, again for greater effect of drawing’s purpose (making things clear in very complex environments).

The specification for this dual function, carried out at the same time in one expression, is here on my 2 page website: https://tangerinefocus.com The specification is completely supportive, of all existing forms of drawing and modeling processes and applications, but bumps all of them up at the same time to a higher level of expression, in a completely non-disruptive non-destructive way.

This is not a proposal for a product.

Instead it’s a redefinition of what drawing is. It’s a proposed evolution in drawing’s form of expression, IN models, designed not to be siloed in any product but instead open so it can be developed in all modeling softwares, all kinds, and is portable between them.

It’s a proposal to transform visual engagement with modeled environments for everyone, but especially motivated for people engaging models for technical purposes, where engagement has to produce beyond-superficial understanding of very complex models, both during model creation and during downstream model use.

It also establishes an accessible step to stand on and engage from, for humans in the loop in AI-generated modeling and other forms of computational model generation:

Visual Engagement with Modeled Worlds

REAL AND MODELED REALITY, MENTAL MODELS, VIEWS, TWEENERS, MEMORY, WOBBLE, AND MY KITCHEN SINK

The heading above is a word jumble, word salad, sure, but with purpose. In this post I want to address more about the reason things are the way they are, why, we’re still dealing with drawings, after 3 decades plus of digital modeling.

It’s not because of any number of hypotheses given over the years: defects in contract law, business culture, faults in software training, and so on. We’re still dealing with drawing for reasons that are fundamental. This means the reasons are true all the way down, to the lowest levels at which you can think about a situation. True, not part of the way down. All the way down.

To illustrate, I show pictures of my kitchen sink.

Take a look at the following 7 images:

I walk out the door and outside.

That’s Rasmus.

And those are views within a spatial reality familiar to me. I live here. You’ve never been here. But there’s enough information in those 7 views for you to imagine a lot of what’s missing between the views.

Through some kind of cognitive action on your part, you supply a bunch of tweeners.

And this happens, it should go without saying, regardless of the type of spatial environment. It’s the same mental behavior either way, in any kind of digital modeled environment and in the real world.

We’re beings. We be (exist) in a reality. That means, largely, we’re exercising (some kind of, our) perceptual equipment, in the reality, to make sense of it. [cogito ergo sum]

It’s worth noting that perceptual equipment operates in a particular way that always involves intermittent attention, and a set of dynamics related to intermittent attention.

Let’s talk about it.

Attention comes and goes. It’s not continuous. Something called focus draws attention somewhere, sometimes. Not everywhere, and not all the time.

In a way maybe it’s useful to think of attention on a sine curve. Attention rides a wave, let’s say, with views that hold in the memory (like the pictures above starting at my sink and ending at Rasmus) at the peaks of the sine wave. Elsewhere on the curve, attention operates at a lower level, or not at all:

It’s how our minds work. They need work and rest, work and rest, work and rest, in a cycle. Perception is engaged through a wave of attention. Attention is not continuous. It wavers on and off.

We can see some things related to this.

Please go back again to the pictures. From those 7 views, which in this experiment are the absolute limit of your attention, you recreate the spatial environment yourself. You do this automatically. You can’t NOT do it. It just happens.

Whether you want it or not, you have in your mind, now, a mental model of my kitchen and back yard.

How did that happen?

What exactly is happening in the arrow in this diagram?:

It’s mostly, I think, for us to just accept the mystery. I mean, we can only go so far in explaining it. It’s beyond the grasp of neuroscience.

But there is some technique in play, that we can see, and think about.

MENTAL MODEL

As sentient beings, we build mental models of sensory input. Without a mental model always in formation, nothing means anything; there’s no ground within which interpretation can grow.

Digital models, just like the real world, are un-interpretable (meaningless), until perceptual equipment engages them adequately enough to produce their (fuzzy and incomplete) approximation in a mental model.

MEMORY

Let’s prove the obvious.

I insert another image:

You know where that is.

You’ve never been here before. You saw a few views from this place. You built a mental model from those views. And you know where this is in your mental model.

That’s remarkable.

We have memory. We compare our vantage now, to our vantage a split second ago, and so on including lifetime memory of sight of similar things. From this we build mental models. And into those models we instantiate new views.

Note the role of memory in model creation from a small set of images using NeRF Neural Radiance Field methods, starting at 23:00 minutes in this presentation, an interesting parallel to natural cognitive processing:

Going back again to natural human perception, with views and memory in play in the formation of usable mental models, another aspect is that we move around what we’re looking at. A shift to the side a bit, and back sometimes. Sometimes a tilt. In fact we’re in continuous motion. That’s the thing, and

We move in a sort of arc, around whatever has our attention. I’m referring to this as:

WOBBLE

Here’s a clip where wobble’s absence in a digital app, is made a joke in Mission Impossible:

Tom Cruise can stick to motorcycles and not return to sad jokes about poor engagement with, and misinterpretation of, digital models

Wobble is an essential part of visualization. Everyone does this, all the time. You’re doing it in this experiement. You’ve seen my 2 cats, my sink, a few images. You built a mental model. It was involuntary. Like a heartbeat, autonomic.

You imagine yourself in motion, slightly, side to side around the cat, around the table, around the door jamb, at the sink. You also wobble to and fro from view to view, image to image, at the peaks of your attention sine curve.

To put it in pictures,

the yellow cloud is your mental model, in formation through the views (the black dots) that hold in the memory at the peaks of your attention curve:

And by the way, why those views? Why are those views held in the memory? Why are those views in particular at the peaks of our attention curve? How do we select, and know to select, where and when to pay attention?

It’s not easily answered, if at all. Another aspect of the mystery of thought and mind. So while again it’s a case of “accept the mystery“, there are functional things we can observe about it, like the relation between the views in which we invest our attention, and the cumulative support those views lend for some task that matters to us.

In the case of just walking from the kitchen out the back door, the experiment above, the task was simple. The views invested in, supported that task with barely more than superficial attention required. In technical fields, beyond-superficial attention is needed, so the amplitude of the attention wave is higher and the frequency greater.

In fact yes, in the cognitive processes involved in the formation of your mental model, frequency matters, and memory and wobble are always in play

You wobble among the peak views. See the curve in the image below connecting the view dots. The curve represents your memory of, and your wobbling between, the views:

But as discussed above in the kitchen experiment, you wobble at each view too:

We can also observe, and this can be proven through experiment, that there is a sweet spot for efficient productive development of the mental model, in terms of the frequency of views that hold in the memory, in which attention is invested. The frequency of the attention curve of course is the wavelength, or cycle of the attention sine curve:

The frequency can’t be too low or too high, the number of views neither too few nor too many:

Mental model formation fails at both extremes.

Discussion, of the mechanism of perception, continues here:

AUTONOMIC AND SOMATIC VISUAL PROCESSING

I’ll come to the end of the post

REAL AND MODELED REALITY, MENTAL MODELS, VIEWS, TWEENERS, MEMORY, WOBBLE, AND MY KITCHEN SINK, well not my kitchen sink, but all the rest, are included in the proposed evolution in the form of technical drawing’s expression in digital models.

The proposal is in one page here, in outline form, with links to long form description too. There is also a demo video, and more commentary:

Visual Engagement with Modeled Worlds

I was asked if the proposal is an API. My attempt at an answer here:

Is TGN an API?

Leave a comment or contact me to discuss:

Contact 

• LinkedIn: https://www.linkedin.com/in/robsnyder3333/
• email: 
  robsnyder@tangerinefocus.com