29:26Got it.
29:27So given that Kinopio is not just like a
more traditional web app where you have
29:33tons of forms and like pages and routes,
et cetera, but it's a spatial canvas.
29:39This seems to be a more common
way now, how software is laid out.
29:44I've never built a canvas-based
app myself, but I'm very interested
29:48in learning more about it.
29:49So I'm curious whether you can
share some of your learnings.
29:53about building a canvas-based tool?
29:56Fundamentally, there are two
main different approaches
29:59when you build a canvas.
30:00So the first approach is you can
build it sort of like a website where
30:05each card is a dom node and you know,
absolutely positioned and you just
30:09kind of move it around with transforms.
30:11Um, the upside of that is it's really fast
and you know, like it'll work everywhere.
30:15The downside is there's a lot of like
interactions you do on a canvas, like
30:20pinch zooming and things like that.
30:22are not really handled well by all
browsers out of the gate, especially
30:27iOS Safari and, and Android Chrome,
for that matter, does a really
30:30poor job in some of these things.
30:32So for example, like I have a UI on my, on
my page and I, but I also have my canvas.
30:37If I pinch zoom the phone, I'm zooming in
the whole thing, including the UI, which
30:41makes the buttons look wild and crazy.
30:43And I don't want to do that.
30:44I can also like add like special touch
handlers to only zoom in that part of
30:49the canvas, but it's really hard to do
it in a way Where it doesn't just crash
30:53the browser because the transformed
texture of the page gets gets too big.
30:57So a Kinopio document can be like tens of
thousands of pixels wide or deep, right?
31:01So it's really hard to design around that.
31:03The second approach, which is by far the
more popular approach, is to recreate
31:08the whole canvas in the HTML canvas
element or some some similar type of
31:14rendering that you control and you
have really fine grained thing over.
31:18And so the upside of that is you have
full control over pinching and zooming.
31:22You can tell things to, to leave space.
31:25The viewport like intersection observer
equivalent, um, is a lot more reliable.
31:31So browsers don't always report their
positions, especially on mobile,
31:34accurately to web apps, as I've found.
31:37And so, being able to control
every part of the rendering
31:40process means you control it.
31:42You can make things appear
precisely how you want to.
31:46The downside is you have to, you lose all
the things you get for free on the web.
31:50You lose accessibility, you lose speed.
31:53You lose like, you know, like special
touch handling and other things like that.
31:56And you have to build it in yourself,
or more likely you don't really.
32:00And you kind of have a canvas
that works great on like desktop
32:03computers, but not so much elsewhere.
32:05And so because of that, Kinopio uses
the first approach, which is fast, but.
32:10It requires a lot of work to not be janky
and I, I do my best, I'll say that much.
32:15Any particular patterns that
you figured out for that first
32:19approach that you laid out?
32:20Yeah, so there's a thing that
I do called counterscaling.
32:24So essentially when you pinch the
screen on mobile, the whole page
32:28will zoom up, including the UI.
32:30But what I do with the UI is I
tell it to scale itself down.
32:34So if let's say you pinched in and
the page is now 2x, I tell the UI.
32:38Hey, it looks like the page is 2x.
32:40Maybe zoom yourself down,
scale yourself down to 0.
32:425x.
32:43And so it kind of like net makes
the UI look the correct size.
32:47So I'm basically like making the UI
smaller so that it appears regular
32:51size while you're doing this.
32:52What I found though is that
browsers, mobile browsers don't send
32:56resized data and pinch zoom data
as quickly as you interact with it.
32:59A lot of events happen in the
browser that just aren't reported.
33:02To the browser, because they
don't want to like, you know, blow
33:05up the JavaScript or whatever.
33:06There's like little tricks that
you do to kind of work around that.
33:09So like, I will fade in the UI during
like touch events so that you don't
33:13see things like jitter around as
they're, they're struggling to resize.
33:17So yeah, I've learned a lot
about how mobile browsers operate
33:20differently than desktop ones.
33:21And, uh, it's, it's a lot.
33:23Would you still choose approach
number one, as opposed to
33:27the canvas-based approach?
33:28I think I would, because I've tried like.
33:30a couple times to, to redo
Kanopio with different approaches.
33:34The other big downside of the
canvas-based approach is it
33:37also makes iteration way harder.
33:40Like adding new features when you have to
also render them or like, you know, like,
33:44like at a lower level and figure out how
to just work around that sort of stuff.
33:48And you don't have like, you
know, the normal flex box type
33:52positioning flow and things like that.
33:53It's a, it's a lot more tricky.
33:55And so I think it's kind of
like I've chosen the net best
33:58approach, at least for Kinopio.
34:00And the one that I've kind of embraced
because one of the other advantages
34:03of doing a full canvas rendering
is you can scroll in 360 degrees.
34:08And with Kinopio, you can only scroll
down and to the right as much as you want.
34:11But because of that constraint,
You always know where the
34:14beginning of the document is.
34:16It's always in the top left for the
most part, as opposed to other canvas
34:20tools where it can be like in the
center of the page, which might not
34:23be visible depending on the size of
your device and you've got to kind of
34:26scroll around to figure things out.
34:28That's super cool.
34:29In regards to the real time collaboration,
so you can share boards with other people
34:35and like work on them at the same time.
34:37Are you going even further to have like
a sense of presence across collaborators?
34:42You can do see other people's
cursors or how did you think about
34:46designing those interactions?
34:47So, Kinopio has this kind of unique
feature called Paint Select or Magic
34:51Paint and basically it lets you select
multiple cards to do bulk actions, like
34:55move them all together, but it lets
you paint over them to select them.
34:59So instead of drawing a box, you can
kind of like make more kind of unique
35:04selections by painting over and it was
kind of also a way to like bridge this
35:07gap between the right side of your
brain and the left side of your brain.
35:11And so the paint stroke kind of disappears
as you're paint selecting, there's like
35:14a exponential decay function there.
35:17And so I also I show those same paint
strokes for other people looking at
35:21your space or interacting with it.
35:23So as you're moving around, you see
their paint strokes kind of moving
35:26around and fading in the background.
35:28You also see their user icons
kind of floating around.
35:30So it kind of creates more than
just like the cursor plus user icon.
35:34It kind of creates more of like a feeling
of like, Oh, like these people are
35:38like, or someone's really interacting
with the space or maybe somebody is
35:40just noodling around in the space.
35:41I can just see their paint strokes and
they're trying to spell out words or
35:44draw things before the paint fades away.
35:47That is super cool.
35:48Yeah.
35:48This is, I think goes back to the design
principles that did you laid out initially
35:53and how you're not directly imitating
paper, but you figure out like what is
35:58the best mechanism here for, for this
medium and the goals you're going for.
36:02So going back to the canvas, I recently
read that I think Obsidian has launched
36:08an initiative called JSON Canvas.
36:11Um, if I remember correctly, and I
think there, do you have some plans
36:15to, to also like integrate with that?
36:18Funny you should mention, I literally,
um, just yesterday launched, uh, Kinopio
36:22support for full import and export, uh,
you know, um, exporting with that, with
36:27that file format, the canvas file format.
36:30So they launched it yesterday.
36:31I really quickly added it yesterday.
36:33So can you share a little bit more about
why that would be useful for, for people?
36:37Sure.
36:38Yeah.
36:38So for the longest time there hasn't been
like a great interop document format.
36:43So of course, each different app has
its own JSON representation of nodes
36:48or cards and edges or connection lines.
36:51But they're not, they're all app specific.
36:53So, you know, it's kind of tough
if you're trying to, you know,
36:55Migrate from one tool or another
or test out one tool or another.
36:59And so the closest equivalent we
had before, just for a little bit of
37:01historical context was OPML, which is also
like kind of a, an off requested feature,
37:06an OPML was kind of made for outliners.
37:09So like collapsible nested tree and you
know, structures I can indent now dent.
37:14The problem with OPML is it doesn't handle
what happens if you have multiple trees
37:19that aren't connected to each other,
or you have multiple trees that might
37:22have child node that they both share.
37:24You know, like you can, you can
create structures that are way
37:26more complex than OPML can capture.
37:29And so I just couldn't figure
out how to do it with OPML.
37:32So I followed the spec for a while and,
and when Capano and Obsidian launched
37:36it, I, I was already kind of in a rework
of the importing and exporting system.
37:41So it's a very simple spec.
37:42It doesn't cover every feature
that either app has, I'm sure.
37:46But the advantage of that is
it's also very easy to implement.
37:48So, you know, it's different.
37:49Transforming from one JSON
object to another, essentially.
37:53Got it.
37:53And I suppose that while the spec
is like very open ended in some
37:58way, maybe that converges to,
towards some more details over time.
38:03And the idea would be that I can
bring some data from one tool,
38:07for example, Obsidian into Kinopio
or the other way around or future
38:12tools that might, might exist.
38:14Yeah.
38:15It's, but there are wrinkles because.
38:17Of how different these tools
are, or at least how different
38:19Kinopio is from all of them.
38:21So there are assumptions and I've,
I've left like feedback about these
38:24things in, in GitHub issues for the
spec, but there are like assumptions
38:28that they make like, um, the node
has to define like how wide it is
38:32explicitly, or what type of node is it?
38:35Is it a file node?
38:36Is there a, you know, like
a website node or whatever.
38:39So problems with both of these is that
like the width, Of one app's node for the
38:45same amount of text might be different for
another app because they're all rendering
38:49it differently in Kinopio's case.
38:51There's also this.
38:52It's also mobile compatible.
38:53So there's a bit extra of padding
and sizing for touch friendliness.
38:57So those numbers don't really translate.
39:00And so it's kind of weird that
they're required in the spec
39:02as opposed to being optional.
39:03Second issue was the node type.
39:05So I can mention a long time ago.
39:07Or earlier in this chat, you know, a
lot of apps make you choose, like, Hey,
39:10are you going to insert a web page?
39:11Are you going to sort of whatever?
39:12On Kinopio, they're all the same type.
39:14They're all just, there's only one
type of card, a text card, and whatever
39:17type of content you put in that,
whether it's a link or a website or
39:20whatever, is what that card transforms
into using the power of regexes.
39:24And so that part of the whole
spec is basically, like, not
39:28necessarily relevant to Kinopio.
39:30Didn't really connect because
It's making the assumption
39:33that that isn't really shared.
39:34So in some ways I think it's too
specific in some ways it's, and I'm sure
39:38a lot and a lot of people's complaint
is that it's not specific enough.
39:42So it's weird because like
it's really hard to make a
39:45spec for wildly different apps.
39:47But I think currently I'm actually
pretty happy with the spec because it
39:50captures like the most important part,
which is the representation of cards
39:55and nodes in a way that doesn't require
them to have this strict tree structure.
40:00Yeah.
40:00That makes a lot of sense.
40:01And yeah, I mean, the devil's always in
the detail and you built Kinopio really
40:06to enable the sort of interactions and
details that you uniquely thought about.
40:11And you might not have that entropy
available in Obsidian and vice versa.
40:17And then it's also, how do
you preserve that intent and
40:20that information across apps?
40:22That's, that's a very tricky problem,
but I, I applaud the initiative to take
40:27some first steps there to at least bring
some data across one app to another.
40:32So that sort of cross app collaboration,
I think that is certainly a goal that I'm
40:37rooting for, but that we still have like
many chapters still ahead of us there.
40:42So the way I've built Kenopio, it's super
impressive, both in terms of like how
40:48the product feels and works and looks,
but also like, given that you've been
40:54on this journey for, for such a long
time, uh, by, by yourself, I'm curious
40:59which corners you might've already cut
along the way that you regret looking
41:04back where you wish you would have like
spent up front or vice versa where you
41:10thought something might be a problem
and it never actually became a problem.
41:13Oh, wow.
41:14Yeah.
41:14There's, there's
definitely a lot of those.
41:16So like, there's like a lot of code kind
of like with the syncing engine, uh, and
41:21the way I merge, um, different changes
to make smaller change sets that get
41:25sent to the server, but I feel like.
41:27are a little hard to work around now
in just in terms of like just how gross
41:32that code is or how like abstractive
and I wish I did a better job writing
41:36it like it's not it's not something
that's painted me into too harsh a
41:39corner it's just something I'm like
oh I'm scared to touch this part of
41:42the app but one day I'll just you know
buckle down and and really get to it.
41:47So I actually think the app is in a
pretty good place just because um I
41:52guess the other thing is like I prefer
to like write code that's like, not
41:57necessarily for me to understand now,
but for me to understand in like two
42:00months or three months or five years from
now, where I forget what the S variable
42:05or the A variable is referring to.
42:07So I, I, I was like kind of influenced
by, by some short time, like,
42:13Coding for Apple platforms where
they're very verbose with variables.
42:16I kind of took some of that
Coco convention into my own
42:20kind of code writing style.
42:22Yeah, I think that's very wise.
42:23Uh, I've made the same observation
about myself since in reality, even
42:28if you just work by yourself, you
still collaborate with yourself.
42:32Over time more instances of you over
time the future you, the past you.
42:34who do not the same
context right in your brain
42:42So just like leaving notes for yourself
of like, why did you do a certain
42:48thing has, I, I've had to learn that
lesson painfully and I'm treating my
42:53future self better or I'm trying to,
so, uh, it looks like you've learned
42:59quite a couple of similar lessons, but
43:01Yeah, I think we all kind of learn that,
but we all have to, like, burn ourselves
43:04a little bit before we truly learn it.
43:06Because it's not, like, an
intuitive thing to know.
43:09And it is, like, in an industry that's
always trying to, like, move fast and
43:12ship fast and whatever, it's a little,
it's mildly antithetical to that.
43:16But it is one of those things where,
take a little extra time to write