All language subtitles for 6-startup-vs-runtime-performance

I think as an industry we have
become little immune to this.

Where everybody keeps claiming how
amazingly performant they are, but

I think they're missing the wrong thing.

If you look at these benchmarks, and
I'm sure you've seen this, right?

They do things like, how long does it
take to render a thousand rows, right?

Sure, it's a use case.

But that's not the problem for
99% of the websites.

None of the print sites are not slow
because they're trying to render 1000

rows, right?

And this benchmarks lie for
several reasons.

First of all, well, not that they lie.

They don't lie, I mean,
they show the truth here.

They're just measuring the thing that
doesn't really necessarily matter.

So the first problem I have is that
all of these benchmarks assume that

JavaScript is already downloaded, right?

Not the case for most sites.

The second thing they assume is that
the jitter has already been warmed up.

So what happens is, they run this
benchmark over and over and over again,

and the first few runs get thrown away
because they're too unpredictable.

The reason that they're unpredictable is
because the jitter is not ready, right?

The jitter hasn't collected sufficient
amount of data about the runtime of

the application to do this.

Now, when you run a code in tight loop,
Jitter does all kinds of crazy things

that are not a representative of
what actually happens in production.

Everything from the fact that a tight loop

will have really good caching in
terms of L2 cache inside the CPU.

It will do really good caching
in the property lookup in

the VM which is not true in
the actual application code, etc.

So there's a whole bunch of reasons why
warmed up Jit is actually incorrect of

the actual performance, and specifically,

it's incorrect because when you navigate
to a page on a fresh page, no jitter.

You are running in interpretive mode and

everything is being
running interpretively.

>> Sorry, when you say jitter,
that has different baggage for

me based on my background.

When I hear jitter, I think of
fluctuations in execution or latency time,

but I think what you're really referring
to is the just-in-time compiler.

>> Sort of yes, here jitter means
just-in-time compiler, right?

So,
>> I just wanted to clarify.

>> Yes, it's good clarification.

So, JavaScript is a interpreted language,
right?

That can be compiled down to assembly.

And that's the job of the Jitter,
to compile it down.

But it turns out it's actually
extremely difficult to compile it down.

And so, what happens in practice is that
the Jitter collects information about what

it thinks your JavaScript
is actually doing.

And based on that,
it takes a whole bunch of shortcuts.

And then if you break one of these
assumptions that the Jitter makes,

then the Jitter does
this thing called deopt.

In other words, it says, yeah, I generated
all this code, I thought it was correct,

but it turns out it's not,
I'm gonna throw it away.

And so, all of these benchmarks that
you have here are my point is that

they're kind of unfair.

None of them are unfair, they're not
measuring the thing that you want.

The problem of our industry is not that
the application is slow once it's up and

running, although there are some
applications that have this problem.

The problem with the industry
is just getting the application

to run is the problem.

Yeah, we talked about the Jitter,

we talked about this totally focuses on
the dumb update and it ignores the fact

that there is this thing called time
to interactive and total blocking time.

And so, my point with these benchmarks
is that while they're important,

they're not actually going after,

an improvement in this benchmark here
will probably have no measurable

improvement in your startup
performance of the application.

So, people will oftentimes be like,
I should use framework x cuz it's twice as

fast as the other thing, on my
application is gonna be twice as fast.

You run it, you rebuild your
app in the new technology and

you realize the application is
just as slow as it was before.

There's no difference in
the thing that we care about.

So, instead we did
a different benchmark here.

So first,
we have this tool called Mitosis.

Mitosis allows you to write kind of
pseudo code of an application and

write it once, and Mitosis will
generate that code in idiomatic format,

in angular, in react,
in whatever technology you want, right?

And this allows you to really have
apples to apples to comparisons, right?

Because a lot of the people that
wrote the movie app complained, well,

there's actually two different
versions of the movie app.

Ours does log in,
yours doesn't do log in, or whatever.

It's all fair points and so
with Mitosis you can actually

literally build the same exact
app across the technologies.

And the thing to kind of compare
here is time to interactive, right?

This is the blue line and
the total amount of JavaScript.

And you can see is that as the total
amount of JavaScript goes up,

the blue line goes up as well, right?

And again, it totally makes sense because
what we're talking about here is,

the way hydration works is it
eagerly executes your application.

The more application you have,
the longer it takes, right?

So, the more JavaScript you ship,
the longer the startup is gonna be.

It's as simple as that.

Figure out a way not to ship JavaScript,
your startup will be fine.