0% found this document useful (0 votes)
16K views201 pages

Data-Oriented Design and C++ - Mike Acton - CppCon 2014

The document discusses data-oriented design principles and how they apply to game engine development at Insomniac Games. It emphasizes understanding the data and hardware in order to optimize performance. Examples are given of how structuring data to better fit cache lines can yield significant speedups of 10x or more. Data-oriented design is presented as a response to object-oriented programming patterns that fail to prioritize data and performance.

Uploaded by

c0der
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
16K views201 pages

Data-Oriented Design and C++ - Mike Acton - CppCon 2014

The document discusses data-oriented design principles and how they apply to game engine development at Insomniac Games. It emphasizes understanding the data and hardware in order to optimize performance. Examples are given of how structuring data to better fit cache lines can yield significant speedups of 10x or more. Data-oriented design is presented as a response to object-oriented programming patterns that fail to prioritize data and performance.

Uploaded by

c0der
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 201

Data-Oriented Design and

C++
Mike Acton
Engine Director, Insomniac Games
@mike_acton
A bit of background…
What does an “Engine” team do?
Runtime systems
e.g.
• Rendering
• Animation and gestures
• Streaming
• Cinematics
• VFX
• Post-FX
• Navigation
• Localization
• …many, many more!
Development tools
e.g.
• Level creation
• Lighting
• Material editing
• VFX creation
• Animation/state machine editing
• Visual scripting
• Scene painting
• Cinematics creation
• …many, many more!
What’s important to us?
What’s important to us?
• Hard deadlines
What’s important to us?
• Hard deadlines
• Soft realtime performance requirements (Soft=33ms)
What’s important to us?
• Hard deadlines
• Soft realtime performance requirements (Soft=33ms)
• Usability
What’s important to us?
• Hard deadlines
• Soft realtime performance requirements (Soft=33ms)
• Usability
• Performance
What’s important to us?
• Hard deadlines
• Soft realtime performance requirements (Soft=33ms)
• Usability
• Performance
• Maintenance
What’s important to us?
• Hard deadlines
• Soft realtime performance requirements (Soft=33ms)
• Usability
• Performance
• Maintenance
• Debugability
What languages do we use…?
What languages do we use…?
•C
• C++
• Asm
• Perl
• Javascript
• C#
What languages do we use…?
•C
• C++  ~70%
• Asm
• Perl
• Javascript
• C#
What languages do we use…?
•C
• C++  ~70%
• Asm
• Perl
• Javascript
• C#
• Pixel shaders, vertex shaders, geometry shaders, compute shaders, …
We don’t make games for Mars but…
How are games like the Mars rovers?
How are games like the Mars rovers?
• Exceptions
How are games like the Mars rovers?
• Exceptions
• Templates
How are games like the Mars rovers?
• Exceptions
• Templates
• Iostream
How are games like the Mars rovers?
• Exceptions
• Templates
• Iostream
• Multiple inheritance
How are games like the Mars rovers?
• Exceptions
• Templates
• Iostream
• Multiple inheritance
• Operator overloading
How are games like the Mars rovers?
• Exceptions
• Templates
• Iostream
• Multiple inheritance
• Operator overloading
• RTTI
How are games like the Mars rovers?
• No STL
How are games like the Mars rovers?
• No STL
• Custom allocators (lots)
How are games like the Mars rovers?
• No STL
• Custom allocators (lots)
• Custom debugging tools
Is data-oriented even a thing…?
Data-Oriented Design Principles

The purpose of all programs,


and all parts of those
programs, is to transform
data from one form to
another.
Data-Oriented Design Principles

If you don’t understand the


data you don’t understand
the problem.
Data-Oriented Design Principles

Conversely, understand the


problem by understanding
the data.
Data-Oriented Design Principles

Different problems require


different solutions.
Data-Oriented Design Principles

If you have different data,


you have a different
problem.
Data-Oriented Design Principles

If you don’t understand the


cost of solving the problem,
you don’t understand the
problem.
Data-Oriented Design Principles

If you don’t understand the


hardware, you can’t reason
about the cost of solving the
problem.
Data-Oriented Design Principles

Everything is a data
problem. Including usability,
maintenance, debug-ability,
etc. Everything.
Data-Oriented Design Principles

Solving problems you


probably don’t have creates
more problems you
definitely do.
Data-Oriented Design Principles

Latency and throughput are


only the same in sequential
systems.
Data-Oriented Design Principles

Latency and throughput are


only the same in sequential
systems.
Data-Oriented Design Principles

Rule of thumb: Where there


is one, there are many. Try
looking on the time axis.
Data-Oriented Design Principles

Rule of thumb: The more


context you have, the better
you can make the solution.
Don’t throw away data you
need.
Data-Oriented Design Principles

Rule of thumb: NUMA


extends to I/O and pre-built
data all the way back
through time to original
source creation.
Data-Oriented Design Principles

Software does not run in a


magic fairy aether powered
by the fevered dreams of CS
PhDs.
Is data-oriented even a thing…?

…certainly not new ideas.

…more of a reminder of first principles.


…but it is a response to the culture of
C++
…but it is a response to the culture of
C++

…and The Three Big Lies it has engendered


i.e. Programmer’s job is NOT to write code;
Programmer’s job is to solve (data transformation) problems
A simple example…
Solve for the most common case first,
Not the most generic.
“Can’t the compiler do it?”
A little review…
(AMD Piledriver)

http://www.agner.org/optimize/instruction_tables.pdf
(AMD Piledriver)

http://www.agner.org/optimize/instruction_tables.pdf
http://research.scee.net/files/presentations/gcapaustralia09/Pitfalls_of_Object_Oriented_Programming_GCAP_09.pdf
http://www.gameenginebook.com/SINFO.pdf
The Battle of North Bridge

L1

L2

RAM
L2 cache misses/frame
(Most significant component)
Not even including shared memory modes…
GPU-visible GPU Coherent
Name Cached
Heap-cacheable No Yes No
Heap-write-combined No No No
Physical-uncached ? No No
GPU-write-combined Yes No No
GPU-write-combined-read-only Yes No No
GPU-cacheable Yes Yes Yes
GPU-cacheable-noncoherent-RO Yes Yes No
Command-write-combined No No No
Command-cacheable No Yes Yes
http://deplinenoise.wordpress.com/2013/12/28/optimizable-code/
2 x 32bit read; same cache line = ~200
Float mul, add = ~10
Let’s assume callq is replaced. Sqrt = ~30
Mul back to same addr; in L1; = ~3
Read+add from new line
= ~200
Time spent waiting for L2 vs. actual work

~10:1
Time spent waiting for L2 vs. actual work

~10:1

This is the compiler’s space.


Time spent waiting for L2 vs. actual work

~10:1

This is the compiler’s space.


Compiler cannot solve the most
significant problems.
Today’s subject:
The 90% of problem space we
need to solve that the compiler
cannot.

(And how we can help it with the 10% that it can.)


Simple, obvious things to look for
+ Back of the envelope calculations
= Substantial wins
L2 cache misses/frame
(Don’t waste them!)
Waste 56 bytes / 64 bytes
Waste 60 bytes / 64 bytes
90% waste!
Alternatively,
Only 10% capacity used*

* Not the same as “used well”, but we’ll start here.


12 bytes x count(5) = 72
12 bytes x count(5) = 72

4 bytes x count(5) = 20
12 bytes x count(32) = 384 = 64 x 6

4 bytes x count(32) = 128 = 64 x 2


12 bytes x count(32) = 384 = 64 x 6

4 bytes x count(32) = 128 = 64 x 2

(6/32) = ~5.33 loop/cache line


12 bytes x count(32) = 384 = 64 x 6

4 bytes x count(32) = 128 = 64 x 2

(6/32) = ~5.33 loop/cache line


Sqrt + math = ~40 x 5.33 = 213.33 cycles/cache line
12 bytes x count(32) = 384 = 64 x 6

4 bytes x count(32) = 128 = 64 x 2

(6/32) = ~5.33 loop/cache line


Sqrt + math = ~40 x 5.33 = 213.33 cycles/cache line
+ streaming prefetch bonus
12 bytes x count(32) = 384 = 64 x 6

4 bytes x count(32) = 128 = 64 x 2

Using cache line to capacity* =


10x speedup (6/32) = ~5.33 loop/cache line
Sqrt + math = ~40 x 5.33 = 213.33 cycles/cache line
+ streaming prefetch bonus

* Used. Still not necessarily as


efficiently as possible
In addition…
1. Code is maintainable
2. Code is debugable
3. Can REASON about cost of change

(6/32) = ~5.33 loop/cache line


Sqrt + math = ~40 x 5.33 = 213.33 cycles/cache line
+ streaming prefetch bonus
In addition…
1. Code is maintainable
2. Code is debugable
3. Can REASON about cost of change

Ignoring inconvenient facts is not engineering;


It’s dogma.

(6/32) = ~5.33 loop/cache line


Sqrt + math = ~40 x 5.33 = 213.33 cycles/cache line
+ streaming prefetch bonus
bools in structs… (3) Extremely low information density
bools in structs… (3) Extremely low information density

How big is your cache line?


bools in structs… (3) Extremely low information density

How big is your cache line?

What’s the most commonly accessed data?

64b?
How is it used? What does it generate? (2) Bools and last-minute decision making
MSVC
MSVC

Re-read and re-test…

Increment and loop…


Why?

Re-read and re-test… Super-conservative aliasing rules…?


Member value might change?

Increment and loop…


What about something more aggressive…?
What about something more aggressive…?

Test once and return…


Okay, so what about…
…well at least it inlined it?
MSVC doesn’t fare any better…
(4) Ghost reads and writes

Don’t re-read member values or re-call functions when


you already have the data.
BAM!
:(
(4) Ghost reads and writes

Don’t re-read member values or re-call functions when


you already have the data.

Hoist all loop-invariant reads and branches. Even super-


obvious ones that should already be in registers.
:)
:)
A bit of unnecessary branching, but more-or-less equivalent.
(4) Ghost reads and writes

Don’t re-read member values or re-call functions when


you already have the data.

Hoist all loop-invariant reads and branches. Even super-


obvious ones that should already be in registers.

Applies to any member fields especially.


(Not particular to bools)
(3) Extremely low information density
(3) Extremely low information density

What is the information density for is_spawn


over time?
(3) Extremely low information density

What is the information density for is_spawn


over time?

The easy way.


Zip the output
10,000 frames
= 915 bytes
= (915*8)/10,000
= 0.732 bits/frame
Zip the output
10,000 frames
= 915 bytes
= (915*8)/10,000
= 0.732 bits/frame

Alternatively,
Calculate Shannon Entropy:
(3) Extremely low information density

What does that tell us?


(3) Extremely low information density

What does that tell us?


Figure (~2 L2 misses each frame ) x 10,000
If each cache line = 64b,
128b x 10,000 = 1,280,000 bytes
(3) Extremely low information density

What does that tell us?


Figure (~2 L2 misses each frame ) x 10,000
If each cache line = 64b,
128b x 10,000 = 1,280,000 bytes

If avg information content = 0.732bits/frame


X 10,000 = 7320 bits
/ 8 = 915 bytes
(3) Extremely low information density

What does that tell us?


Figure (~2 L2 misses each frame ) x 10,000
If each cache line = 64b,
128b x 10,000 = 1,280,000 bytes

If avg information content = 0.732bits/frame


X 10,000 = 7320 bits
/ 8 = 915 bytes

Percentage waste (Noise::Signal) =


(1,280,000-915)/1,280,000
What’re the alternatives?
(1) Per-frame…
(1) Per-frame… (decision table)

1 of 512 (8*64) bits used…


(1) Per-frame… (decision table)

1 of 512 (8*64) bits used…

(a) Make same decision x 512


(1) Per-frame… (decision table)

1 of 512 (8*64) bits used…

(a) Make same decision x 512

(b) Combine with other reads / xforms


(1) Per-frame… (decision table)

1 of 512 (8*64) bits used…

(a) Make same decision x 512

(b) Combine with other reads / xforms

Generally simplest.
- But things cannot exist in abstract bubble.
- Will require context.
(2) Over-frames…
(2) Over-frames…

i.e. Only read when needed


Arrays of command buffers for future
(2) Over-frames… frames…

i.e. Only read when needed

e.g.
Let’s review some code…
http://yosoygames.com.ar/wp/2013/11/on-mike-actons-review-of-ogrenode-cpp/
(1) Can’t re-arrange memory (much)

Limited by ABI

Can’t limit unused reads

Extra padding
(2) Bools and last-minute decision making
Are we done with the constructor?

(5) Over-generalization
Are we done with the constructor?

(5) Over-generalization

Complex constructors tend to imply that…


- Reads are unmanaged (one at a time…)
Are we done with the constructor?

(5) Over-generalization

Complex constructors tend to imply that…


- Reads are unmanaged (one at a time…)
- Unnecessary reads/writes in destructors
Are we done with the constructor?

(5) Over-generalization

Complex constructors tend to imply that…


- Reads are unmanaged (one at a time…)
- Unnecessary reads/writes in destructors
- Unmanaged icache (i.e. virtuals)
=> unmanaged reads/writes
Are we done with the constructor?

(5) Over-generalization

Complex constructors tend to imply that…


- Reads are unmanaged (one at a time…)
- Unnecessary reads/writes in destructors
- Unmanaged icache (i.e. virtuals)
=> unmanaged reads/writes
- Unnecessarily complex state machines (back to bools)
- E.g. 2^7 states
Are we done with the constructor?

(5) Over-generalization

Complex constructors tend to imply that…


- Reads are unmanaged (one at a time…)
- Unnecessary reads/writes in destructors
- Unmanaged icache (i.e. virtuals)
=> unmanaged reads/writes
- Unnecessarily complex state machines (back to bools)
- E.g. 2^7 states

Rule of thumb:
Store each state type separately
Store same states together
(No state value needed)
Are we done with the constructor?

(5) Over-generalization

(6) Undefined or under-defined constraints


Are we done with the constructor?

(5) Over-generalization

(6) Undefined or under-defined constraints

Imply more (wasted) reads because pretending you


don’t know what it could be.
Are we done with the constructor?

(5) Over-generalization

(6) Undefined or under-defined constraints

Imply more (wasted) reads because pretending you


don’t know what it could be.

e.g. Strings, generally. Filenames, in particular.


Are we done with the constructor?

(5) Over-generalization

(6) Undefined or under-defined constraints

Imply more (wasted) reads because pretending you


don’t know what it could be.

e.g. Strings, generally. Filenames, in particular.

Rule of thumb:
The best code is code that doesn’t need to exist.
Do it offline. Do it once.
e.g. precompiled string hashes
Are we done with the constructor?

(5) Over-generalization

(6) Undefined or under-defined constraints

(7) Over-solving (computing too much)

Compiler doesn’t have enough context to know


how to simplify your problems for you.
Are we done with the constructor?

(5) Over-generalization

(6) Undefined or under-defined constraints

(7) Over-solving (computing too much)

Compiler doesn’t have enough context to know


how to simplify your problems for you.

But you can make simple tools that do…


- E.g. Premultiply matrices
Are we done with the constructor?

(5) Over-generalization

(6) Undefined or under-defined constraints

(7) Over-solving (computing too much)

Compiler doesn’t have enough context to know


how to simplify your problems for you.

But you can make simple tools that do…


- E.g. Premultiply matrices

Work with the (actual) data you have.


- E.g. Sparse or affine matrices
How do we approach “fixing”
it?
(2) Bools and last-minute decision making
Step 1: organize
Separate states so you can reason about them
Step 1: organize
Separate states so you can reason about them

Step 2: triage
What are the relative values of each case
i.e. p(call) * count
Step 1: organize
Separate states so you can reason about them

Step 2: triage
What are the relative values of each case
i.e. p(call) * count

e.g. in-game vs. in-editor


Step 1: organize
Separate states so you can reason about them

Step 2: triage
What are the relative values of each case
i.e. p(call) * count

Step 3: reduce waste


(back of the envelope read cost)

~200 cycles x 2 x count


(back of the envelope read cost)

~200 cycles x 2 x count

~2.28 count per 200 cycles


= ~88
(back of the envelope read cost)

~200 cycles x 2 x count

~2.28 count per 200 cycles


= ~88

t = 2 * cross(q.xyz, v)
v' = v + q.w * t + cross(q.xyz, t)
(back of the envelope read cost)

~200 cycles x 2 x count

~2.28 count per 200 cycles


= ~88

(close enough to dig in and


measure)
t = 2 * cross(q.xyz, v)
v' = v + q.w * t + cross(q.xyz, t)
Apply the same steps recursively…
Apply the same steps recursively…

Step 1: organize
Separate states so you can reason about them

Root or not; Calling function with context can distinguish


Apply the same steps recursively…

Step 1: organize
Separate states so you can reason about them

Root or not; Calling function with context can distinguish


Apply the same steps recursively…

Step 1: organize
Separate states so you can reason about them
Apply the same steps recursively…

Step 1: organize
Separate states so you can reason about them

Can’t reason well about the cost from…


Step 1: organize
Separate states so you can reason about them
Step 1: organize
Separate states so you can reason about them

Step 2: triage
What are the relative values of each case
i.e. p(call) * count

Step 3: reduce waste


Good News:
Most problems are
easy to see.
Good News:
Side-effect of solving the 90%
well, compiler can solve the 10%
better.
Good News:
Organized data makes
maintenance, debugging and
concurrency much easier
Bad News:
Good programming is hard.
Bad programming is easy.
While we’re on the subject…
DESIGN PATTERNS:


http://realtimecollisiondetection.net/blog/?p=81
http://realtimecollisiondetection.net/blog/?p=44

You might also like