Recently I consumed a lot of information about Data-Oriented Design. For me, the concept started with Mike Acton’s talk (Data-Oriented Design and C++). And I don’t think I’m alone in that — it’s often cited as a classic introduction to the subject, not just by me but by the wider community.

I don’t want to repeat the whole talk or give an introduction to the topic, but let’s recap the general principles:

• Data is the primary abstraction.
  Don’t start by thinking about objects and interfaces. Start by answering the question: “What data do I have, and how will I transform it?”
• Design for actual usage patterns.
  Your architecture should follow real access patterns: who performs which operations, how frequently they are performed, in what order, and which data is accessed together or separately.

So, when designing something, you should resist thinking about objects the way we naturally tend to think about them and instead focus on the data and how it is transformed.

Now, let’s look at a few techniques that can help us apply these principles in practice:

• Improve data locality.
  If you can keep commonly used data together in one place, do that.
  This way, you can reduce the cost of loading that data.
• Separate data by access pattern.
  If two fields have different lifetimes or access frequencies, there is no need to store them together just because they are logically related.
• Process data in batches.
  It often helps.
• Minimize unnecessary data movement.
  Moving data can be more expensive than the computation itself.
• Measure, don’t assume.
  Base your design on actual data, not on your mental model of it.
• Transform data to make computation efficient.
  Sometimes, it’s better to have multiple data representations for different workloads rather than one universal representation.

Knowing these general principles is great. But how do we apply them in practice?

The article is divided into two parts:

• how to understand your data
• examples of decisions driven by knowledge of the data

Since most of my experience is in backend development, we’re going to discuss these ideas mostly in the context of backend systems. That said, there’s nothing stopping us from applying the same concepts elsewhere.

How to understand your data

I don’t want to turn this into a full course on data analytics. I’m not really qualified to teach one anyway. But let’s see what we can learn from our data.

First of all, understand your data lifecycle. It can be represented as a graph (or a tree). For some abstract objects, it might look like this:

created
   ↓
updated ───→ read
   ↓           ↓
deleted      aggregated
               ↓
             exported

Now we need to investigate a few things.

Who performs this operation?

For example, creation, updates, and deletion might be performed by system administrators through system A, while reading, aggregation, and exporting might be done by analysts through system B.

How frequently is this operation performed, and how much data do we need?

That already gives you an idea of what is more important and what is less important.

How can you collect that data?

Often, good observability will do the work for you. You need to know about:

• query volume (per second or per day)
• the distribution of those queries over time Writing code for 1,000 QPS throughout the day with spikes up to 1,500 QPS is not the same as optimizing for a usual 300 QPS with spikes up to 20,000 QPS.

Queries can have different shapes. Someone might want to retrieve just one object, while someone else might need huge batches containing thousands of objects.

You also need to collect as many metrics about your workload as possible. Log the sizes of arrays in queries, requests, and responses, data cardinality, types of queries, and even individual object IDs.

You need to find correlations in your data. Maybe, for some reason, you have more queries with batch sizes between 100 and 500. Maybe three particular pages are 50% slower on average. Maybe your search performs worse for queries with seven characters. Find those patterns.

And don’t think only about the amount of something. When you hear 100k QPS and 10k QPS, you might immediately assume that the first case is more challenging. But what about the overall traffic pattern?

100k req/s * 100 bytes = 10 MB/s

but

10k req/s * 100KB = 1 GB/s

Architecturally, these situations are very different.

And all the data you have should be inspected for any skews. As always, we can’t rely only on averages. We learned long ago that tail latency also matters, and we need to check different percentiles. We eventually have “celebrities” in our systems who have a lot of followers or other links to all kinds of data. We need to handle these cases differently, or at least keep them in mind when designing our systems, because these outliers can heavily affect caching, replication, sharding, and partitioning.

Sometimes, you need to find some temporal locality. Maybe some data is accessed a lot, but only for a short period of time. This could heavily affect your caching strategy, for instance: LRU might work well for the pattern above, while LFU might be better for data that is accessed frequently but with accesses more spread out over time.

The thing we often forget, but DOD reminds us, is that usage frequency itself is part of our data. Even if it’s not represented in the schema.

Which data is actually needed?

Do all users really need all the data about your objects? Maybe some of them need only 2 fields out of 10, while others need all the data you can possibly provide. If your requirements are heavily skewed toward one use case, use that to your advantage.

And which data do you usually provide together, and which separately? It’s reasonable to expect that someone who needs your first name also needs your last name. But that doesn’t mean they need your driving license number.

You can create a data access matrix that tells you how frequently data is needed together. For instance, for 5 objects A, B, C, and D, you might have something like this:

       A    B    C    D
A      -   90%  2%   1%
B     90%   -   3%   1%
C      2%   3%   -  80%
D      1%   1%  80%   -

And now you have 2 natural groups: A + B and C + D. We found correlated access patterns.

Sometimes it’s not only about overall frequency, but also about conditional frequency. Ask yourself: “How likely am I to need B if I already need A? And vice versa?”

How fresh should your data be?

If you develop some trading system where you need instant ability to ban some untrustworthy trader you design system in one way. Such read-heavy systems usually allow you to move workload to write operations. But you need to be sure that it’s applicable to your specific situation.

But if you develop some analytical data storage (telemetry/metrics system) you can choose other tradeoffs You can process new metrics in batches and deliver them in minutes, not miliseconds one by one. We can maintain append-only storage to increase write throughput. We can put data as we get it and since it’s being read rarely we can perform really heavy queries on client side.

You can choose resources to save and to spend according to your tradeoffs.

What’s next?

Those questions are really tough to answer, to be honest. And, unfortunately, I don’t have any advice on how to do that easily. You just need to bite the bullet. Usually, we have so much data that we need to spend days or weeks getting all the answers we need. And it’s great if your data is stored somewhere in a database. Sometimes, you want to analyze client requests and your responses; maybe even client code could help. You need a strong observability platform and the ability to add a lot of different metrics in order to dig into your workload. It’s also great to understand your system deeply so you can save some time by avoiding unuseful information.

I’m not saying that you need to know all those facts about your data. You need to know more than nothing. But of course, the more you know, the better decisions you can make.

Some examples

Okay. I researched my data. And what now?

Now you basically have your workload model. Good for you!

Let’s check some synthetic cases where we can apply DOD to make things better.

Data structure

One of the most well-known optimizations that is actually an example of DOD is Small Object Optimization in string implementations. We have it because our systems tend to have a lot of short strings. And the libraries we often use try to optimize for such cases.

But you can go further and think about your specific cases. For instance, let’s suppose you have a function:

std::vector<Converted> Convert(std::span<const Item> items);

And then you collected items.size() data from a lot of Convert calls:

0 items   40%
1 item    35%
2 items   15%
3 items    6%
4 items    3%
5+ items  1%

Optimize for 4 elements:

boost::container::small_vector<Converted, 4> Convert(std::span<const Item> items);

Now you don’t waste your time on 99% of all cases. Great! Unless you have something like this:

0–4       1%
5–100     20%
100–10000 79%

In that case, no improvement can be achieved. Formally, our data is still an array. But the distribution can radically change which data structure is the best choice.

Algorithm

It’s obvious to us that, usually, algorithms with better complexity work faster. But what if that’s not the case?

Suppose you need to implement a straightforward query: find an object by its object_id. My usual first step would be to create some map:

std::unordered_map<ObjectId, Object> index;

But then we found that:

95% collections: N <= 8
99% collections: N <= 32

Let’s just do linear search!

for (auto& obj : objects) {
    if (obj.id == id) {...}
}

Of course, it’s not only about cOmPlEXitY. We also have other factors:

• predictable memory access
• caches
• no hash computation or indirect accesses through pointers
• smaller memory footprint.

Don’t forget to remember about how computers work in general.

Data storage type

This is a classic example.

Suppose in your game you have some object:

struct Entity {
    Position position;
    Velocity velocity;
    Health health;
    Name name;
    Weapon weapon;
    Animation animation;
};

And we have a world with millions of them:

struct World {
    std::vector<Entity> entities;
};

Every frame, you need to update the world state: entity positions. For this step, you only need position and velocity. If you do something like this:

for (auto& entity : entities) {
    entity.position += entity.velocity * dt;
}

The thing is, when reading the data from memory, you read the whole entity into your cache. But you only need 2 fields! We can change that by moving from an Array of Structs (AoS) to a Struct of Arrays (SoA):

struct World {
    std::vector<Position> positions;
    std::vector<Velocity> velocities;
    std::vector<Health> health;
    std::vector<EntityInfo> entities;
};
// somewhere in update
for (size_t i = 0; i < N; ++i) {
    positions[i] += velocities[i] * dt;
}

Control flow

You can even go further.

Suppose

struct Entity {
    Position position;
    Velocity velocity;
    Physics physics;
};

and you know that

95% entities — regular move
4%           — collision-enabled
1%           — complex physics

If you put all the data into one struct and then try to process an object with complex physics, you will read all the other data for nothing. We can split the data:

struct World {
    std::vector<SimplePhysics> simple_entities;
    std::vector<ComplexPhysics> complex_entities;
};

And later, you can process them separately. You moved from:

for (auto& entity : entities) {
    if (entity.IsSimple()) {
        DoSimple(entity);
    } else if (entity.IsComplex()) {
        DoComplex(entity);
    }
}

to

DoSimple(simple_entities);
DoComplex(complex_entities);

You literally changed the control flow of your program because you knew something about your data.

The same example could be applied in another area: storage.

Suppose you have some backend system which partitions data in Kafka or some database. And your partitioning is based on hashing some data:

hash(customer_id) % N

But your production data tells you that

1 top customer:     18% traffic
Next 100:           30%
Remaining millions: 52%

In that case, using hashing for partitioning may lead to unbalanced hot partitions. Now you are better off using a special way of storing (and processing) hot customers’ data. It could be dedicated partitions or even sharding only their data. And all other data could be stored in the ordinary way with hash partitioning.

Here, data distribution changes not only the control flow but also the topology of the system.

Encode data

Suppose somewhere in your codebase you have

std::string country;

But from the data distribution, you know that there aren’t many countries. Fewer than 256. Let’s encode that data:

// everywhere
uint8_t country;

// country_encode.hpp
std::string DecodeCountry(uint8_t country) {
    static std::array<std::string, kCountriesCnt> mapping{
        "Belarus",          // 0
        "United Kingdom",   // 1
        "Poland",           // 2
        ...
    };
    return mapping[country];
}

You trade 24 or 32 bytes of storage for std::string every time for a 1-byte value. This not only helps you save memory, but can also make your code more cache-friendly in some cases. In some even more specific cases, you can prepare your code for SIMD operations.

Caching

Suppose you have 100M objects. We want to make our system work faster and want to introduce a cache. What should we cache? Maybe everything?

The access distribution tells us that

1% objects  → 80% reads
10% objects → 18% reads
90% objects → 2% reads

Now we can use a small hot cache for 11% of the data and a large cold storage for everything else.

If the access pattern is evenly distributed, a cache may not help.

But what about the temporal distribution of data? What if all your responses are useful for a long period of time? Maybe you know that

TTL usefulness:
< 1 sec  → 90% requests
1–10 sec → 9%
> 10 sec → 1%

In that case, having a cache with a TTL of 5 minutes could be useless.

Don’t store some information

Suppose we have

struct Order {
    ...
    Currency currency;
};

But from production stats you know:

GBP 97.8%
USD 1.5%
EUR 0.6%
BYN 0.2%
other 0.1%

We don’t need to store the currency in every order. We can store it only for deviations:

struct OrderBook {
    Currency default_currencu;
    std::vector<Order> orders;
    std::unordered_map<OrderId, Currency> deviations; // only 2.4% of object here
};

Or in case when you have some flag

bool is_deleted = false; // 99.99% of object aren't deleted

you can just store all deleted objects ids separately.

Read or write?

Suppose you have 3 systems and еру read/write QPS for all of them:

1st:
- 10M writes/sec
- 10M reads/sec

2nd:
- 10k writes/sec
- 10M reads/sec

3rd:
- 10M writes/sec
- 10k reads/sec

You don’t need to know the absolute values to understand what is more important to optimize. The ratio is what you need.

It’s obvious that in the 1st system, we probably want to achieve similar performance for both types of operations. This is just a heavily loaded system.

The 2nd system is read-heavy. That means that we probably want to make writes more expensive operations in order to make reads as fast as possible. For instance, we could create different data representations for different types of queries. So when reading, we could just choose the proper data source and return the data without any computation.

In the 3rd system, we want to speed up writes as much as we can. Maybe that means that we want to append new data when writing and calculate everything from the beginning to the end when reading.

Or we might also know some latency requirements, and they could turn everything upside down. Who knows?

In the Meta’s article about Dragon you can find example of similar reasoning.

How data size can affect decisions

Suppose you have Redis and queries with multiple keys. Redis is a single-threaded storage, which means that one big command can cause a delay for all clients. So it’s better to do some batching.

But how could we choose the batch size? Maybe 500 or 1,000 objects per command?

Of course, we need to think about the data. Suppose we know that one value is around 100 bytes. In that case, 10,000 could be a reasonable size, so we could stay around 1 MB per command. But if one value is 5,000 bytes, maybe 200 values per command is enough.

Or suppose you have a data storage. Some of the objects are small, while others are huge. If you have a lot of small objects, it’s possible that metadata could take up more memory than the objects themselves. In that case, you may want to compact small objects into larger ones. That what AWS descirbed in their article.

Do we actually need complex system?

One mistake I made once (fortunately, it was during a system design interview at a big social media company, not on a real-life project), and one I’ve seen real engineers make a few times at work, is assuming that your task is really complex. Starting a new project, you could ask yourself, “Which distributed database should I use?” You choose Postgres, write all the code, and push everything to production. Everything works fine!

But you only have 20 MB of data. Was it worth it?

Maybe an in-memory cache would have solved the problem? Sometimes, we can do things more simply if we know what we actually need to do.

The end

Remember that your system should be built around the data you have, how you transform it, and how you need to return it. Detach yourself from the way people naturally think about things. We’re not really good at thinking like machines. But if we try to think like them, we can do a much better job of creating better, more performant systems.

And don’t assume. Measure.

Additional stuff

In a talk «A Practical Guide to Applying Data Oriented Design» by Andrew Kelly (Zig creator) you can find more DOD usage examples. Mainly on example of changing the objects’ memory layout.