07 · Performance at Scale

Performance & profiling covered measuring and fixing single-function hotspots. At scale, two different techniques matter more: spreading CPU-bound work across multiple isolates, and batching I/O so many small operations don't each pay a fixed overhead cost individually.

Parallelizing CPU work across isolates

A single isolate can only use one CPU core. Splitting a large computation into chunks and running each on its own isolate lets independent work actually run in parallel on a multi-core machine.

import 'dart:isolate';

bool isPrime(int n) {
  if (n < 2) return false;
  for (int i = 2; i * i <= n; i++) {
    if (n % i == 0) return false;
  }
  return true;
}

int countPrimesInRange(List<int> range) {
  int count = 0;
  for (int i = range[0]; i < range[1]; i++) {
    if (isPrime(i)) count++;
  }
  return count;
}

Future<void> main() async {
  const total = 2000000;
  const chunks = 4;
  final chunkSize = total ~/ chunks;

  final sw1 = Stopwatch()..start();
  final sequentialCount = countPrimesInRange([0, total]);
  sw1.stop();
  print('Sequential: $sequentialCount primes in ${sw1.elapsedMilliseconds}ms');

  final sw2 = Stopwatch()..start();
  final futures = <Future<int>>[];
  for (int i = 0; i < chunks; i++) {
    final start = i * chunkSize;
    final end = (i == chunks - 1) ? total : start + chunkSize;
    futures.add(Isolate.run(() => countPrimesInRange([start, end])));
  }
  final results = await Future.wait(futures);
  final parallelCount = results.reduce((a, b) => a + b);
  sw2.stop();
  print('Parallel ($chunks isolates): $parallelCount primes in ${sw2.elapsedMilliseconds}ms');
}
// Sequential: 148933 primes in 213ms
// Parallel (4 isolates): 148933 primes in 83ms

A ~2.5x speedup from 4 isolates on this machine, not a full 4x — spawning isolates and copying the chunk boundaries/results across isolate boundaries (see module 05) has its own cost, and that overhead eats into the theoretical maximum. Chunking too finely (many tiny isolates) can make that overhead dominate and net out slower than staying sequential — always measure the actual speedup for your workload's size rather than assuming isolates are free parallelism.

Batching database writes

Databases showed db.prepare for parameterized inserts. At any real volume, wrapping many writes in a single transaction is the difference between each write paying disk-sync overhead individually versus once for the whole batch.

import 'package:sqlite3/sqlite3.dart';

void main() {
  const n = 5000;

  final db1 = sqlite3.open('/tmp/individual.db');
  db1.execute('CREATE TABLE items (id INTEGER PRIMARY KEY, value INTEGER);');
  final sw1 = Stopwatch()..start();
  for (int i = 0; i < n; i++) {
    db1.execute('INSERT INTO items (value) VALUES ($i)'); // its own implicit transaction
  }
  sw1.stop();
  print('$n individual inserts: ${sw1.elapsedMilliseconds}ms');
  db1.dispose();

  final db2 = sqlite3.open('/tmp/batched.db');
  db2.execute('CREATE TABLE items (id INTEGER PRIMARY KEY, value INTEGER);');
  final sw2 = Stopwatch()..start();
  db2.execute('BEGIN TRANSACTION');
  final stmt = db2.prepare('INSERT INTO items (value) VALUES (?)');
  for (int i = 0; i < n; i++) {
    stmt.execute([i]);
  }
  stmt.dispose();
  db2.execute('COMMIT');
  sw2.stop();
  print('$n batched inserts in one transaction: ${sw2.elapsedMilliseconds}ms');
  db2.dispose();
}
// 5000 individual inserts: 2257ms
// 5000 batched inserts in one transaction: 8ms

That's roughly a 280x difference on a real on-disk database (an in-memory database shows far less of a gap, since there's no disk sync cost to amortize in the first place — always benchmark against the storage you actually deploy on). Every statement outside an explicit transaction runs in its own implicit one, and each commit involves a disk sync; wrapping 5,000 writes in one BEGIN/COMMIT pays that sync cost once instead of 5,000 times.

The trap: over-batching risks losing more work on failure

Batching isn't free of tradeoffs — a batch that fails partway through (a constraint violation on row 4,999 of 5,000) rolls back the entire transaction by default, discarding rows 1 through 4,998 that were individually valid. The fix is choosing a batch size that balances throughput against acceptable retry cost: commit every few hundred/thousand rows instead of one all-or-nothing transaction, so a failure only loses one batch's worth of work, not the whole run.

Cheat sheet

Technique	When to reach for it
Isolate.run per chunk	CPU-bound work large enough that isolate overhead is worth it
Measure actual speedup	Isolate overhead can make fine-grained chunking net slower
BEGIN/COMMIT around many writes	Batches disk-sync cost across many operations instead of paying it per-row
Implicit per-statement transactions	The default without an explicit BEGIN — one sync per write
Bounded batch size	Balances throughput against how much work one failure can roll back
Benchmark on real storage	In-memory vs. on-disk DBs show very different batching payoffs

How It Actually Works

Parallelizing CPU work across isolates gets genuine multi-core speedup because each isolate the VM spawns is scheduled by the OS onto its own native thread with its own heap — unlike async/await concurrency on a single isolate (which only overlaps waiting time on one core), multiple isolates can execute Dart bytecode/machine code simultaneously on separate CPU cores, because there's no shared mutable state between them requiring locks or synchronization to protect. The tradeoff, covered in the isolates lesson, is that getting data in and out costs a deep-copy serialization step across each isolate boundary — which is exactly why "parallelize CPU work" only pays off when the actual computation per unit of work significantly outweighs the fixed cost of spawning isolates and copying inputs/outputs across the boundary; splitting a small array into many tiny isolate tasks can lose to just doing it all on one isolate.

Batching database writes trades round-trip/transaction overhead for increased in-flight risk — most databases (SQLite included) pay a real, fixed cost per transaction commit (fsync-level durability guarantees, lock acquisition, WAL bookkeeping), so wrapping many individual inserts into fewer, larger transactions amortizes that fixed cost across more rows. The over-batching trap is a direct consequence of that same durability mechanism: since nothing is durably committed until the transaction commits, a crash or error partway through building a very large batch loses every row accumulated in that not-yet-committed transaction, not just the one that failed — batch size is fundamentally a throughput-versus-blast- radius tradeoff, not a free efficiency win with no downside.

Exercise

Extend the prime-counting example to accept a chunks parameter and try 1, 2, 4, and 8 chunks against the same total, printing elapsed time for each and noting where adding more chunks stops helping (or starts hurting) on your machine. Separately, modify the batched-insert example to commit every 500 rows instead of one transaction for all 5,000, and time it against both the fully-individual and fully-batched versions — confirm it lands between the two.