07 · Working with JSON

The Level 1 project stored tasks in a hand-rolled pipe-delimited text format specifically to avoid needing JSON before it was covered. This module fills that gap using upickle, a small, dependency-light JSON library for Scala. (Its sibling library circe is the other library you'll see often in production Scala codebases — same ideas, more type-class-heavy API — but upickle's derives syntax is the gentler on-ramp.)

Adding upickle to a project

// build.sbt
libraryDependencies += "com.lihaoyi" %% "upickle" % "3.3.1"

Deriving JSON for a case class

upickle can generate a ReadWriter for any case class automatically with Scala 3's derives clause — no manual encoder/decoder boilerplate:

import upickle.default._

case class Address(city: String, zip: String) derives ReadWriter
case class Person(name: String, age: Int, address: Option[Address]) derives ReadWriter

val p = Person("Ada", 30, Some(Address("London", "E1")))

val json = write(p)
println(json)
// {"name":"Ada","age":30,"address":[{"city":"London","zip":"E1"}]}

val parsedBack = read[Person](json)
println(parsedBack)              // Person(Ada,30,Some(Address(London,E1)))
println(parsedBack == p)         // true -- case class equality makes round-tripping easy to verify

write serializes any type with a ReadWriter in scope to a JSON string; read[T] parses a JSON string back into T, given the exact type annotation (upickle needs read[Person], not just read, since the target type can't be inferred from a String).

The trap: Option doesn't serialize the way you'd guess

It's natural to assume Some(x) becomes x or null and None becomes a missing key or null. upickle actually represents Option[T] as a JSON array of zero or one elements — [] for None, [value] for Some(value):

val noAddress = Person("Bob", 25, None)
println(write(noAddress))
// {"name":"Bob","age":25,"address":[]}

val roundTripped = read[Person](write(noAddress))
println(roundTripped)   // Person(Bob,25,None) -- round-trips correctly...

// ...but if you're consuming JSON from a NON-Scala API (a real HTTP
// endpoint, not one you serialized with upickle yourself), it almost
// certainly represents "no address" as either a missing "address" key or
// "address": null -- neither of which upickle's default Option encoding
// expects. Model fields coming from external APIs as plain (non-Option)
// types with sensible defaults, or write a custom ReadWriter, rather than
// assuming Option "just works" against arbitrary JSON.

This is the single most common surprise when picking up upickle: it's perfectly consistent for JSON upickle produced itself, but it is not the "idiomatic JSON" convention (missing key / null) most external APIs use for optional fields.

Pretty-printing

println(write(p, indent = 2))
// {
//   "name": "Ada",
//   "age": 30,
//   "address": [
//     {
//       "city": "London",
//       "zip": "E1"
//     }
//   ]
// }

Parsing JSON you don't have a case class for

Sometimes you just need to pull a couple of fields out of a response without modeling the whole shape. ujson.read parses into a generic, dynamically-navigable ujson.Value tree:

val raw = """{"name":"Cleo","age":5,"tags":["cat","fluffy"]}"""
val parsed = ujson.read(raw)

println(parsed("name").str)                        // Cleo
println(parsed("age").num)                          // 5.0 -- JSON numbers are Double by default
println(parsed("tags").arr.map(_.str).mkString(","))// cat,fluffy

.str, .num, .arr, .obj, and .bool unwrap a ujson.Value to the Scala type you expect — and throw if the value is actually a different JSON type, which is the generic-tree equivalent of Option.get: fine for a quick script, risky against JSON you don't fully control. For anything long-lived, model it as a case class with derives ReadWriter instead so the compiler checks the shape for you.

Handling malformed or unexpected JSON

Reading into a typed case class throws when the input doesn't match — combine it with Try (from Module 4) to turn a parse failure into an Option/Either instead of an uncaught exception:

import scala.util.Try

def parsePerson(json: String): Either[String, Person] =
  Try(read[Person](json)).toEither.left.map(_ => s"invalid person JSON: $json")

println(parsePerson("""{"name":"Ada","age":30,"address":[]}"""))
// Right(Person(Ada,30,None))

println(parsePerson("""{"name":"NoAge"}"""))
// Left(invalid person JSON: {"name":"NoAge"})

Wrapping every external read[T] call this way is worth the small amount of ceremony — it turns "a malformed response crashes the program" into "a malformed response is a value the caller has to explicitly handle," exactly the philosophy Module 4 built up.

How It Actually Works

"Deriving JSON for a case class" without writing any encoder by hand relies on the same compile-time mechanism as implicit resolution: libraries like upickle generate a ReadWriter[YourCaseClass] at compile time using macros (or, in some libraries, Scala 3's inline/derivation features) that inspect the case class's constructor parameter list — available because a case class's field names and types are part of its public, compiler-visible signature — and mechanically produce code that walks each field, converting it via that field's own derived ReadWriter, recursively. Nothing is discovered via reflection at runtime; the encoding/decoding logic for Employee(name: String, salary: Double) is generated once, as ordinary methods, when the project compiles — which is also why forgetting to import the right implicit gives a compile error ("no ReadWriter found") rather than a runtime surprise.

The Option serialization "trap" comes directly from this mechanism: there's no single universal JSON representation of "a missing field" vs. "a field present with value null" vs. "None," so each library's derived ReadWriter[Option[A]] makes its own choice (upickle typically omits the key entirely for None, or represents it depending on version/config) — the behavior is a property of the specific derived instance you're using, not something intrinsic to Option itself.

Parsing JSON you don't have a case class for works because underneath the case-class layer, every JSON library has an untyped AST type (e.g. upickle's ujson.Value, a sealed hierarchy of Str, Num, Obj, Arr, Bool, Null) — the exact same sealed-trait-plus-case-classes pattern from Level 1 and Module 8, just modeling "any JSON value" instead of your domain type. read[YourCaseClass](json) is really "parse to this AST, then run the derived decoder against the AST" — two separate compiler-generated/generic stages, which is why parsing to ujson.Value and then manually navigating it (.obj("field").str) is always available as a fallback when no case class fits the shape.

Cheat sheet

Task	Code
Derive JSON support for a case class	case class Foo(...) derives ReadWriter
Serialize to a JSON string	write(value)
Pretty-print	write(value, indent = 2)
Parse into a known type	read[Foo](jsonString)
Parse into a generic tree	ujson.read(jsonString)
Navigate a generic tree	.obj, .arr, .str, .num, .bool, parsed("key")
Handle parse failure as data	Try(read[Foo](s)).toEither

Exercise

Define case class Book(title: String, author: String, year: Int, tags: List[String]) derives ReadWriter. Create a List[Book] with at least three entries, serialize the whole list with write (hint: write(books) works directly on a List[Book] since upickle can derive collection support once the element type has a ReadWriter), then parse it back with read[List[Book]] and confirm it equals the original list. Then take a hand-written JSON string missing the year field and confirm read[Book] throws — wrap that call in a Try and print a friendly error message instead of letting the exception propagate.