Nybble documentation

Core concept

Every field the runtime produces carries its exact offset and size. That per-field byte range is what powers byte ↔ field highlighting: select a field and its bytes light up in the hex view; click a byte and the owning field is selected. A schema executes against a file to produce a tree of { name, type, value, offset, size, children } nodes.

Quick start

A struct lists fields in order. Each field is a name followed by a type:

struct Header {
    magic   char[4]
    version u16
    flags   u16
    size    u32
}

Arrays can take their length from an earlier field:

struct File {
    header       Header
    player_count u32
    players      Player[player_count]
}

Primitive types

u8  u16  u32  u64      // unsigned integers
i8  i16  i32  i64      // signed integers
f32 f64                // IEEE floats
bool  char             // 1-byte boolean / character
string  bytes          // sized text / raw bytes (see below)

Endianness is set per schema (little or big) and defaults to little-endian. Any multi-byte field can override it — see byte order.

Byte order

A schema reads in one byte order, set in its header. Some formats mix the two — a little-endian file with one big-endian timestamp, a network header inside a host-order capture. A be or le suffix on a primitive reads that one field in the stated order:

// @entry Record
// @endian le
struct Record {
    id      u32       // little-endian, the schema default
    stamp   u64be     // this one is big-endian
    scale   f64le     // explicit, same as the default
}

The override covers the field's whole subtree and is unwound afterwards, so it never leaks into the fields that follow. Single-byte types have no byte order, so u8be is rejected.

Varints (LEB128)

varint reads an unsigned LEB128 integer; svarint reads the signed form (SLEB128 — sign-extended, not zig-zag). Each consumes only as many bytes as the encoding needs, up to ten, so the same field can be a different size in every file. The tree still records exactly which bytes it used.

size  varint         // section length
count varint         // how many entries follow
idx   varint[count]  // an array of varints
delta svarint        // signed LEB128

A varint serves as an array length like any other integer, so formats built almost entirely on them — WebAssembly, DWARF, Protocol Buffers — need no special handling.

Arrays & strings

T[N] is an array of N — a literal or an earlier field. string[N] and bytes[N] read a fixed number of bytes; a fixed string stops at NUL padding but still occupies all N bytes.

name  string[32]     // 32 bytes, decoded up to the first NUL
data  bytes[len]     // 'len' bytes, where len is an earlier field
rows  Row[count]     // array of a struct type

Enums

An enum gives symbolic names to an integer. In the tree it shows the matching name, or N (unknown) when no variant matches.

enum ColorType : u8 {
    Grayscale = 0
    RGB       = 2
    Palette   = 3
    RGBA      = 6
}

struct Ihdr {
    colorType ColorType   // shows e.g. "RGBA (6)"
}

The repr after : must be an integer type. Variant values are non-negative literals (decimal or 0x hex).

Bitfields

A bitfield unpacks an integer into named bits. Each member is a single bit (name N) or an inclusive range (name lo..hi), with bit 0 the least-significant. It expands to one child per member.

bitfield GzipFlags : u8 {
    text    0
    hcrc    1
    extra   2
    name    3
    comment 4
    // level 5..6   // a multi-bit sub-field
}

Conditional fields

An if guard reads a field only when the condition holds; otherwise it is absent and consumes no bytes, so later fields keep their offsets.

struct Record {
    version u8
    payload u32 if version >= 2   "added in v2"
}

// XLEN exists only when the FEXTRA flag bit is set:
extraLen u16 if flags.extra

Conditions are a (dotted) field reference, optionally with a comparison:

if <path>             // truthy — non-zero / true
if <path> == <int>    // also != < <= > >=
if <path> == "IEND"   // compare a text field against a string

Pointers & relative offsets

An at clause reads a field from an offset stored elsewhere (ELF's e_phoff, BMP's pixel data, ZIP's central directory) and consumes no sequential bytes. Prefix with + to make the offset relative to the enclosing struct.

firstPixels at dataOffset bytes[4]          // absolute, from a field
progHeaders at +phoff ProgramHeader[phnum]  // relative to this struct
table       at 0x200  TableEntry            // absolute literal offset

A pointer field's offset/size point at the target bytes, so selecting it highlights the pointed-to region wherever it lives.

Discriminated unions — match

When a field's type depends on an earlier tag (TLV records, tagged chunks, protocol messages), match selects the variant to read. Only the selected variant's bytes are consumed.

struct TLV {
    tag  u8
    body match tag {
        0 => u32
        1 => string[16]
        2 => Point            // any type, including a struct
        default => bytes[8]   // optional catch-all
    }
}

Without a default, an unmatched value is a parse error. The tree shows the resolved variant.

Variable-length fields

Three forms handle data whose size isn't a fixed number:

• cstring — a NUL-terminated string (reads up to and including the NUL; to end-of-file if there's no NUL).
• [*] — rest-of-file: bytes[*] / string[*] consumes everything from the field to the end.
• repeat T — read T over and over, for formats built from a run of records (see Repeats).

name     cstring if flags.name   // NUL-terminated file name
payload  bytes[*]                // the rest of the file
sections repeat Section          // records until the bytes run out

Repeats

Most container formats are a run of records — chunks, boxes, TLVs, archive entries — whose count is never stored. repeat T reads T until something tells it to stop:

chunks  repeat Chunk                              // until the input runs out
chunks  repeat Chunk until chunkType == "IEND"    // ...or this element says stop
entries repeat LocalFile while 0x50 0x4b 0x03 0x04  // while these bytes come next
records repeat Record until 0xff 0xff             // until these bytes come next

• until <condition> is tested against each element after it is read, so the terminating element is included — PNG's IEND chunk is part of the tree.
• while <bytes> and until <bytes> peek at the bytes at the cursor before reading. A terminator that is not itself a valid element is never consumed, so the next field starts right on it.

The byte forms are what signature-delimited formats need. The built-in ZIP schema walks a whole archive this way — every local file entry, then the central directory, then the end record, each section ending exactly where its signature stops matching.

Computed fields

A field written name = expr is computed from earlier fields and reads no bytes — handy for a length the format only stores implicitly.

struct Blob {
    total  u32
    header u32
    dataLen = total - header    // computed, zero bytes
    data    bytes[dataLen]      // ...usable as a later length
}

Expressions support + - * / %, parentheses, integer literals, and (dotted) field references, with normal precedence.

Decoding & transforms

Real files hide their structure behind compression or light obfuscation. A decode clause transforms a field's bytes after reading them, and an optional as re-parses the result as another type — so a compressed blob becomes a real tree instead of a wall of noise.

payload bytes[len] decode zlib               // inflate, keep as bytes
module  bytes[*]   decode gunzip as Module   // inflate, then parse as Module
save    bytes[*]   decode xor(0x5A) as Save  // de-obfuscate, then parse

The available transforms:

• zlib / zlib_inflate — zlib stream (RFC 1950)
• inflate / deflate_raw — raw DEFLATE (RFC 1951)
• gunzip / gzip — gzip member (RFC 1952)
• base64 — standard-alphabet Base64
• xor(k, ...) — repeating-key XOR, one or more key bytes
• rolling_xor(seed, mul, add) — the key updates as k = k * mul + add after every byte
• add(n) / sub(n) — shift every byte by a constant

A decoded field keeps its real span in the file — the compressed bytes — so highlighting still points at the region the data actually occupies. The fields beneath it are positioned within the decoded stream, since those bytes exist nowhere on disk.

Checksums

A check clause says a field holds a checksum of other bytes. Nybble recomputes it on every parse and shows whether the stored value still matches — and, when it doesn't, the value it should hold.

struct Chunk {
    length    u32
    chunkType char[4]
    data      bytes[length]
    crc       u32 check crc32 over(chunkType .. data)
}

crc  u16be check crc16_ccitt over(header .. payload)
sum  u8    check sum8 over(body)          // a single field

over(a .. b) covers every byte from the start of a to the end of b; over(a) covers one field. The algorithms:

• crc32, adler32
• crc16 (ARC, the usual bare meaning), crc16_modbus, crc16_ccitt (CCITT-FALSE), crc16_xmodem — a datasheet that says "CRC-16" could mean any of them
• sum8, sum16, sum32, xor8

A mismatch is not a parse error, so the tree survives it. After you edit a file, one click rewrites every stale checksum so it still opens in the tool that made it.

When a schema is wrong

Most of the time you spend writing a schema, it does not yet fit the file. That is the normal state, so it is not treated as a crash: a parse that hits a field it cannot read keeps every field it already decoded and stops there. You get the tree so far, plus a fault describing the wall it hit.

• the byte offset where it stopped — click the fault to jump the hex view there
• the field path, e.g. PNG.chunks[3].length, so you know which element of which repeat went wrong
• the line of your schema that declared the failing field, marked inline in the editor

Parsing stops at the first fault rather than trying to resynchronise. Once a field is misread, every offset after it is a guess — a tree full of plausible nonsense is harder to debug than an honest stop.

A truncated field reports only the bytes it actually covered, so the hex highlight never claims bytes that were never read. One exception to offsets: a fault inside a decode block is measured in the decoded stream, since those bytes have no position on disk.

Coverage

A schema that parses cleanly is not necessarily finished. Every parse reports how much of the file it explained: the bytes no field claims are the part you haven't worked out yet. The file map shades those runs, and the readout steps to the next one so you can drag it into a field.

Only fields that read bytes themselves count, so a gap between two fields shows up instead of hiding inside the struct around them. Fields beneath a decode are left out, since their offsets index the decoded bytes, not the file.

For a region you can't place yet, the shape-hints panel looks for repeating records and their stride, tables of offsets, string pools, and padding — each stated with the evidence behind it, so it reads as a lead rather than a verdict.

Command line

The app explores one file at a time. The nybble binary is for the rest: running a finished schema over a hundred files, diffing two captures in a script, or asserting in CI that a format still parses.

nybble parse <schema> <file> [--json]   # the field tree
nybble diff <a> <b>                     # changed regions, first file to second
nybble detect <file>                    # identify by signature (reads 512 bytes)
nybble strings <file> --min 6           # readable text, ASCII and UTF-16
nybble entropy <file> --buckets 128     # where the packed data is
nybble hints <file> --at 0x4000         # what shape are these bytes?
nybble check <schema>                   # does the schema compile?

A schema can name its own entry struct and byte order in a header comment, so the usual invocation needs no flags (--entry and --endian override them):

// @entry PNG
// @endian be
struct PNG { ... }

The exit status is what scripts care about: 0 matched, 1 did not (a fault, a failed checksum, a difference), 2 a usage error. Because a repeat that stops early still parses without a fault, --min-coverage puts a floor under how much of the file the schema must explain:

nybble parse firmware.schema build/out.bin --min-coverage 99 --quiet