An open-source PKCS#11 v3.2 cryptographic module under Simorgh Labs
slotN.tok)Roadmap item #108. Documents the format as implemented in src/fhsm_token.c
(v1.4.0 line). CC class: ADV_TDS.3 / ADV_FSP.4 supporting evidence.
One file per slot (slot0.tok, slot1.tok, …), owner-only permissions
(0600), written atomically (full image to <path>.tmp, fsync,
single rename(2)) so a power loss can never leave a half-written token.
The format is compact fixed-layout binary — not JSON. Rationale: constant-time parse with no allocator on the hot path, no base64 round-trips, and minimal parser attack surface per CC EAL4+ ADV_TDS.3 (“the simpler the better”).
Historical note. The Python POC used a JSON token layout, and an early revision of
fhsm_token.hstill described it. The C implementation uses the binary layout below; the stale header comment was corrected when this document was introduced. There is no byte-level interop with POC token files — migration is by re-importing objects, not by copying.tokfiles.
[ 317-byte fixed header ]
[ optional encrypted objects section ]
| Offset | Size | Field | Notes |
|---|---|---|---|
| 0 | 4 | Magic "FHSM" |
ASCII, no NUL |
| 4 | 4 | Version = 1 | LE u32 |
| 8 | 64 | label |
NUL-padded ASCII |
| 72 | 64 | serial |
NUL-padded ASCII, FHSM- + 16 hex chars (8 DRBG bytes), generated once at init |
| 136 | 4 | pbkdf2_iter |
LE u32, currently 200 000 |
| 140 | 16 | salt_so |
PBKDF2 salt, SO role |
| 156 | 12 | so_wrap_nonce |
GCM nonce for SO DEK wrap |
| 168 | 48 | so_wrap_ct |
32-byte encrypted DEK ‖ 16-byte GCM tag |
| 216 | 16 | salt_user |
PBKDF2 salt, USER role |
| 232 | 12 | user_wrap_nonce |
|
| 244 | 48 | user_wrap_ct |
|
| 292 | 1 | user_initialized |
0/1 |
| 293 | 4 | failed_so |
LE u32, persisted so lockout survives restart |
| 297 | 4 | failed_user |
LE u32 |
| 301 | 8 | reserved, written as zero | was throttle_so_until_ms; see below |
| 309 | 8 | reserved, written as zero | was throttle_user_until_ms |
All multi-byte integers are little-endian.
u32 ct_len (LE)
u8 nonce[12]
u8 ct[ct_len] AES-256-GCM ciphertext
u8 tag[16]
Plaintext of ct (before encryption):
u32 object_count (LE, ≤ FHSM_MAX_OBJECTS = 64)
u32 next_handle (LE, monotonic handle allocator)
object_count × 5 620-byte records
FHSM_OBJ_REC_SZ)| Offset | Size | Field | Notes |
|---|---|---|---|
| 0 | 4 | handle |
CK_OBJECT_HANDLE, opaque |
| 4 | 4 | class |
CKO_SECRET_KEY, … |
| 8 | 4 | key_type |
CKK_AES, … |
| 12 | 4 | value_len |
actual bytes used in value |
| 16 | 64 | label |
CKA_LABEL, NUL-padded |
| 80 | 5500 | value |
key material; unused tail zeroized |
| 5580 | 32 | id |
CKA_ID |
| 5612 | 4 | id_len |
|
| 5616 | 1 | flags |
bit 0 = CKA_PRIVATE, bit 1 = CKA_EXTRACTABLE |
| 5617 | 3 | padding | zero |
FHSM_OBJ_VALUE_LEN = 5500 is sized for the largest supported key
(ML-DSA-87 PKCS#8 private ≈ 4 896 B, with margin; RSA-4096 ≈ 2 400 B,
ML-KEM-1024 ≈ 3 168 B).
Maximum objects-section plaintext: 8 + 64 × 5 620 = 359 688 bytes.
The loader accepts up to FHSM_OBJ_BLOB_MAX (this bound); see the
regression note below.
| Aspect | Value |
|---|---|
| KEK derivation | PBKDF2-HMAC-SHA-256, 200 000 iterations, 16-byte per-role salt |
| DEK | 256-bit, generated by the FIPS DRBG, resident in the mlock-ed secure heap while logged in, zeroized on logout/close |
| DEK wrap (per role) | AES-256-GCM, 12-byte DRBG nonce, AAD = token serial |
| Objects blob | AES-256-GCM under the DEK, fresh nonce per write, AAD = token serial |
| DEK rotation | SO C_SetPIN rotates the DEK and re-wraps both roles (SP 800-57 §5.4); USER C_SetPIN re-wraps USER only |
Binding the GCM AAD to the serial means a header/blob spliced from a different token fails authentication even if the PIN-derived keys match.
FHSM_PIN_MAX_FAILED = 5 → role locked (FHSM_RV_PIN_LOCKED).The counter is persisted in the header (plaintext by design — it must be readable before login). The deadline is not: it is derived from the counter on load, and offsets 301/309 are written as zero.
They used to be stored, in the CLOCK_MONOTONIC domain, so that date -s
could not bypass the throttle. That reasoning was right and its consequence
was not examined: CLOCK_MONOTONIC restarts at boot while the file does
not, so a deadline written after thirty days of uptime was still thirty days
in the future on the next boot. A 500 ms throttle became a 29.8-day refusal
of the correct PIN, reported as PIN_THROTTLED with nothing to explain it.
Measured, then fixed; tests/test_throttle_reboot.c plants a long-uptime
deadline in the file and asserts the reload stays within the cap.
The deadline was never independent state — it is a function of the counter. Deriving it removes both the reboot trap and the possibility of the two disagreeing. A restart does not clear the delay: it is re-imposed from the counter, so restarting is not a way to skip a wait that was earned.
date -s still cannot bypass anything, for the same reason as before: the
derived deadline is monotonic within the boot that computed it.With FHSM_TPM_SEALING=1, fhsm_token_init additionally seals the DEK
to the TPM 2.0 and writes a companion file <path>.tpm
(see fhsm_token_tpm.h). If sealing is requested but no TPM is
reachable, initialization fails hard (loud misconfiguration, audit
event FHSM_EV_SEAL_FAILURE). The .tok format itself is unchanged by
sealing.
ct_len + 16… i.e.
317 + 4 + 12 + ct_len + 16 bytes; anything else is corruption.ct_len ≡ 8 + k × 5 620 for some 1 ≤ k ≤ 64.write_atomic() (single rename).value fields exist
only inside the GCM-protected blob.objects_blob absent ⇒ object_count = 0 (fresh or re-initialized
token). fhsm_token_reinit intentionally drops the blob.The v1.4.0 loader rejected any objects section with ct_len > 65 536,
while the writer happily produces up to 359 688 bytes. Result: a token
holding more than 11 objects persisted fine but failed to load at the
next login (data intact on disk, store unreadable). Fixed alongside
this document by raising the loader sanity bound to FHSM_OBJ_BLOB_MAX
(= 8 + FHSM_MAX_OBJECTS × FHSM_OBJ_REC_SZ), with a regression test.
Implemented with CKO_CERTIFICATE support. The 317-byte file header is
unchanged; the blob plaintext is now self-versioned:
u32 magic = 0xF5B20002 (cannot collide with a v1 count <= FHSM_MAX_OBJECTS)
u32 object_count (<= FHSM_MAX_OBJECTS)
u32 next_handle
object_count x records:
u32 rec_len (bytes after this field = 120 + value_len)
u32 handle | class | key_type | value_len (16 bytes)
u8 label[64]
u8 id[32]
u32 id_len
u8 flags ; u8 usage_flags ; u8 pad[2]
u8 value[value_len] (<= FHSM_OBJ_VALUE_MAX)
Properties:
u32
(<= 64 => legacy v1 fixed records; magic => v2). Any write
re-serializes as v2, so tokens migrate transparently on first
mutation.FHSM_OBJ_VALUE_MAX = 2 MiB accommodates X.509 certificates carrying
PQC / composite keys and signatures (ML-DSA-87 cert ~ 8-10 KB).
FHSM_OBJ_VALUE_LEN = 5 500 is frozen as the legacy v1 field size.FHSM_OBJ_BLOB_MAX is derived from
the current record size and covers the v1 and v2 bounds too.value_len must
equal rec_len - 120 and id_len <= 32, else the whole blob is
rejected (GCM already authenticates it — a mismatch is a logic bug,
not an attacker).count = 0xF5B20002 > FHSM_MAX_OBJECTS and fails loudly
(FHSM_RV_FUNCTION_FAILED), never silently.CKA_START_DATE, CKA_END_DATE and CKA_APPLICATION had nowhere to live:
they were accepted at creation and read back empty. v3 is v2 with an 84-byte
metadata block inserted between the fixed fields and the value.
u32 magic = 0xF5B30003
u32 object_count (<= FHSM_MAX_OBJECTS)
u32 next_handle
object_count x records:
u32 rec_len (bytes after this field = 204 + value_len)
u32 handle | class | key_type | value_len (16 bytes)
u8 label[64]
u8 id[32]
u32 id_len
u8 flags ; u8 usage_flags ; u8 pad[2] (-- end of v2 layout, 120 --)
u8 start_date[8] ; u8 start_date_len
u8 end_date[8] ; u8 end_date_len
u8 application[64]; u8 application_len
u8 pad
u8 value[value_len]
Properties:
C_GetAttributeValue, and
the payload alone cannot distinguish them.CKA_APPLICATION is bounded at 64, the
same as CKA_LABEL, which makes the fixed layout trivial.parse_objects() dispatches on the leading
u32: v3 magic, else v2 magic, else a v1 count. Those readers are kept
because they already exist and are exercised — the failure mode of deleting
them is not a red test, it is a key nobody can reach. Every write emits v3,
so conversion is one-way, as v1 → v2 already was.FHSM_OBJ_BLOB_MAX = 12 + 1024 x (4 + 204 + 2 097 152)
= 2 147 696 652 bytes. It is a bound only – nothing allocates it. It was
derived from FHSM_OBJ_REC_V3_FIXED while v3 was the written format; it is
derived from FHSM_OBJ_REC_V4_FIXED now, and the figure above is v3’s.
Forgetting to move it with the format is what broke loading past 11 objects
in v1.4.0 (see the regression note above).<= 8, <= 8, <= 64), else the whole blob is rejected.The metadata is not consulted by anything. PKCS#11 v3.2 calls the dates informational and states that Cryptoki does not enforce them; the module stores them and stops there. Enforcing an expiry would be the mirror of the bug this record fixes — claiming a control nobody asked for is no better than dropping one that was.
CKO_CERTIFICATE (0x1) objects store the complete DER certificate in
value; the store’s key_type field carries CKA_CERTIFICATE_TYPE
(CKC_X_509 = 0). Certificates are non-sensitive and extractable
(flags = EXTRACTABLE): C_GetAttributeValue(CKA_VALUE) returns the
DER. The module never parses X.509 — validation is the PKI layer’s job.
Four attributes had nowhere to live, for the reason the v3 section predicted a
v4 would be needed: CKA_ALLOWED_MECHANISMS, CKA_WRAP_TEMPLATE,
CKA_UNWRAP_TEMPLATE and CKA_DERIVE_TEMPLATE are per-object policy of
variable length. Three were refused outright; the fourth had partial support
that turned out to overlap nothing (see below). v4 is v3 with a 708-byte
policy block inserted between the metadata and the value.
u32 magic = 0xF5B40004
u32 object_count | u32 next_handle
object_count x records:
... v3 fixed fields, unchanged, 204 bytes ...
u8 allowed_count ; u32 allowed[8] (33)
u8 wrap_count ; entry wrap[4] (225)
u8 unwrap_count ; entry unwrap[4] (225)
u8 derive_count ; entry derive[4] (225)
u8 value[value_len]
entry (56 bytes, 8-aligned):
u32 type the CKA_ attribute constrained
u8 kind 1 bool, 2 ulong, 3 bytes
u8 len significant bytes of value
u8 pad[2]
u8 value[48] bool in value[0]; ulong as u64 LE; bytes over len
FHSM_OBJ_REC_V4_FIXED = 204 + 708 = 912, pinned by a _Static_assert.
The writer places the value at that offset and the reader reads it from there;
an edit to a cap that moved one without the other would read a key as policy.
Properties:
CKA_ALLOWED_MECHANISMS with one mechanism at all
seventeen literal sites, and the templates with one or two entries. Eight and
four are above anything observed. But the entries are CKA_LABEL — 33 to 41
octets — and CKA_KEY_TYPE, a CK_ULONG; not one boolean. The first draft of
this record held (attribute, boolean) pairs, which would have grown every
record by 228 bytes and still refused every nested-template test in the
corpus. The cap was never the risk; the value type was.kind is recorded, not derived. A one-byte CK_BBOOL and the first byte
of a one-character label are the same octet, and deriving the shape from the
attribute number would make the store’s reading of a stored value depend on a
table that changes over releases. The PKCS#11 layer still holds the caller’s
CK_ATTRIBUTE when it stores the entry, so it says which shape it was handed
and the store never guesses.CKA_ALLOWED_MECHANISMS
permits every mechanism; one set to the empty list permits none, and the
corpus sends that case deliberately. Both store zero. Presence is therefore
four bits in flags2 (FHSM_OBJF2_HAS_*, 0x08–0x40), whose polarity is
negative — set means restricted — so a record written before those bits
existed reads as no policy, which is what it means. Same problem
start_date_len answered, same answer.flags2 bit, which a reader can safely ignore, a count it
cannot store is a policy it would silently drop — and a dropped policy reads
as permission. A build wanting nine mechanisms or five entries must emit v5,
not more entries in a v4 record.HAS_* bit, which is what they mean. tests/fixtures/token-v3.tok
is a real file written by the pre-v4 build and kept as a fixture;
tests/test_v3_fixture.c loads it. A synthesised v3 blob would have been
built from the same beliefs the reader holds and would have agreed with
itself.FHSM_OBJ_BLOB_MAX is now derived from
FHSM_OBJ_REC_V4_FIXED. Forgetting to move it is what broke loading past 11
objects in v1.4.0; the same is true of write_atomic’s pt_sz, which sizes
the plaintext buffer and would have overflowed it on the first write.CKA_UNWRAP_TEMPLATE support coveredfhsm_parse_unwrap_template accepted CKA_SENSITIVE=TRUE and
CKA_EXTRACTABLE=FALSE and refused the rest, described in its own comment as
deliberately partial. Its first test on each nested entry was
ulValueLen != 1, and every entry anything sends is a CK_ULONG or a 33-to-41
byte CKA_LABEL. The two attributes it supported were the only two nobody
sends, so the overlap with what is tested was zero — since #125, invisibly,
because a skip for “not supported” and a skip for “not seen” are the same line
in a report.
Storage is half of it. CKA_ALLOWED_MECHANISMS is consulted at ten sites (the
four C_*Init, C_DeriveKey, C_WrapKey, C_UnwrapKey, C_EncapsulateKey,
C_DecapsulateKey, C_DigestKey). CKA_UNWRAP_TEMPLATE and
CKA_DERIVE_TEMPLATE are compared against the creation template in
C_UnwrapKey and C_DeriveKey. CKA_WRAP_TEMPLATE is the odd one: C_WrapKey
names a key that already exists, so it is compared against that object’s
attributes, which is why it can only name attributes the module can read back
and why a template naming anything else is refused at creation.
Offset 118 of the record, the first of the two pad bytes after
usage_flags, which every writer since v2 has emitted as zero and no reader
has looked at. v4 now writes and reads it as flags3 (FHSM_OBJF3_* in
fhsm_token.h); 119 stays pad. The record size and the magic do not change.
One bit is defined. FHSM_OBJF3_ED_CURVE_NAME (0x01) records that an Edwards
key was given CKA_EC_PARAMS as the RFC 8032 curveName PrintableString
(edwards25519, edwards448) rather than the RFC 8410 OID. PKCS#11 v3.2
tells the two EdDSA profiles apart by that encoding: on an edwards448 key
CKM_EDDSA requires its parameter block, on an id-Ed448 key a NULL
parameter is pure Ed448. The key material is the same either way, so only
the store can remember which was asked for.
Why this is not a v5:
edwards448 key opened by such a build accepts a NULL parameter again and
reads back its CKA_EC_PARAMS as the OID, which is that build’s own
behaviour, not a new one. No restriction or policy rides on the bit, so
there is nothing a downgrade could silently lift. That is the line a future
bit in this byte has to stay on; one that restricts belongs in a v5, where
an old reader fails loudly.tests/test_eddsa_params.c (8) generates a token edwards448 pair and
checks, after C_Finalize and C_Initialize, that the parameter is still
required and CKA_EC_PARAMS still reads back as the string.
Header Version (offset 4) is bumped on any header layout change;
the objects blob is self-versioned by its leading magic. Loaders must
reject unknown versions/magics rather than guess. Future extensions are additive and documented here. The v3 record chose a
fixed layout over a CBOR/TLV extras blob deliberately (see above): with no
installed base, bumping the magic again is cheaper than carrying another
hand-written parser.
v4 is that bump, and it is the evidence for the reasoning rather than a repetition of it. The extension cost 708 fixed bytes per record and no new parser shape — the same field-at-offset reads v3 already used, with four more bounded counts. What it did cost was a measurement: the first draft sized the entries for booleans because that is what the attributes looked like from the spec, and what the corpus sends is labels. A TLV blob would have absorbed that mistake silently and carried it.
Every mutation rewrites the whole file. write_atomic builds the 317-byte
header, serialises all objects, encrypts the complete blob under
AES-256-GCM, writes it to <path>.tmp, fsyncs, and renames. There is no
incremental path: fourteen call sites reach it, and adding one key costs the
same as rewriting every key.
That sounds like the thing to fix. Measured, it mostly is not.
tests/bench_capacity, RSA-2048-sized values (1 191 bytes), ext4, this
sandbox:
| objects | total | average per key |
|---|---|---|
| 64 | 255 ms | 3.99 ms |
| 128 | 525 ms | 4.10 ms |
| 256 | 1 059 ms | 4.14 ms |
| 512 | 2 350 ms | 4.59 ms |
| 1 024 | 5 468 ms | 5.34 ms |
Fitting T(N) = aN + bN² over those five points lands within ±2.3 % at every
one of them:
So the last key at the 1 024 ceiling costs about 6.9 ms, and filling an empty token takes about 5.5 s.
The fixed cost is durability, not the rewrite. A separate probe —
open/write/fsync/rename in a loop, same directory — gives 2.94 ms with
fsync and 0.26 ms without, for a 317-byte file. On tmpfs the same loop is
0.011 ms. Roughly 2.7 of the 3.82 ms is the disk barrier: the price of not
losing a freshly generated key to a power cut, and not removable while that
guarantee stands.
Which redirects the fix. Making the format incremental would remove the bN²
term and nothing else — about 1.5 ms out of 6.9 ms at the ceiling, and
essentially nothing on a token holding a few dozen keys. The lever that
matters is batching: one fsync per group of mutations rather than per
mutation, with the durability boundary stated so callers know what they are
trading. That is a different change, in a different place, and cheaper.
Noted because it was carried in conversation for weeks as “7.25 ms per key creation, whole-store rewrite per mutation” and never written down. Two things were wrong with that. The number was a marginal cost near the ceiling on other hardware, not an average — quoting it for a token with fifty keys would overstate by 70 %. And the cause was misattributed: the rewrite is real, but at today’s ceiling it is the smaller half.
FHSM_MAX_OBJECTS (default 1024) is overridable via -DFHSM_MAX_OBJECTS=N
for general-purpose deployments needing a larger per-token store. It
bounds both the in-memory object array and the on-disk blob
(FHSM_OBJ_BLOB_MAX scales with it). A token that reaches the limit
returns CKR_DEVICE_MEMORY on the next C_CreateObject (#125 I1).