AnnouncementsIntroducing Arkoren

    Menu

    Storing Data

    Persist encrypted records with explicit schema, identity binding, encoding, and guarded re-encryption.

    What's on this page
    1. Store binary ciphertext through typed writes while keeping plaintext out of database records and logs.
    2. Bind credentials to their expected record and handle text encoding and equality-lookup trade-offs.
    3. Replace old ciphertext with concurrency guards and preserve key, context, and schema compatibility.
    On this page

    Encrypting a credential protects its contents, but your application still owns the row that contains it, the metadata used to interpret it, and the rules for replacing it. Decide those boundaries before writing ciphertext so future deployments and backups remain readable.

    Choosing Storage

    Prefer a binary column for ciphertext: PostgreSQL BYTEA, MySQL VARBINARY or BLOB, or SQLite BLOB. Scan it into []byte. Both shipped drivers return a 12-byte nonce followed by encrypted bytes and a 16-byte tag, adding 28 bytes to the raw plaintext length. Encrypt adds JSON encoding first, so field names and JSON escaping also affect size. Size a bounded column for the largest encoded payload or choose an appropriate unbounded column.

    Encrypt only fields that need recovery. Leaving identifiers and routing fields outside ciphertext permits normal queries, but exposes those fields to anyone who can read storage. Encrypting a whole payload hides more content while removing ordinary filtering and indexing over its contents. Ciphertext still reveals its length and does not replace database access controls or encrypted transport.

    Writing Credentials

    Assume enc is your configured *crypto.Encrypter, db is your configured *database.Database, and ctx is the operation context. Imports are arkoren.dev/core/service/crypto, arkoren.dev/core/service/database, and arkoren.dev/foundation/query. The application owns a calendar_connections table with id, account_id, and binary credentials columns. The connection and account identifiers come from an authorized application operation, not arbitrary writable request fields.

    Use a named payload that keeps record identity with the credential. Here credentials is a validated CalendarCredentials value from Overview, and connectionID and accountID identify the authorized target:

    type StoredCalendar struct {
    	Version      int                 `json:"version"`
    	ConnectionID int64               `json:"connection_id"`
    	AccountID    int64               `json:"account_id"`
    	Credentials  CalendarCredentials `json:"credentials"`
    }

    Encrypt that payload before building the database update:

    sealed, err := enc.Encrypt(StoredCalendar{
    	Version:      1,
    	ConnectionID: connectionID,
    	AccountID:    accountID,
    	Credentials:  credentials,
    })
    if err != nil {
    	return err
    }

    Do not write a result from a failed encryption. The identity fields and version are encrypted and authenticated together with the credentials. For the database write, expose only the intended ciphertext column through a separate typed input:

    type CredentialUpdate struct {
    	Credentials []byte `db:"credentials"`
    }
    
    input := CredentialUpdate{Credentials: sealed}
    statement := query.Table("calendar_connections", func(statement *query.Builder) {
    	statement.Where("id", connectionID)
    	statement.Where("account_id", accountID)
    	statement.Update(query.Values(input))
    })
    affected, err := db.Exec(ctx, statement)

    Check the write error and affected-row count before reporting success. No matching row is not a successful credential replacement. The database db tags control column mapping; the payload’s json tags control the encrypted JSON. Do not persist the plaintext alongside the ciphertext or include it in query logs.

    Binding Records

    A purpose string such as calendar-credentials:v1 in driver AAD prevents using that ciphertext under a different purpose. It does not stop copying ciphertext between two calendar connections that share the same driver context. AAD is fixed at driver construction; the facade has no per-operation AAD argument.

    To reject record substitution with the payload above, compare its authenticated identity against the independently established account and requested connection. With standard-library errors imported, check both decryption and the embedded metadata before using a credential:

    payload, err := enc.Decrypt[StoredCalendar](stored)
    if err != nil {
    	return err
    }
    if payload.Version != 1 {
    	return errors.New("unsupported calendar payload version")
    }
    if payload.ConnectionID != connectionID || payload.AccountID != accountID {
    	return errors.New("calendar payload belongs to another connection")
    }

    Here stored is the selected row’s ciphertext. Do not take the expected identity from the decrypted payload itself or trust an attacker-supplied account ID. Cleartext columns and external version headers are not automatically authenticated by encryption. Put security-relevant metadata inside the encrypted payload or bind it through the exact configured AAD, then verify its meaning. Neither approach replaces access authorization or prevents replay of an older valid payload for the same record.

    Encoding Text

    Where a transport accepts only text, encode the complete ciphertext. With standard-library encoding/base64, URL-safe unpadded encoding is suitable for an application token:

    token := base64.RawURLEncoding.EncodeToString(sealed)

    Encoding adds roughly one third to the length; it is not another encryption layer. Keep cookie payloads small enough for browser limits and account for headers and other cookie fields. Avoid putting credentials in URLs, where history, referrers, and access logs can expose replayable values.

    Decode using the matching encoding before decrypting:

    stored, err := base64.RawURLEncoding.DecodeString(token)
    if err != nil {
    	return err
    }

    A base64 error is an encoding failure, not crypto.ErrInvalidCiphertext. Successful decoding says nothing about authenticity; decryption must still succeed. Apply transport size limits before allocating or decoding untrusted input. Never use base64 to justify logging a key, plaintext, or ciphertext token.

    Searching Values

    Random nonces mean encrypting a search term again does not reproduce stored ciphertext. Query by an ordinary identifier whenever possible. If equality lookup over encrypted data is necessary, an application-owned indexed keyed hash can accompany the ciphertext, but reveals which rows share a value.

    For an application that deliberately treats contact addresses case-insensitively, this helper uses standard-library crypto/hmac, crypto/sha256, and strings:

    func contactLookup(secret []byte, email string) []byte {
    	normalized := strings.ToLower(strings.TrimSpace(email))
    	mac := hmac.New(sha256.New, secret)
    	mac.Write([]byte(normalized))
    
    	return mac.Sum(nil)
    }

    Choose normalization according to your account policy and apply it identically on writes and searches. Use an independent protected lookup key, not the encryption key. Store the result in an indexed binary column such as contacts.email_lookup, updating it consistently with the encrypted address. A plain unkeyed hash permits guessing common addresses from a database copy.

    With validated email, protected bytes lookupKey, and your application row type Contact, restrict the lookup to the authorized account:

    contact, err := db.Find[Contact](ctx, query.Table("contacts", func(statement *query.Builder) {
    	statement.Where("account_id", accountID)
    	statement.Where("email_lookup", contactLookup(lookupKey, email))
    }))

    The hash supports equality, not arbitrary encrypted search. Add it only when the lookup is needed, and plan a separate index migration if its key or normalization changes. Arkoren does not generate or rotate this index for you.

    Re-encrypting Values

    Key rotation only changes future encryption. To replace old stored ciphertext, configure current and previous drivers and ask whether a previous driver authenticated the value. For an authorized loaded row connection, read its Credentials []byte field:

    payload, previous, err := enc.DecryptWithRotation[StoredCalendar](connection.Credentials)
    if err != nil {
    	return err
    }

    Check err first: previous can be true even if later JSON decoding fails. Apply the version and identity checks from Binding Records before using or rewriting payload. If previous is false, no key replacement is needed. Otherwise encrypt the verified payload with the current driver:

    fresh, err := enc.Encrypt(payload)
    if err != nil {
    	return err
    }

    Do not overwrite a credential changed by another request. Use the original ciphertext as a compare-and-swap condition alongside the authorized target. The update reuses CredentialUpdate from the write example:

    input := CredentialUpdate{Credentials: fresh}
    affected, err := db.Exec(ctx, query.Table("calendar_connections", func(statement *query.Builder) {
    	statement.Where("id", connection.ID)
    	statement.Where("account_id", accountID)
    	statement.Where("credentials", connection.Credentials)
    	statement.Update(query.Values(input))
    }))
    if err != nil {
    	return err
    }
    if affected == 0 {
    	return errors.New("calendar credentials changed; reload before rotating")
    }

    These replacement steps run only for a successfully decoded, identity-checked payload with previous == true. A failed write leaves rotation incomplete. Alternatively, lock the row within your transaction before reading and replacing it.

    Read-time replacement gradually migrates active rows. To finish, run a bounded application-owned batch job over every relevant row, handle conflicts by reloading, and record completion without logging payloads. Already-migrated rows report previous == false, so the key-replacement check is repeatable. Do not retire old keys until no live writer or stored value needs them, and retain keys required by backups.

    For opaque byte payloads, DecryptRawWithRotation and EncryptRaw preserve the byte format without JSON decoding. Application format validation still belongs before any replacement.

    Keeping Compatibility

    Persist complete ciphertext bytes; do not split or reinterpret the nonce and tag in application code. The raw format carries no cipher, key identifier, AAD, or schema version. Keep that configuration recoverable for old values and backups. Changing AAD requires the original context on the old driver and the new context on the new driver; changing a purpose string without retaining the old decoder makes existing values fail authentication.

    The version inside StoredCalendar is an application field, not an Arkoren version protocol. Reject unsupported versions and preserve the original bytes. If incompatible payload shapes need separate decoders, define an explicit application format and validate its discriminator rather than guessing from decryption failures. Unknown external discriminators must not select arbitrary keys or cause plaintext fallback.

    A successful JSON decode does not prove that a schema change preserved meaning: renamed or absent fields can still produce zero values. Validate required fields and version semantics before use or re-encryption. Key replacement and schema migration are separate decisions; verify both before rewriting durable data.

    For typed and raw rotation operations, see the crypto API reference.

    Open a bulk documentation archive?

    This file contains every guide in its scope, not API declarations. Loading it all into an agent can exhaust its context window and increase processing time and cost. For coding tasks, start with the index and read only the relevant pages.

    Use the full archive for offline indexing, bulk processing, or deliberate large-context use.

    Browse the index instead
    Open archive