Skip to content
BranchesTags

Repository files navigation

lib-commons

lib-commons is Lerian's shared Go toolkit for service primitives, connectors, HTTP/server utilities, security, resilience, tenant-manager primitives, outbox, DLQ, certificate, JWT, and transaction helpers.

The current API surface is published on the v7 line. This split-library line intentionally extracts observability/logging/runtime instrumentation to lib-observability, runtime configuration to lib-systemplane, and CloudEvents/Kafka streaming to lib-streaming.

Migrating from older packages?
Use the library boundary table below as the canonical direction for renamed, redesigned, removed, or extracted APIs in the split-library lib-commons line. Observability, logging, runtime, and assertion APIs are no longer exposed from lib-commons; import the owning library directly.

Requirements

  • Go 1.26.3 or newer

Installation

go get github.com/LerianStudio/lib-commons/v7

Lerian Library Boundaries

Lerian's shared platform code is split across four libraries:

Library Ownership
github.com/LerianStudio/lib-commons Core helpers, connectors, HTTP/server utilities, security, resilience, tenant-manager primitives, outbox, DLQ, certificate, JWT, transaction helpers
github.com/LerianStudio/lib-observability Logging, zap adapter, tracing, metrics, redaction, panic instrumentation, assertions, observability constants
github.com/LerianStudio/lib-systemplane Runtime configuration, hot reload, systemplane admin routes, tenant-scoped runtime knobs, systemplane contract tests
github.com/LerianStudio/lib-streaming CloudEvents/Kafka streaming, event emitters, streaming DLQs, outbox replay for streaming events

What is in this library

Core (commons)

  • app.go: Launcher for concurrent app lifecycle management with NewLauncher(opts...) and RunApp options
  • context.go: request-scoped logger/tracer/metrics/header-id tracking via ContextWith* helpers, safe timeout with WithTimeoutSafe, span attribute propagation
  • errors.go: standardized business error mapping with ValidateBusinessError
  • utils.go: UUID generation (GenerateUUIDv7 returns error), struct-to-JSON, map merging, CPU/memory metrics, internal service detection
  • stringUtils.go: accent removal, case conversion, UUID placeholder replacement, lowercase hexadecimal SHA-256 hashing for strings (HashSHA256) and byte slices (HashSHA256Bytes), server address validation
  • time.go: date/time validation, range checking, parsing with end-of-day support
  • os.go: environment variable helpers (GetenvOrDefault, GetenvBoolOrDefault, GetenvIntOrDefault, GetenvDurationOrDefault), struct population from env tags via SetConfigFromEnvVars
  • commons/constants: shared constants for datasource status, errors, headers, metadata, pagination, transactions, and obfuscation values

Observability and logging

Observability has moved to github.com/LerianStudio/lib-observability. Use that library directly for logging, zap adapters, tracing, metrics, redaction, panic instrumentation, assertions, and observability constants.

The former commons/opentelemetry, commons/opentelemetry/metrics, commons/opentelemetry/constants, commons/opentelemetry/redaction, commons/log, commons/zap, commons/runtime, and commons/assert packages are not part of lib-commons. Consumers must import github.com/LerianStudio/lib-observability/{log,zap,assert,runtime,tracing,metrics,constants,redaction} directly.

Data and messaging connectors

  • commons/postgres: Config-based constructor (New), Resolver(ctx) for dbresolver access, Primary() for raw *sql.DB, NewMigrator for schema migrations, backoff-based lazy-connect; dual-driver SQLSTATE error classification that unwraps both pgx (*pgconn.PgError) and lib/pq (*pq.Error) through wrapped chains via errors.As — accessors SQLState(err) (string, bool) / Constraint(err) (string, bool) / DriverMessage(err) (string, bool) and predicates IsUniqueViolation (23505) / IsForeignKeyViolation (23503) / IsCheckViolation (23514) / IsUndefinedTable (42P01); all nil-safe (nil or non-driver errors classify false / report absent); bounded snapshot reads via Client.RunReadOnly(ctx, opts, fn) — opens REPEATABLE READ READ ONLY so every statement in fn sees ONE snapshot, prefers the configured replica and falls back to the primary, applies SET LOCAL statement_timeout from ReadOnlyOptions.StatementTimeout (REQUIRED and positive, else ErrReadOnlyStatementTimeoutRequired before the transaction opens, because zero is how PostgreSQL spells "no timeout"), bounds the whole transaction with ReadOnlyOptions.TransactionTimeout (zero inherits the caller's deadline), and ALWAYS rolls back; the package-level RunReadOnly(ctx, db, opts, fn) takes any TxBeginner (*sql.DB, *sql.Conn) for callers holding their own pool; the two ways a read runs out of time are distinguishable with errors.Is on ErrReadOnlyStatementTimeout (one slow plan — wants an index) and ErrReadOnlyTxDeadline (the whole read — wants fewer statements), plus ErrNilReadOnlyFunc and ErrNilClient (the package-level helper returns it for a nil TxBeginner, including a typed-nil *sql.DB inside a non-nil interface); a caller CANCEL is reported as neither timeout — the cause is returned as it stands, so errors.Is(err, context.Canceled) holds
  • commons/mongo: Config-based client with functional options (NewClient), URI builder (BuildURI), Client(ctx)/ResolveClient(ctx) for access, EnsureIndexes (variadic), TLS support, credential clearing
  • commons/redis: topology-based Config (standalone/sentinel/cluster), GCP IAM auth with token refresh, distributed locking via LockManager interface (NewRedisLockManager, LockHandle) — WithLockOptions reports a lock held elsewhere at debug with a clean span and an error matching ErrLockContended — including when the caller's own deadline expires while it waits — reports a caller whose context ended at debug with a clean span and an error unwrapping to that context error, and reserves error logs and error spans for a Redis that is unreachable or stopped answering, whether or not the caller carried a deadline; (*RedisLockManager).TryLockWithOptions is TryLock with the caller's expiry and tries instead of a fixed 10s single attempt, and is on the concrete type only so the shipped interface stays stable; the handles it and TryLock return also implement the optional LockExtender interface (Extend(ctx) (bool, error), discovered by type assertion so external LockHandle implementations are unaffected), which renews the lease to the expiry it was acquired with — (true, nil) renewed, (false, nil) lost because the key expired or another holder took it (WARN, clean span), an error unwrapping to the caller's context error when its context had already ended, and an error with an error span when Redis did not answer or when a quorum accepted the renewal but the round trip outlived the lease (unwraps to redsync.ErrExtendFailed; the lease may still be ours) — SetPackageLogger for diagnostics, pool controls including ConnectionOptions.MaxActiveConns, TLS defaults to a TLS1.2 minimum floor with AllowLegacyMinVersion as an explicit temporary compatibility override, and TLS without a custom CA uses the host system trust store
  • commons/rabbitmq: connection/channel/health helpers for AMQP with *Context() variants, HealthCheck() (bool, error), Close()/CloseContext(), OpenChannel()/OpenChannelContext() for a caller-owned dedicated channel on the managed connection, confirmable publisher with broker acks and auto-recovery, DLQ topology utilities, and health-check hardening (AllowInsecureHealthCheck, HealthCheckAllowedHosts, RequireHealthCheckAllowedHosts)
  • commons/dlq: Redis-backed dead letter queue with New(conn, keyPrefix, maxRetries, opts...) returning nil when conn is nil (all methods guard nil receiver via ErrNilHandler); key operations: Enqueue (RPush, stamps CreatedAt/MaxRetries on first enqueue), Dequeue (LPop, at-most-once), QueueLength, ScanQueues (non-blocking SCAN for background consumers without tenant context), PruneExhaustedMessages (dequeue-discard-reenqueue cycle up to limit), ExtractTenantFromKey; tenant-scoped Redis keys ("<prefix><tenantID>:<source>"), backoff via exponential-with-jitter (base 30s, floor 5s, AWS Full Jitter); functional options WithLogger/WithTracer/WithMetrics/WithModule; DLQMetrics interface (RecordRetried/RecordExhausted, nil-safe); NewConsumer(handler, retryFn, opts...) (*Consumer, error) for background poll loop — Run(ctx) blocks until stop, Stop() idempotent, ProcessOnce(ctx) exported for tests; consumer options WithConsumerLogger/WithConsumerTracer/WithConsumerMetrics/WithConsumerModule/WithPollInterval/WithBatchSize/WithSources; sentinel errors ErrNilHandler, ErrNilRetryFunc, ErrMessageExhausted
  • Streaming has moved to github.com/LerianStudio/lib-streaming; runtime configuration has moved to github.com/LerianStudio/lib-systemplane.

HTTP and server utilities

  • commons/net/http: Fiber HTTP helpers -- response (Respond/RespondStatus/RespondError/RespondErrorEnvelope/RenderError; RespondErrorEnvelope preserves a caller-supplied status code and machine-readable error envelope), health (Ping/HealthWithDependencies), SSRF-protected reverse proxy (ServeReverseProxy with ReverseProxyPolicy), pagination (offset/opaque cursor/timestamp cursor/sort cursor), validation (ParseBodyAndValidate/ValidateStruct/ValidateSortDirection/ValidateLimit), context/ownership (ParseAndVerifyTenantScopedID/ParseAndVerifyResourceScopedID), middleware (WithHTTPLogging/WithGrpcLogging/WithCORS/WithBasicAuth/NewTelemetryMiddleware), FiberErrorHandler
  • commons/net/http/ratelimit: Redis-backed distributed rate limiting middleware for Fiber — New(conn, opts...) returns a *RateLimiter (nil when disabled, nil-safe for pass-through), WithDefaultRateLimit(conn, opts...) as a one-liner that wires New + DefaultTier into a ready-to-use fiber.Handler, fixed-window counter via atomic Lua script (INCR + PEXPIRE), RedisStorage.Increment(ctx,key,window) as the storage-only atomic primitive, WithRateLimit(tier) for static tiers, WithDynamicRateLimit(TierFunc) for per-request tier selection, MethodTierSelector for write-vs-read split, preset tiers (DefaultTier / AggressiveTier / RelaxedTier) configurable via env vars, identity extractors (IdentityFromIP / IdentityFromHeader / IdentityFromIPAndHeader — uses # separator to avoid conflict with IPv6 colons), fail-open/fail-closed policy, WithOnLimited callback, WithExceededHandler for caller-controlled 429 response bodies after standard rate-limit headers are set, and standard X-RateLimit-* / Retry-After headers; also exports RedisStorage (NewRedisStorage) for use with third-party Fiber middleware
  • commons/net/http/idempotency: atomic at-most-once request middleware for Fiber — the shipped New(conn, opts...) *Middleware go-redis API remains fail-open by default and returns nil for a nil connection; NewWithStore(store, opts...) *Middleware accepts a backend-neutral Store and always fails closed on a missing/errored backend; NewRedisStore(conn) exposes the built-in Redis adapter for store-contract composition; Store preserves middleware-owned opaque bytes through only atomic Acquire, compare-safe Complete, and compare-safe Release, with reusable adapter contract tests in idempotency/idempotencytest.Run; applies only to mutating methods (POST/PUT/PATCH/DELETE), passes GET/HEAD/OPTIONS unconditionally; reads X-Idempotency, or the value returned by an opt-in WithKeyProvider — a request-scoped key source (the authenticated principal, a channel, a device) whose value drives BOTH the storage key and the fingerprint, while the middleware never writes the request header, so a handler binding the caller's raw key still reads exactly what the caller sent; a provider error is refused with the pre-handler 503 IDEMPOTENCY_UNAVAILABLE and an empty provider return is the unkeyed branch — (missing key passes through by default, or is refused with 400 IDEMPOTENCY_KEY_REQUIRED under opt-in WithRequireKey; the tenant comes from the tenant-manager context, or ONLY from an opt-in WithTenantProvider(func(fiber.Ctx) (string, error)), which lets a service supply the tenant without overwriting the request context every reader below shares — the record is rooted at exactly the string returned, so the consumer owns canonicalisation, and an empty return or a provider error is the absent tenant; a KEYED request with no tenant likewise bypasses by default, or is refused with 400 IDEMPOTENCY_TENANT_REQUIRED under opt-in WithRequireTenant — the checks are ordered, so WithRequireTenant alone never sees an unkeyed request and both options are needed to refuse every unprotected mutation); key length defaults to 256 UTF-8 bytes; duplicate outcomes are matching exact response replay (status, content type, body, and multi-value headers) with X-Idempotency-Replayed: true, matching in-flight → 409 IDEMPOTENCY_CONFLICT plus Retry-After: 1, and different method/path/body or an opt-in application scope from WithFingerprintScopeProvider → 422 IDEMPOTENCY_KEY_REUSE, plus a terminal → 422 IDEMPOTENCY_OUTCOME_UNRECORDED for a key spent by a request that left no recorded outcome (answered by WithPostHandlerUnavailableHandler when set, and never with a replayed body, since none was stored), and a completed operation whose response exceeded WithMaxBodyCache and was therefore never stored → 409 IDEMPOTENCY_REPLAY_UNAVAILABLE, answered by WithReplayUnavailableHandler alone and never by the post-handler or terminal-refusal seams, with no Retry-After and no replay header, because it reports a KNOWN outcome whose receipt is simply missing rather than an outcome in doubt; a replay is the HANDLER's contribution and only that — the middleware snapshots the response headers immediately before the handler runs and captures the names it added or overwrote (Set-Cookie by cookie name), so Location, ETag, a Cache-Control the handler overrode and a cookie the handler minted replay byte-identical, each captured name REPLACED rather than appended so nothing arrives twice, while every header a middleware above set on THIS request — a correlation id, CORS, helmet, a rotated session, a freshly minted CSRF token — stays live on the duplicate instead of being overwritten by the original request's value; the fingerprint covers method, path and the raw body, or, through opt-in WithFingerprintProvider, method, path and bytes the application declares to BE the request's identity, so a route served with Fiber's StreamRequestBody is never drained to be fingerprinted and a byte-identical multipart retry is no longer refused for the fresh boundary its encoder picked (a provider error is the pre-handler 503 IDEMPOTENCY_UNAVAILABLE, since a request whose identity cannot be established must not run unprotected; answers to a streamed request whose body is still unread (any chunked body, or a declared length past what fasthttp pre-read) that the middleware refuses or replays close the connection, because the handler never ran and nobody drained the upload, so the next request on that keep-alive connection would be parsed from the middle of this one's body — a pooled client dials again and loses no request; a declared body within the min(BodyLimit, Content-Length, 8 KiB) fasthttp lifts out of the connection before the chain starts is already drained and keeps its connection, so the app-wide StreamRequestBody does not charge every ordinary duplicate a handshake; and turning the option on CHANGES the digest of the same logical request, so a retry straddling the rollout is refused IDEMPOTENCY_KEY_REUSE for one retention window — shorten the route's retention below the rollout or accept it); scoped fingerprints use a versioned domain plus the scope byte length and bytes, while omitting the provider preserves legacy fingerprint vectors exactly; an exact response that cannot be encoded, persisted, or decoded fails closed with 503 IDEMPOTENCY_UNAVAILABLE instead of fabricating success — a SIZE is not one of those faults, and an over-cap response takes the branch described further down instead of this one; the POST-HANDLER half of those — marshalling, WithResponseCodec encoding (including a codec that produces no bytes at all), a Store.Complete error, or a stale owner — additionally fences the key in a terminal outcome-unknown state held for the RETENTION TTL rather than leaving it to lapse with the in-flight lease, so a resend inside the retry window can no longer execute the mutation a second time, whereas a DUPLICATE whose stored response cannot be decoded or replayed fences nothing and leaves the existing completed record exactly as it is, since that record is already under the key and nothing new needs writing (best-effort by construction: the fence writes through Store.Complete to the store that just failed, closing a transient failure and not a total outage); the terminal mark is a FIELD on the record, not a third state value, so a mixed-version fleet sharing one store is safe in both directions and needs no upgrade ordering — a middleware that predates the field decodes the fenced record as a completed record with no replay response and refuses it through WithPostHandlerUnavailableHandler, where a third state value would have reached the unknown-state branch and, under the fail-open default, re-executed the mutation; the cost is that the state field of a fenced record is a compatibility encoding rather than the plain truth, so anything reading these records outside the package must read outcome to tell a real completion from a fence; an EXISTING record this version cannot interpret is refused REGARDLESS of WithFailClosed — 422 IDEMPOTENCY_STATE_UNRECOGNISED for an unrecognised state, 422 IDEMPOTENCY_RECORD_UNREADABLE for bytes that decode as neither the current nor the legacy format (separate codes because damaged bytes are not version skew) — the line drawn at whether the store handed back an existing value, not at whether this version can parse it, since Acquire reporting the key taken is proof it holds a live record, left untouched for a reader that understands it, and logged at ERROR naming the state — the same mixed-version problem pointed forward, since fail-open on a key that demonstrably holds someone's record is not running unprotected but running on top of an outcome sitting in the store (a store that actually errors, with no record to read, keeps its configured policy); a completion failure whose fence could NOT be persisted answers 503 IDEMPOTENCY_UNFENCED rather than the ordinary post-handler 503 and does not route through WithPostHandlerUnavailableHandler, so a service cannot mistake an unprotected key for a held one (a route whose handler already committed something irreversible, and which therefore owes its client that outcome rather than a failure the client will resend, wires WithUnfencedHandler — a seam for this case alone, never a fallback for the fenced one, and the header still reports false whatever it answers), and X-Idempotency-Fenced: true|false carries the same fact on every response where a fence was attempted — on the handler-failure branch under ServerErrorPolicyFence it is the SOLE carrier, because the application's error handler owns the body there, so a consumer rewriting that 5xx must read the header or it reproduces the indistinguishability one layer up; a failed fence logs at ERROR with the tenant, the acquisition owner and a SHA-256 digest of the store key (never the raw key, which is client-supplied); mid-rollout the same key answers 422 from an upgraded pod and 503 from one that is not — neither executes, and the split disappears for a service that wires WithPostHandlerUnavailableHandler, so wire it before rolling out if a uniform answer matters; successful responses and handler failure/5xx releases are owner-compared so an expired acquisition cannot overwrite or delete its replacement; a response whose body exceeds WithMaxBodyCache is delivered to its client UNCHANGED and completes its key marked as carrying no replayable receipt, so a resend is refused 409 IDEMPOTENCY_REPLAY_UNAVAILABLE and the handler never runs a second time; 4xx responses are cached by default and may instead release ownership via WithClientErrorPolicy(ClientErrorPolicyRelease), which is the ONLY thing that decides whether a rejection may re-execute and is applied before the response is captured, so an over-cap 4xx reaches its client unchanged and then follows that same policy rather than the size of its document — released under ClientErrorPolicyRelease, and under the default kept, with the resend refused 409 IDEMPOTENCY_REPLAY_UNAVAILABLE (a document that reports a known OUTCOME without claiming which way it went, since the same mark covers an over-cap success and an over-cap rejection) instead of re-running a rejection path the route asked to have cached, or decided per RESPONSE through WithClientErrorPolicyFunc(ClientErrorPolicyFunc), which is consulted only for a 4xx the chain WROTE and replaces the enum entirely when set, for a guard mounted above a rate limiter or a quota gate whose refusals are not the handler's answer and must not spend the caller's key — a 4xx RETURNED as an error instead (fiber.NewError(429, …), which lib-commons' own rate limiter produces when a WithExceededHandler returns one, while its built-in path writes the 429 and returns nil) has written no response and therefore takes the handler-failure branch, reaching the SERVER seam instead, so a route that wants one rule for both shapes installs both functions; handler failure/5xx releases the key by default and may instead fence it terminally via WithServerErrorPolicy(ServerErrorPolicyFence), for routes where a 5xx may still have committed — a handler cut off by its deadline answers 500 while it is still running, and releasing there frees the key before the application's own error handler has even seen the failure; the same question is decided per RESPONSE through WithServerErrorPolicyFunc(ServerErrorPolicyFunc), consulted for a handler error or a 5xx and replacing the enum entirely when set, for a route that knows which of its failures did not apply — err is what the handler returned and is nil when the handler only wrote the status, and is the only one of the two that is a fact; status is the EFFECTIVE status — what the handler wrote, or, when the handler returned an error having written nothing, the code inside that error when it is a *fiber.Error (so a returned fiber.NewError(429, …) arrives as 429 rather than as the untouched 200), which is a forecast of what the application's Fiber error handler will write rather than a status anything has written, since that handler has not run and may map, wrap or replace the code; a returned error that is not a *fiber.Error leaves status at the untouched 200, and a handler that wrote a status and ALSO returned an error reports the written status; request-specific retention is available through WithTTLProvider; the in-flight lease held from acquisition until the completed record is stored is sized separately from that retention through WithProcessingTTL (unset, it borrows the retention TTL, which is the shipped behaviour) — the lease must cover the complete handler-and-finalization interval, meaning the handler plus response capture, serialization, WithResponseCodec encoding and the Store.Complete round-trip, with margin, because nothing else holds the key through any of it; a lease that lapses anywhere in that span, including in the tail after the handler already returned and the mutation already committed, lets a redelivery under the same key execute the mutation a SECOND time, so a caller wanting a short replay window must not let that choice cap the work it protects, and WithProcessingTTLProvider resolves that lease per request the way WithTTLProvider resolves the retention, evaluated before each acquisition attempt and above the store deadline (so its own I/O is never charged to WithRedisTimeout), leaving a lease already in the store at the value it was taken with while a reloaded value lands on the next acquisition, and falling back on a provider error or non-positive value to exactly WithProcessingTTL — the retention when that is unset — rather than refusing the request the way an unresolvable retention does, which makes that constant an obligation: configure both and a short constant is what an unresolvable provider lands on, with the mid-flight lapse it implies; sensitive replay payloads can use authenticated encryption through WithResponseCodec; WithMaxBodyCache bounds the raw response and encoded output is bounded to twice that value; tenant-scoped keys remain byte-identical as "<prefix><tenantID>:<idempotencyKey>" whether fingerprint scoping is configured or not; rejection bodies remain customizable through WithRejectedHandler, WithUnavailableHandler, WithPostHandlerUnavailableHandler, WithConflictHandler, WithKeyReuseHandler, WithKeyRequiredHandler, WithTenantRequiredHandler, WithUnfencedHandler, WithReplayUnavailableHandler, and WithTerminalRefusalHandler — the two unavailability seams split one 503 into its two opposite instructions, a pre-handler failure where nothing ran and retrying is correct, and a post-handler one where the mutation is committed and the caller must reconcile instead of retrying under a new key (WithPostHandlerUnavailableHandler unset falls back to WithUnavailableHandler, so existing callers are unchanged), while WithTerminalRefusalHandler(func(c fiber.Ctx, code string) error) is the opt-in seam for the three 422 refusals a DUPLICATE receives before its handler runs when the key holds a record this version cannot act on — IDEMPOTENCY_STATE_UNRECOGNISED, IDEMPOTENCY_RECORD_UNREADABLE, IDEMPOTENCY_OUTCOME_UNRECORDED, exported as RefusalCodeStateUnrecognised, RefusalCodeRecordUnreadable and RefusalCodeOutcomeUnrecorded and passed as code so one handler tells the three apart — consulted FIRST in each and never reached by the post-handler receipt failure or by IDEMPOTENCY_UNFENCED, because those belong to a request whose own handler already committed, which is exactly why a service wanting its own envelope on these three cannot get it from WithPostHandlerUnavailableHandler without also rewriting the committed-mutation answer; unset, all three keep the existing routing and every shipped body is unchanged; the FOURTH refusal code, IDEMPOTENCY_REPLAY_UNAVAILABLE, is exported as RefusalCodeReplayUnavailable and sits deliberately outside that trio, because it reports a known outcome whose receipt was never stored rather than a record this version cannot act on, which is why WithReplayUnavailableHandler owns it alone
  • commons/net/http/pacing: Redis-backed distributed pacing for OUTBOUND calls — an http.RoundTripper, not inbound middleware, so redirects and client retries are each paced. NewPacer(conn, prefix, opts...) (*Pacer, error) owns the evaluation script and the retry timing; NewRoundTripper(next, pacer, BucketsFunc) (*RoundTripper, error) wires it into an http.Client (nil next falls back to http.DefaultTransport). Buckets are built by TenantBucket(id, RateProvider) — canonicalized through tenant-manager/core.CanonicalTenantID, so dashed and dashless UUID spellings collapse onto one budget, and slugs plus default are accepted — and InstitutionBucket(id, RateProvider), validated for identifier grammar only and namespaced separately. Pacer.Acquire(ctx, buckets...) charges every supplied bucket in ONE Lua evaluation or charges none, so a tenant permit is never burned while an institution bucket blocks; a refusal writes no bucket at all. RateProvider func(ctx) (float64, error) is read on every wait, so a rate changed at runtime applies without a restart; 0 pauses the bucket until it turns positive or the context ends, and a rate above WithMaxRate (default DefaultMaxRate = 1000/s) or below one call per day is refused. Emission spacing is GCRA with a burst of one — a bucket stores its LAST GRANT and the next admission is derived from the interval in force at evaluation time, so a raised rate shortens an in-flight wait and a lowered rate cannot be expired by a shorter key lifetime. Time comes from the Redis TIME command, never a local clock, and a per-prefix high-water mark refuses evaluation when the backend clock moves backwards. Keys are pacing:{<prefix>}:{tenant|inst}:<sha256-8-hex> — the brace is a Redis Cluster hash tag keeping one EVAL in one slot, and the identity is digested so no key, error, log field, or span attribute carries it. Everything fails closed: ErrPacerUnavailable, ErrInvalidPrefix, ErrInvalidIdentity, ErrNoBuckets, ErrDuplicateBucket, ErrInvalidRate, ErrInvalidPollInterval, ErrRateUnavailable, ErrBackendUnavailable, ErrClockWentBackwards, ErrWaitAborted (also wraps the context error). There is no fail-open mode and no burst option. Options: WithMaxRate, WithPollInterval (default DefaultPollInterval = 250ms, which also bounds how often rates are re-read), WithLogger
  • commons/net/http/signedcursor: opaque, HMAC-SHA256-signed keyset pagination cursors, for a multi-term ordering tuple over a tenant-scoped aggregate — where the unsigned Cursor/SortCursor/TimestampCursor in commons/net/http let the caller rewrite the tuple and steer the query. Fiber-free (pure crypto, so a worker or a gRPC service can mint one). New(key []byte) (*Codec, error) takes exactly KeySize (32) raw bytes; Encode(payload []byte, binding Binding) (string, error) signs the caller's already-serialized ordering tuple; Decode(token string, binding Binding) ([]byte, error) returns it. Binding ties a token to two OPAQUE term lists the package never interprets — an Identity (who read the page) and a Context (what it was read over) — each reduced to a keyed fingerprint, so neither travels in the token. THE RULE: the consumer never reads either FROM the cursor, it re-resolves both from its own trusted state (the validated JWT, the request) and only COMPARES; read out of the token, a stolen cursor would BE the authorization. Identity is mandatory (ErrEmptyIdentity, a 500 — the request never resolved who it was reading as), context may be empty. The MAC is verified before any byte of the body is parsed and a length guard runs before base64, so an unauthenticated token never reaches the decoder. Caller-fault rejections form a closed vocabulary wrapping one parent ErrInvalidCursor (typically 422): ErrMalformed, ErrSignature, ErrVersion, ErrIdentityMismatch, ErrContextMismatch; ErrInvalidKey, ErrPayloadTooLarge and ErrEmptyIdentity sit outside it as deployment and programming faults
  • commons/webhook: outbound webhook delivery with NewDeliverer(lister, opts...) *Deliverer returning nil when lister is nil (both Deliver/DeliverWithResults guard nil receiver); Deliver(ctx, *Event) error fans out to all active endpoints concurrently, returns errors only for pre-flight failures (nil receiver, nil event, listing failure) — per-endpoint failures are logged and metricked but do not propagate; DeliverWithResults(ctx, *Event) []DeliveryResult returns per-endpoint outcomes for callers needing individual results; SSRF protection via resolveAndValidateIP: single DNS lookup validates all resolved IPs against private/loopback/link-local/CGNAT/RFC-reserved ranges then pins URL to first resolved IP (eliminates DNS rebinding TOCTOU); WithAllowPrivateNetwork() only relaxes blocking for explicit private/loopback IP-literal URLs (for example 127.0.0.1, 10.0.0.5) when local/development tier allows it or ALLOW_WEBHOOK_PRIVATE_NETWORK supplies an explicit override reason; hostnames resolving to private IPs remain blocked; redirects blocked entirely to prevent 302-to-internal bypass; HMAC-SHA256 signing via X-Webhook-Signature: sha256=<hex> over raw payload (timestamp not included — replay protection is the receiver's responsibility); encrypted secrets via SecretDecryptor func (receives ciphertext with enc: prefix stripped, no decryptor + encrypted secret = fail-closed); retry with exponential backoff+jitter (base 1s), non-retryable on 4xx except 429; concurrency capped by semaphore (default 20); EndpointLister interface (ListActiveEndpoints), DeliveryMetrics interface (RecordDelivery); functional options WithLogger/WithTracer/WithMetrics/WithMaxConcurrency/WithMaxRetries/WithHTTPClient/WithSecretDecryptor/WithAllowPrivateNetwork; sentinel errors ErrNilDeliverer/ErrSSRFBlocked/ErrDeliveryFailed/ErrInvalidURL
  • commons/server: ServerManager-based graceful shutdown with WithHTTPServer for Fiber, WithStdlibHTTPServer for caller-owned *net/http.Server, WithStdlibHTTPListener for pre-bound stdlib listeners (stdlib HTTP variants are mutually exclusive with Fiber HTTP), WithAdditionalStdlibHTTPServer/WithAdditionalStdlibHTTPListener for a second stdlib server on its own port that composes with every other slot and drains concurrently with the main HTTP server under one shared shutdownTimeout (no ordering between the two, whichever main variant; gRPC and admin follow with their own budgets; every HTTP drain, Fiber included, is bounded by shutdownTimeout; past the deadline an active Fiber connection is abandoned, not closed, and can serve further keep-alive requests until the process exits, so shutdown hooks must tolerate a late request against closed resources) (address must differ from every other server, else ErrAdditionalHTTPAddressConflict; a nil Handler is refused with ErrAdditionalHTTPHandlerMissing instead of serving http.DefaultServeMux; the slot holds one server, a second non-nil one is refused with ErrAdditionalHTTPServerAlreadyConfigured, nil clears it), WithGRPCServer/WithShutdownChannel/WithShutdownTimeout/WithShutdownHook, StartWithGracefulShutdown()/StartWithGracefulShutdownWithError(), ServersStarted() for test coordination

Resilience and safety

  • commons/certificate: thread-safe TLS certificate manager with hot reload — NewManager(certPath, keyPath string) (*Manager, error) loads PEM files at construction; both paths empty returns unconfigured manager (TLS optional), exactly one path → ErrIncompleteConfig; key file must have mode 0600 or stricter (checked before reading); PKCS#8 → PKCS#1 (RSA) → EC (SEC 1) key parsing order; full PEM chain parsed (all CERTIFICATE blocks, leaf first then intermediates); Rotate(cert *x509.Certificate, key crypto.Signer) error atomically hot-reloads under write lock — validates NotBefore/NotAfter temporal bounds and public-key match (ErrKeyMismatch) before swapping; read accessors (all nil-safe, read-locked): GetCertificate()/GetSigner()/PublicKey()/ExpiresAt()/DaysUntilExpiry(); TLS integration: TLSCertificate() tls.Certificate builds populated struct with full chain; GetCertificateFunc() func(*tls.ClientHelloInfo) (*tls.Certificate, error) for assignment to tls.Config.GetCertificate for transparent hot-reload; package-level LoadFromFiles(certPath, keyPath string) (*x509.Certificate, crypto.Signer, error) for pre-flight validation without touching manager state; sentinel errors ErrNilManager/ErrCertRequired/ErrKeyRequired/ErrExpired/ErrNoPEMBlock/ErrKeyParseFailure/ErrNotSigner/ErrKeyMismatch/ErrIncompleteConfig
  • commons/circuitbreaker: Manager interface with error-returning constructors (NewManager), TenantAwareManager tenant/service overloads for isolated per-tenant breakers (tenant-aware methods require non-empty valid tenant IDs; legacy Manager methods are the no-tenant/process-wide path), NewPassthroughManager/NewPassthroughTenantAwareManager for feature-flagged bypass while preserving validation contracts, config validation, preset configs (DefaultConfig/AggressiveConfig/ConservativeConfig/HTTPServiceConfig/DatabaseConfig), health checker (NewHealthCheckerWithValidation), metrics via WithMetricsFactory using tenant_hash for tenant-aware breakers instead of raw tenant IDs while preserving the legacy no-tenant metric label set
  • commons/backoff: exponential backoff with jitter (ExponentialWithJitter) and context-aware sleep (WaitContext)
  • commons/errgroup: error-group concurrency with panic recovery (WithContext, Go, Wait), configurable logger via SetLogger
  • commons/safe: panic-safe math (Divide/DivideRound/Percentage on decimal.Decimal, DivideFloat64), regex with caching (Compile/MatchString/FindString), slices (First/Last/At with *OrDefault variants)
  • commons/security: sensitive field detection (IsSensitiveField), default field lists (DefaultSensitiveFields/DefaultSensitiveFieldsMap)
  • commons/security/sanitize: credential and PII redaction for anything that becomes a log line, a span attribute or a stored column — String(s) string strips URL userinfo (keeping scheme, host, port and database so the failure stays diagnosable, and anchored PER URL so a comma- or semicolon-separated broker list is redacted entry by entry), Authorization/Proxy-Authorization/Cookie/X-Api-Key header values (scheme kept only when it is a recognised one, since an unrecognised leading word is credential material; the value ends at ], }, ; or an escaping \ so the rest of a map dump or JSON object survives and stays parsable), sensitive query parameters such as ?apikey=/?sslpassword= (the key=value value class admits = and would otherwise swallow the whole URL as one non-sensitive pair), Azure SAS sig=, PEM blocks, sensitive key=value and "key":"value" pairs by field name — delegated to lib-observability/v4/redaction plus an addendum for AWS/SASL names and the Brazilian document and bank-account names that taxonomy never carried (cpf, cnpj, rg, conta, agencia, chave_pix, nome_titular, data_nascimento, ...), with every quote independently optional-escaped so a %q-quoted or nested JSON body is still matched — and bare values with no field name around them: AWS/GCP/GitHub/Stripe/Slack keys, JWTs, e-mail addresses, and card numbers (12-19 digits, grouped as printed on the card or unbroken, GATED ON LUHN so an order id or an epoch-millisecond timestamp of the same length survives). Error(err) error wraps an error with the redacted message while errors.Is/errors.As keep classifying the cause — there is deliberately NO Unwrap, because errors.Unwrap(sanitized).Error() would hand the raw DSN straight back, and the wrapper implements fmt.Stringer and fmt.Formatter itself so an errors.As naming either interface is assigned the WRAPPER rather than the cause; the redaction covers what the wrapper PRINTS, not what the chain CONTAINS — errors.As on a target that reaches the cause (interface{ Unwrap() error }, interface{ Unwrap() []error }, json.Marshaler, encoding.TextMarshaler, fmt.GoStringer, or the concrete driver type) hands back a value whose own printing is the raw text, and all of those stay open on purpose because closing the interface ones would take legitimate classification (interface{ SQLState() string }, Timeout(), Temporary()) down with them while errors.As on *pgconn.PgError reaches the text anyway. Ask, classify, and print only the wrapper. Nil-safe including a typed-nil cause. Replacement marker is SecretRedactionMarker (****). Input above MaxInputLen (64 KiB) is REFUSED with a marker sentence naming its size, never truncated: a cut landing before redaction strands a readable partial secret at the cut

Domain and support packages

  • commons/transaction: intent-based transaction planning (BuildIntentPlan), balance eligibility validation (ValidateBalanceEligibility), posting flow (ApplyPosting), operation resolution (ResolveOperation), typed domain errors (NewDomainError)
  • commons/outbox: transactional outbox contracts, dispatcher, sanitizer, and tenant-aware persistence adapters. PostgreSQL supports pool-per-tenant, schema-per-tenant, and column-per-tenant. A pool-per-tenant service with generic and module databases wraps its existing resolvers with postgres.NewModulePoolResolver(genericResolver, defaultTenantID, loadConfig, postgres.ModulePool{Name: "consignado", Resolver: consignadoResolver}), then passes the result as MultiTenantConfig.PoolResolver. TenantDispatchScope identity is the exact (real TenantID, opaque PoolKey) pair: handlers always receive the real tenant, table-presence cache entries cannot leak between generic and module pools, and physical databases with the same canonical host/port/database/schema are scanned once. Empty scopes back off to ColdDispatchInterval (one minute by default, configured with WithColdDispatchInterval) while active and recently active scopes retain DispatchInterval. Module topology is cached and refreshed by one caller per interval (one minute by default); use NewModulePoolResolverWithConfig to align ModulePoolResolverConfig.TopologyRefreshInterval with a service-specific cold interval. A failed refresh may use last-known-good topology under the existing fail-open contract, but every failure is retried after the interval and a failed first ownership lookup is never made permanent. ModulePoolResolver.InvalidateTopology() forces the next enumeration to refresh after tenant additions or topology changes. ModulePoolResolver.EvictTenant(tenantID) removes every scope immediately and forces refresh after removal or suspension; stale in-flight refreshes cannot restore evicted scopes. Newly committed/retryable/stuck rows remain governed by dispatcher cold-scope polling, which is unchanged. Legacy TenantPoolResolver implementations and direct ManagerPoolResolver wiring remain one-scope-per-tenant and unchanged. MongoDB retains row-scoped tenants plus optional module database resolution through mongo.WithModule / mongo.WithTenantDatabaseResolver. Tenant-aware repositories return ErrInvalidTenantID for IDs rejected by tenant-manager/core.IsValidTenantID. Retention is opt-in: WithRetentionPublished(d) makes the dispatcher delete PUBLISHED events created more than d ago, one bounded batch per dispatch scope per WithRetentionSweepInterval (one hour by default), at most WithRetentionBatchSize events per batch (500 by default), oldest first, never the types listed in WithRetentionKeepEventTypes. PENDING, PROCESSING and FAILED events are not deleted at any age; INVALID events are deleted only with WithRetentionInvalid(d), which makes the same sweep also delete those that became INVALID more than d ago (PostgreSQL only, through the optional outbox.InvalidPurger; column-per-tenant also lists and sweeps tenants holding them). Pool-per-tenant and schema-per-tenant PostgreSQL, and MongoDB with a tenant database resolver, sweep each known tenant as it is dispatched. Column-per-tenant PostgreSQL and row-scoped MongoDB (tenant field, no database resolver) dispatch only tenants with PENDING, PROCESSING or FAILED rows, so they also implement the optional outbox.PublishedTenantLister: once per sweep interval the dispatcher lists every tenant holding a PUBLISHED event older than d and sweeps each one, idle tenants included. The interval is per dispatcher instance: N replicas produce up to N batches per scope per interval, and deletes are idempotent. Each sweep adds to the outbox.events.purged counter; a failed sweep logs WARN and never blocks dispatch. The sweep needs the optional outbox.PublishedPurger capability, DeletePublishedBefore(ctx, before, keepEventTypes, limit), which the PostgreSQL and MongoDB repositories implement; enabling retention on a repository without it makes NewDispatcher fail with ErrOutboxRetentionUnsupported. Existing OutboxRepository implementations are unaffected. PostgreSQL additionally implements the optional outbox.TransactionalBatchWriter contract: CreateManyWithTx(ctx, tx, events) validates the full batch before issuing one set-wise INSERT, returns rows in input order, and treats an empty batch as a no-op.
  • commons/crypto: hashing (GenerateHash) and symmetric encryption (InitializeCipher/Encrypt/Decrypt) with credential-safe fmt output (String()/GoString() redact secrets); NewSealer(purpose, secret) seals bytes with AES-256-GCM under a key derived by HKDF-SHA256 from an operator secret of any format, at least 32 bytes (ErrSecretTooShort otherwise, on NewSealer and Rotate), binds additional data and opens payloads under the current or previous secret after Rotate
  • commons/jwt: HS256/384/512 JWT signing (Sign), signature verification (Parse), combined signature + time-claim validation (ParseAndValidate), standalone time-claim validation (ValidateTimeClaims/ValidateTimeClaimsAt)
  • commons/license: license validation with functional options (New(opts...), WithLogger, WithFailClosed), fail-closed default termination (Terminate exits with code 1 unless a custom handler is configured), handler management (SetHandler), error-returning validation (TerminateWithError/TerminateSafe)
  • commons/pointers: pointer conversion helpers (String, Bool, Time, Int, Int64, Float64)
  • commons/cron: cron expression parser (Parse) and scheduler (Schedule.Next)
  • commons/secretsmanager: M2M and external credential custody over a selectable backend — AWS Secrets Manager (default) or HashiCorp Vault KV v2, for deployments outside AWS. Retrieval via GetM2MCredentials / GetExternalCredentials; version-addressed external credentials use the opaque ExternalCredentialReference capability, created by BuildExternalSecretVersionReference or parsed from storage with ParseExternalCredentialReference(reference, trustedScope) before GetExternalCredentialsByReference; canonical UUID-versioned SecretIds (tenants/{env?}/{tenant}/{app}/external/{target}/credentials/versions/{uuid}), exact scope binding, strict input validation, typed retrieval errors, non-null string-only JSON objects, and the SecretsManagerClient test seam
    • Backend selection: Config{Backend: BackendVault, Vault: VaultConfig{...}}.NewReader(awsClient) and .NewWriter(awsClient). The zero Config keeps AWS, so existing deployments change nothing. Selection never falls back between backends: a misconfigured or unreachable backend is an error, never a silent switch to the other one.
    • Writes: SecretWriter (CreateSecretString / DeleteSecret) is create-only on both backends — rotation allocates a new versioned reference instead of overwriting — and deletion leaves no recovery window.
    • Vault auth: VaultConfig takes a static token (or VAULT_TOKEN); deployments using AppRole or Kubernetes auth authenticate their own *vaultapi.Client and pass it to NewVaultClientFrom, keeping token renewal where they can see it.
    • commons/secretsmanager/secretsmanagertest: the backend-agnostic contract suite both backends must pass, so "the backend is an infrastructure choice" stays a measured claim

Multi-tenant packages

  • commons/tenant-manager/core: shared tenant types, context helpers (ContextWithTenantID, GetTenantIDFromContext), and tenant-manager error contracts
  • commons/tenant-manager/cache: exported tenant-config cache contract (ConfigCache), ErrCacheMiss, and in-memory cache implementation used by the HTTP client
  • commons/tenant-manager/client: Tenant Manager HTTP client with circuit breaker, cache options (WithCache, WithCacheTTL, WithSkipCache), cache invalidation, and response hardening
  • commons/tenant-manager/consumer: dynamic multi-tenant queue consumer lifecycle management with tenant discovery, sync, retry, and per-tenant handlers
  • commons/tenant-manager/event: canonical tenant lifecycle dispatcher. Module-aware services register every PostgreSQL manager with WithPostgresManagers and every MongoDB manager with WithMongoManagers; removal events close all registered pools, while connection-setting events route only to the PostgreSQL manager matching the payload module. Existing singular options remain supported.
  • commons/tenant-manager/middleware: Fiber middleware for tenant extraction, upstream auth assertion checks, and tenant-scoped DB resolution
  • commons/tenant-manager/postgres: tenant-scoped PostgreSQL connection manager with LRU eviction, async settings revalidation, pool controls, and Module() for canonical lifecycle routing ("" identifies the generic resource)
  • commons/tenant-manager/mongo: tenant-scoped MongoDB connection manager with LRU eviction and idle-timeout controls
  • commons/tenant-manager/rabbitmq: tenant-scoped RabbitMQ connection manager with soft connection-pool limits and eviction
  • commons/tenant-manager/s3: tenant-prefixed S3/object storage. NewStorage preserves the general upload/create/download/delete/list API. NewRetainedStorage adds immutable version custody: CreateRetained performs an atomic If-None-Match: * write with explicit COMPLIANCE retention, canonicalizes retain-until to S3's whole-second UTC precision, and returns exact-version metadata; DownloadVersion and StatVersion require a VersionID; ValidateDefaultRetention fails closed unless Object Lock is enabled with COMPLIANCE retention of at least five years (Years >= 5 or Days >= 1827). NewRecoverableRetainedStorage adds deterministic create-or-recover for callers that can grant s3:ListBucketVersions: after a duplicate or ambiguous PUT timeout, it uses a detached, bounded lookup and returns only one exact-key version that is both sole and latest, has a non-empty exact VersionID, remains under COMPLIANCE retention, and exactly matches the caller's expected content type, content length, and canonical retain-until time. Missing permission, multiple versions, a latest delete marker, truncated/empty listings, or metadata drift fail closed. Payload digest verification remains the caller's responsibility. The retained surfaces expose no delete or retention-bypass operation.
  • commons/tenant-manager/valkey: tenant-prefixed Redis/Valkey key and pattern helpers with delimiter validation

Build and shell

  • commons/buildinfo: compiled build identity -- Set(Build) takes the -ldflags values, Get()/Modules(full) read them plus runtime/debug, Handler(service) serves GET /version, HandleFlag() answers --version, Scope(pkg) gives libraries their OTel instrumentation scope. See Build identity
  • commons/shell/: Makefile include helpers (makefile_colors.mk, makefile_utils.mk), shell scripts (colors.sh, ascii.sh), ASCII art (logo.txt)

Minimal usage

import (
    "github.com/LerianStudio/lib-commons/v7/commons"
)

func newRequestID() (string, error) {
    id, err := commons.GenerateUUIDv7()
    if err != nil {
        return "", err
    }

    return id.String(), nil
}

Read-side primitives

A bounded, internally consistent snapshot read on the replica: every statement in the body sees one instant of the database, each is capped server-side, and the transaction always rolls back.

err := client.RunReadOnly(ctx, postgres.ReadOnlyOptions{
    StatementTimeout:   10 * time.Second,
    TransactionTimeout: 15 * time.Second,
}, func(ctx context.Context, tx *sql.Tx) error {
    if err := tx.QueryRowContext(ctx, "SELECT count(*) FROM deliveries").Scan(&total); err != nil {
        return err
    }

    return tx.QueryRowContext(ctx, "SELECT count(*) FROM deliveries WHERE failed").Scan(&failed)
})

switch {
case errors.Is(err, postgres.ErrReadOnlyStatementTimeout): // one slow plan: index it
case errors.Is(err, postgres.ErrReadOnlyTxDeadline):       // the whole read: fewer statements
}

A signed cursor for the next page of that read. The binding is re-resolved from the validated JWT and the request on every page, never read out of the token.

codec, err := signedcursor.New(key) // exactly signedcursor.KeySize raw bytes

binding := signedcursor.Binding{
    Identity: []string{tenantID, scope},                 // who read the page
    Context:  []string{window.From, window.To},          // what it was read over
}

token, err := codec.Encode([]byte(`{"rank":42,"id":"sub_01"}`), binding)

payload, err := codec.Decode(token, binding)
if errors.Is(err, signedcursor.ErrInvalidCursor) {
    return fiber.NewError(http.StatusUnprocessableEntity, "invalid cursor")
}

Redaction at the logging boundary, where an error message becomes a log line. The chain survives; the credential does not.

if err := db.PingContext(ctx); err != nil {
    // "ping ledger pool: dial postgres://****:****@db.internal:5432/ledger: refused"
    return sanitize.Error(fmt.Errorf("ping ledger pool: %w", err))
}

Build identity

A binary carries its own identity: version, git revision, build time, Go version and the manifest of the Lerian modules linked into it. Those values are compiled in by the CI, never read from the environment, and commons/buildinfo is the single source for --version, GET /version and the OTel service.version.

Field GET /version --version
schemaVersion yes yes
service yes no
version, revision, buildTime, modified, goVersion yes yes
dependencyManifest (Lerian modules) no yes

The dependency manifest is available only through --version; run kubectl exec <pod> -- /service --version in a cluster. It never leaves the process over HTTP, and GET /version takes no query parameters.

Build stage of the service Dockerfile:

ARG TARGETARCH
ARG VERSION=dev
ARG REVISION=unknown
ARG BUILD_TIME=unknown
RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} \
    go build -trimpath -buildvcs=false \
      -ldflags="-s -w -X main.version=${VERSION} -X main.revision=${REVISION} -X main.buildTime=${BUILD_TIME}" \
      -o /service ./cmd/app

Without the three ARG lines the build args are ignored and the image ships as dev. ENTRYPOINT must be the binary itself, with no shell wrapper, so that docker run <image> --version works.

main.go of each binary:

package main

import "github.com/LerianStudio/lib-commons/v7/commons/buildinfo"

// Filled at build time via -ldflags -X main.<name>. Empty on local builds.
var version, revision, buildTime string

func main() {
    buildinfo.Set(buildinfo.Build{Version: version, Revision: revision, BuildTime: buildTime})
    buildinfo.HandleFlag() // "--version" prints the identity as JSON and exits 0

    // ... normal bootstrap
}

The symbol names main.version, main.revision and main.buildTime are stable forever; a major bump of lib-commons changes its module path, not these.

NewAdminApp returns a Fiber app with GET /version already mounted; the service adds /health, /readyz and /metrics to it:

admin := server.NewAdminApp("midaz-ledger")
admin.Get("/health", commonsHTTP.Ping)

server.NewServerManager(licenseClient, telemetry, logger).
    WithHTTPServer(app, ":3000").
    WithAdminHTTPServer(admin, ":8081").
    StartWithGracefulShutdown()

The admin address must differ from the main HTTP address, or the manager returns server.ErrAdminAddressConflict before starting anything. The admin server is the last one drained at shutdown: it keeps answering, with whatever your handlers return, while the API and gRPC servers finish in-flight work, then closes. To report 503 on /readyz during the drain, do it in your own handler.

Rules:

  • version is SemVer without the v (4.0.3, 1.2.0-beta.1), falling back to dev when nothing was injected. Never 0.0.0, and never a value read from an environment variable.
  • revision and buildTime fall back to the toolchain VCS stamp (vcs.revision, vcs.time), then to unknown.
  • commons/buildinfo.Scope(pkg) gives a library its OTel instrumentation scope (module path plus module version); libraries announce their version that way and never expose an endpoint.
  • The VERSION environment variable and commons/net/http.Version are the legacy path and are removed in v8.

Instrumentation scope names (v7.5.0)

Spans emitted by lib-commons now carry the package's full module path as otel.scope.name and the linked lib-commons version (v7.5.0, or (devel) when the binary has no module information) as otel.scope.version.

Package Old otel.scope.name New otel.scope.name
commons/postgres postgres github.com/LerianStudio/lib-commons/v7/commons/postgres
commons/mongo mongo github.com/LerianStudio/lib-commons/v7/commons/mongo
commons/redis redis github.com/LerianStudio/lib-commons/v7/commons/redis
commons/rabbitmq rabbitmq github.com/LerianStudio/lib-commons/v7/commons/rabbitmq
commons/net/http (reverse proxy) http.proxy github.com/LerianStudio/lib-commons/v7/commons/net/http
commons/net/http/ratelimit ratelimit github.com/LerianStudio/lib-commons/v7/commons/net/http/ratelimit
commons/net/http/pacing pacing github.com/LerianStudio/lib-commons/v7/commons/net/http/pacing

Update dashboards, alerts, sampling rules and collector processors that filter on otel.scope.name, otel.library.name or the Prometheus label otel_scope_name. On span-derived metrics, the otel_scope_version label now changes on every lib-commons upgrade, which starts new Prometheus series.

Environment Variables

The following environment variables are recognized by lib-commons or by canonical sibling libraries that lib-commons integrates with. Observability variables are owned by lib-observability.

Variable Type Default Package Description
VERSION string "NO-VERSION" commons Legacy. Application version, printed at startup by InitLocalEnvConfig. The runtime version comes from commons/buildinfo, compiled in, never from this variable
ENV_NAME string "local" commons Environment name; when "local", a .env file is loaded automatically
ENV string (none) lib-observability/assert When set to "production", stack traces are omitted from assertion failures
GO_ENV string (none) lib-observability/assert Fallback production check (same behavior as ENV)
LOG_LEVEL string "debug" (dev/local) / "info" (other) lib-observability/zap Log level override (debug, info, warn, error); Config.Level takes precedence if set
LOG_ENCODING string "console" (dev/local) / "json" (other) lib-observability/zap Log output format: "json" for structured JSON, "console" for human-readable colored output
LOG_OBFUSCATION_DISABLED bool false commons/net/http Set to "true" to disable sensitive-field obfuscation in HTTP access logs (not recommended in production)
METRICS_COLLECTION_INTERVAL duration "5s" commons/net/http Background system-metrics collection interval (Go duration format, e.g. "10s", "1m")
ACCESS_CONTROL_ALLOW_CREDENTIALS bool "false" commons/net/http CORS Access-Control-Allow-Credentials header value
ACCESS_CONTROL_ALLOW_ORIGIN string "*" commons/net/http CORS Access-Control-Allow-Origin header value
ACCESS_CONTROL_ALLOW_METHODS string "POST, GET, OPTIONS, PUT, DELETE, PATCH" commons/net/http CORS Access-Control-Allow-Methods header value
ACCESS_CONTROL_ALLOW_HEADERS string "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization" commons/net/http CORS Access-Control-Allow-Headers header value
ACCESS_CONTROL_EXPOSE_HEADERS string "" commons/net/http CORS Access-Control-Expose-Headers header value
RATE_LIMIT_ENABLED bool "false" commons/net/http/ratelimit Explicit opt-in: set to "true" to enable rate limiting. When unset or falsy, New returns nil and all requests pass through
RATE_LIMIT_MAX int 500 commons/net/http/ratelimit Maximum requests per window for DefaultTier
RATE_LIMIT_WINDOW_SEC int 60 commons/net/http/ratelimit Window duration in seconds for DefaultTier
AGGRESSIVE_RATE_LIMIT_MAX int 100 commons/net/http/ratelimit Maximum requests per window for AggressiveTier
AGGRESSIVE_RATE_LIMIT_WINDOW_SEC int 60 commons/net/http/ratelimit Window duration in seconds for AggressiveTier
RELAXED_RATE_LIMIT_MAX int 1000 commons/net/http/ratelimit Maximum requests per window for RelaxedTier
RELAXED_RATE_LIMIT_WINDOW_SEC int 60 commons/net/http/ratelimit Window duration in seconds for RelaxedTier
RATE_LIMIT_REDIS_TIMEOUT_MS int 500 commons/net/http/ratelimit Timeout in milliseconds for Redis operations; exceeded requests follow fail-open/fail-closed policy
SECURITY_ENFORCEMENT bool false commons Enables hard enforcement for configured security-tier checks that otherwise warn during migration phases
ALLOW_INSECURE_OTEL string "" lib-observability/tracing Justification override that allows insecure OTEL exporter endpoints in strict tier
ALLOW_WEBHOOK_PRIVATE_NETWORK string "" commons/webhook Justification override that enables WithAllowPrivateNetwork outside permissive tier for explicit private IP-literal webhook targets
OTEL_EXPORTER_OTLP_ENDPOINT string (none) lib-observability/tracing General OTLP endpoint read by the OTel SDK; bare host:port values are normalized to http://host:port
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT string (none) lib-observability/tracing Traces-specific OTLP endpoint; bare host:port values are normalized to http://host:port
OTEL_EXPORTER_OTLP_METRICS_ENDPOINT string (none) lib-observability/tracing Metrics-specific OTLP endpoint; bare host:port values are normalized to http://host:port
OTEL_EXPORTER_OTLP_LOGS_ENDPOINT string (none) lib-observability/tracing Logs-specific OTLP endpoint; bare host:port values are normalized to http://host:port

Additionally, commons.SetConfigFromEnvVars populates any struct using env:"VAR_NAME" field tags, supporting string, bool, integer types, time.Duration and []string. Consuming applications define their own variable names through these tags.

Defaults

A field may carry an envDefault tag giving the value to use when its variable is unset, blank, or unparseable for the field's type:

type Config struct {
    AuthEnabled bool          `env:"PLUGIN_AUTH_ENABLED"  envDefault:"true"`
    Port        int           `env:"SERVER_PORT"          envDefault:"8080"`
    Timeout     time.Duration `env:"REQUEST_TIMEOUT"      envDefault:"30s"`
    Origins     []string      `env:"CORS_ALLOWED_ORIGINS" envDefault:"https://app.example.com"`
}

An explicit, non-blank, parseable value always wins, including false — the default only fills a gap, it does not override an operator.

A value that is present but unparseable for the field's type takes the default instead, and GetenvBoolOrDefault/GetenvIntOrDefault/GetenvDurationOrDefault warn to stderr when they do. That predates this tag and is deliberately unchanged: the alternative is refusing to boot on a typo in a variable the field has a working default for. It does mean PLUGIN_AUTH_ENABLED=flase yields true here rather than an error — so a guard that must reject an explicitly disabled value in production belongs in a validator that reads the raw variable, not in the default.

Without the tag a field takes its zero value, and for a bool that is false. A flag that must be ON unless an operator turns it off therefore MUST declare the default; relying on the variable being present ships the feature OFF to whoever forgets it. envDefault is the only accepted spelling — default is read by nothing, and a tag that is silently ignored is worse than no tag, because a reviewer sees it and passes.

An envDefault the field's type cannot hold — envDefault:"maybe" on a bool, or envDefault:"999" on an int8 — returns ErrInvalidDefaultValue at load time rather than falling back to zero. A default that does not apply is indistinguishable from no default at all, which is the failure mode this tag exists to remove.

Durations

A time.Duration field takes a value with a unit — 30s, 2m, 720h, 150ms — in both the environment variable and the envDefault tag. It is matched on its type, ahead of the integer types, because time.Duration is defined as an int64: a switch on reflect kind cannot tell the two apart, so until this was handled explicitly envDefault:"30s" failed the load outright and envDefault:"30" silently meant thirty nanoseconds.

A unit-less integer remains a nanosecond count, matching time.Duration's own numeric meaning and this loader's historical reading. Deployed configuration relies on it — a Helm value of "2000000000" means two seconds — so re-reading a bare integer as seconds would silently stretch a two-second timeout to roughly 63 years. Write the unit; the unit-less spelling is legacy that keeps working.

commons.GetenvDurationOrDefault(key, fallback) applies the same parsing to a single variable, for code that reads one value rather than populating a struct.

Development commands

Core

  • make build -- build all packages
  • make ci -- run the local fix + verify pipeline (lint-fix, format, tidy, check-tests, sec, vet, test-unit, test-integration)
  • make clean -- clean build artifacts and caches
  • make tidy -- clean dependencies (go mod tidy)
  • make format -- format code with gofmt
  • make help -- display all available commands

Testing

  • make test -- run unit tests (uses gotestsum if available)
  • make test-unit -- run unit tests excluding integration
  • make test-integration -- run integration tests with testcontainers (requires Docker)
  • make test-all -- run all tests (unit + integration)

Coverage

  • make coverage-unit -- unit tests with coverage report (respects .ignorecoverunit)
  • make coverage-integration -- integration tests with coverage
  • make coverage -- run all coverage targets

Code quality

  • make lint -- run lint checks (read-only)
  • make lint-fix -- auto-fix lint issues
  • make vet -- run go vet on all packages
  • make sec -- run security checks using gosec (make sec SARIF=1 for SARIF output)
  • make check-tests -- verify test coverage for packages

Test flags

  • LOW_RESOURCE=1 -- reduces parallelism and disables race detector for constrained machines
  • RETRY_ON_FAIL=1 -- retries failed tests once
  • RUN=<pattern> -- filter integration tests by name pattern
  • PKG=<path> -- filter to specific package(s)

Git hooks

  • make setup-git-hooks -- install and configure git hooks
  • make check-hooks -- verify git hooks installation
  • make check-envs -- check hooks + environment file security

Tooling and release

  • make tools -- install test tools (gotestsum)
  • make goreleaser -- create release snapshot

Project Rules

For coding standards, architecture patterns, testing requirements, and development guidelines, see docs/PROJECT_RULES.md.

License

This project is licensed under the terms in LICENSE.

About

lib-commons is a shared Go library that extracts and centralizes common functionalities from the pkg of midaz, making them reusable across all our applications. It aims to promote code reuse, maintainability, and consistency by providing core utilities, shared models, and essential business logic in a modular way.

Resources

Readme
License
Activity
Custom properties

Stars

8 stars

Watchers

2 watching

Forks

6 forks
Report repository

Releases

Packages

Used by

Contributors

Languages