Skip to content

Normalize rack-manager alert severity to upper case at ingest - #32

Open
chinmoy-nexthop wants to merge 1 commit into
sonic-net:masterfrom
nexthop-ai:chinmoy.Leak.Sev.redis
Open

Normalize rack-manager alert severity to upper case at ingest#32
chinmoy-nexthop wants to merge 1 commit into
sonic-net:masterfrom
nexthop-ai:chinmoy.Leak.Sev.redis

Conversation

@chinmoy-nexthop

Copy link
Copy Markdown
Contributor

Why:

Redfish clients send the SonicSeverity enum in mixed case (Normal, Minor,
Major, Critical), but the platform STATE_DB schema and its consumers compare
against upper-case constants: thermalctld reads SYSTEM_LEAK_STATUS and bmcctld
dispatches on LEAK_CONTROL_POLICY with checks such as severity equals CRITICAL.
sonic-dbus-bridge stored whatever the client sent, verbatim, so a standards
compliant "Critical" leak alert was written to RACK_MANAGER_ALERT as "Critical"
and the consumer comparison silently missed it. The result was a critical leak
that produced no dispatch and no visible error, which is a safety-relevant
failure mode rather than a cosmetic one.

How:

The casing is fixed once, at the ingest boundary, instead of being worked
around in every consumer:
- Added a small normalizeSeverity helper in rack_manager_receiver.cpp that
  upper-cases the string with an unsigned-char-safe std::toupper, with a
  comment recording why the platform schema requires it.
- effectiveSeverity now returns the normalised value for every severity that
  is present, so both SubmitAlert and SubmitTelemetry payloads are covered by
  the single unified extraction path.
- The fallback used when no severity is supplied anywhere along the
  inheritance chain changed from "Normal" to "NORMAL", and the accompanying
  warning text was aligned with it, so the default takes the same shape as a
  supplied value.
- The behaviour is documented in the OEM extension README so clients know the
  stored value is upper-cased.
The leak-to-action decision itself was deliberately left in bmcctld's policy;
the bridge only owns the RACK_MANAGER write, so no policy moved here.

How to verify:

- POST a SubmitAlert payload carrying "Severity": "Critical" and confirm the
  stored row holds CRITICAL, with
  redis-cli -n 6 HGETALL "RACK_MANAGER_ALERT|Rack_level_leak".
- Repeat with lower-case "critical" and mixed-case "CrItIcAl" and confirm both
  land as CRITICAL.
- POST a payload with the Severity key omitted and confirm the row holds NORMAL
  and the bridge logs the missing-severity warning.
- Confirm bmcctld now dispatches on the critical alert that previously produced
  no action.
- make unit-test and make test, to confirm nothing else regressed.

Why:
Redfish clients send the SonicSeverity enum in mixed case (Normal, Minor,
Major, Critical), but the platform STATE_DB schema and its consumers compare
against upper-case constants: thermalctld reads SYSTEM_LEAK_STATUS and bmcctld
dispatches on LEAK_CONTROL_POLICY with checks such as severity equals CRITICAL.
sonic-dbus-bridge stored whatever the client sent, verbatim, so a standards
compliant "Critical" leak alert was written to RACK_MANAGER_ALERT as "Critical"
and the consumer comparison silently missed it. The result was a critical leak
that produced no dispatch and no visible error, which is a safety-relevant
failure mode rather than a cosmetic one.

How:
The casing is fixed once, at the ingest boundary, instead of being worked
around in every consumer:
- Added a small normalizeSeverity helper in rack_manager_receiver.cpp that
  upper-cases the string with an unsigned-char-safe std::toupper, with a
  comment recording why the platform schema requires it.
- effectiveSeverity now returns the normalised value for every severity that
  is present, so both SubmitAlert and SubmitTelemetry payloads are covered by
  the single unified extraction path.
- The fallback used when no severity is supplied anywhere along the
  inheritance chain changed from "Normal" to "NORMAL", and the accompanying
  warning text was aligned with it, so the default takes the same shape as a
  supplied value.
- The behaviour is documented in the OEM extension README so clients know the
  stored value is upper-cased.
The leak-to-action decision itself was deliberately left in bmcctld's policy;
the bridge only owns the RACK_MANAGER write, so no policy moved here.

How to verify:
- POST a SubmitAlert payload carrying "Severity": "Critical" and confirm the
  stored row holds CRITICAL, with
  redis-cli -n 6 HGETALL "RACK_MANAGER_ALERT|Rack_level_leak".
- Repeat with lower-case "critical" and mixed-case "CrItIcAl" and confirm both
  land as CRITICAL.
- POST a payload with the Severity key omitted and confirm the row holds NORMAL
  and the bridge logs the missing-severity warning.
- Confirm bmcctld now dispatches on the critical alert that previously produced
  no action.
- make unit-test and make test, to confirm nothing else regressed.

Signed-off-by: Chinmoy Dey <chinmoy@nexthop.ai>
@mssonicbld

Copy link
Copy Markdown

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@chinmoy-nexthop chinmoy-nexthop changed the title Normalise rack-manager alert severity to upper case at ingest Normalize rack-manager alert severity to upper case at ingest Jul 31, 2026
@chinmoy-nexthop

Copy link
Copy Markdown
Contributor Author

@judyjoseph

@judyjoseph

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@judyjoseph

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 23, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 24, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 25, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 26, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 27, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 27, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 28, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 29, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 29, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 31, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Aug 31, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 1, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 1, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 3, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 3, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
@judyjoseph

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 5, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 5, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 7, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 7, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 9, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
nh-grecs Bot pushed a commit to nexthop-ai/sonic-redfish that referenced this pull request Sep 9, 2026
rebuild-source: sonic-net#32 @ nexthop-ai/sonic-redfish 2f1d44a [case: upstream:open]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants