Skip to content

[Bug]: POST /v1/resolutions returns 500 on every update while persisting the value #367

Description

@smaramwbc

Summary

POST /v1/resolutions is documented as "Create or update a resolution" and upserts by (subject_id, session_id, tenant_id). The first write to a logical key returns 200. Every subsequent write returns 500 internal_error — and persists anyway.

So the only upsert-by-logical-key on the consumer surface tells callers their write failed, after committing it. That is the worse of the two possible wrong answers: a client that retries on 500 re-sends, a client that treats 500 as failure diverges from server state, and a client that trusts the response never learns the value landed.

Reproduction

Against main (91f4377), no auth, no LLM key:

for s in open resolved open resolved; do
  curl -s -o /dev/null -w "%{http_code}\n" -X POST localhost:8100/v1/resolutions \
    -H 'Content-Type: application/json' \
    -d "{\"subject_id\":\"demo\",\"session_id\":\"k1\",\"status\":\"$s\"}"
done
200
500
500
500

Read back — the last write is there, so all four writes committed:

curl -s 'localhost:8100/v1/resolutions?subject_id=demo'
# [{"session_id":"k1","status":"resolved", ...}]   # one row, latest value

Root cause

create_resolution (server/api/resolutions.py) serialises the row it gets back after committing:

result = await repo.upsert_resolution(session, row)
await session.commit()

return ResolutionResponse.from_row(result)

session.commit() expires every instance in the session, so reading a column off result afterwards is lazy IO. On the async engine that raises sqlalchemy.exc.MissingGreenlet, which falls into the catch-all handler in server/core/errors.py and reaches the client as 500 internal_error:

sqlalchemy.exc.MissingGreenlet: greenlet_spawn has not been called; can't call
await_only() here. Was IO attempted in an unexpected place?
  File "server/api/resolutions.py", line 46, in create_resolution

Why only the update path. upsert_resolution returns the instance this request just constructed on INSERT, whose attributes are still populated in Python and need no refresh. On UPDATE it returns the row it loaded from the database, which the commit expired.

Not a regression from #295 / #356

The obvious suspect is #356, which replaced the inline field mapping with ResolutionResponse.from_row(result). It is not the cause: from_row reads exactly the same nine fields the inline code read, also after the commit. I checked out b99596b~1 and ran it — 200, 500, 500, identical. The defect dates to the endpoint's introduction in d4fef67.

Not covered by #366

#366 maps sqlalchemy.exc.OperationalError to 503. MissingGreenlet is an InvalidRequestError, not an OperationalError, so it still falls to the catch-all 500.

Fix

POST /v1/episodes already has the house pattern for this — await session.commit() followed by await session.refresh(row). resolutions.py is the one route that commits and then reads ORM attributes without refreshing.

I audited the other routers: subjects.py reads only plain ints after commit, and health.py I verified empirically returns 200 on repeated calls. The defect is isolated to resolutions.py.

PR to follow.

Environment

  • main @ 91f4377 (v1.5.0), Postgres 16 + pgvector, no API key, no LLM key
  • Reproduced on the published statewavedev/statewave:1.5.0 image and from source

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions