Skip to content

test(smollm3): add unit tests for HumanEval+ and MBPP+ evaluation utilities (#137) - #148

Open
SiddardhaShayini wants to merge 1 commit into
huggingface:mainfrom
SiddardhaShayini:test-smollm3-eval-plus
Open

SiddardhaShayini wants to merge 1 commit into
huggingface:mainfrom
SiddardhaShayini:test-smollm3-eval-plus

Conversation

@SiddardhaShayini

Copy link
Copy Markdown

Fixes #137

What changes are proposed in this PR?

  • Adds comprehensive unit tests for HumanEval+ and MBPP+ evaluation utilities under text/evaluation/smollm3/.
  • Covers critical edge cases including empty/whitespace model outputs, markdown code block variations, unclosed code fences, and malformed EvalPlus JSON result payloads.
  • Runs entirely on CPU without requiring GPU acceleration or heavy model/dataset downloads.

How to test?

You can run the tests locally via pytest:

cd text/evaluation/smollm3
pytest test_eval_plus.py -v

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

No HumanEval+ task for SmolLM3

1 participant