Upgrading Stashai to ONNX endpoint ; Same HF Host#730
Conversation
Details Hosted by the original creator at the same HF Spaces URL This was tested and verified by hand with multiple different media formats across both Tags and Markers and is confirmed operational on my machine. Used OpenCode to help understand Gradio and how it handles API interactions as well as how that interacts with stash and base64 data URL. There is a built in retry logic loop as well as built in compatibility for older browsers. if it is not working on your end i would suggest checking the HF Spaces URL above and ensuring that it is fully online and operational before blaming the script. Both the stashai.js and the stashai.yml must be deployed for the change to take effect and be functional. only deploying the stashai.js will result in errors and result in failure of the plugin function completely. Post-Update migrated from a Gradio 3 Space to a Gradio 5 Space, which meant completely rewriting how the plugin talks to the API. After changing stashai.js you must hard refresh your browser (Ctrl + Shift + r). In the Stash App, reload plugins in plugins settings in the main Settings screen. Regular browser refresh won't pick up the changes. Some-Context The ONNX Space has a bigger, better model but runs on HF's free tier (2GB RAM). The 335MB model barely fits and it cold-starts, OOMs occasionally, and then HF auto-restarts it. The plugin needs retry logic to handle this. The 335MB ONNX model takes ~30 seconds to load on HF's free tier. During that time, the Space might OOM and restart. The first request often fails, but the second or third works. This is solved by the retries with delays in the script.
|
Apologies about the original pull requests to the wrong branch of the project. |
Changed from version 1.0.2 to version 1.0.3 per maintainer/contributor directions.
Drewlius
left a comment
Drewlius
left a comment
There was a problem hiding this comment.
Verified to be a changed version per the maintainer directive. The version now reflects 1.0.3 compared to previously showing 1.0.2
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
|
Got it to work. My bad for not hard refreshing. Upon testing it looks to be noticeably slower. Ignoring first request, the subsequent requests are 2-5 times slower than non-ONNX endpoint. Results looks to be same, so not sure what is the benefit of this bigger and better model. |
|
I get more returns with better results based on the testing I did, with a larger tag database to pull from. From what I could find for the reason it wasn't working with the other code it appears some back end Gradio changes. The ONNX Space's 'requirements.txt' pins 'gradio>=5.0.0' no minor-version lock. On container rebuild (cold start, cache eviction, or infrastructure update), pip resolves to whatever latest Gradio 5.x satisfies the constraint. The '/gradio_api/call/{event_id}' SSE path was never the correct endpoint for a queue-enabled Gradio 5.x space. it just so happened to work with whatever version or cache state the container had when I tested it previously. I am about to update the files now. Tested several times and verified functional. if it still fails to load it could be due to cold start of the huggingface server. But i definitely seem to get more results with a larger tags pool to pull from with this version. Maybe try again with the newly pushed commit.
|
Mismatched output format - Old '/gradio_api/call/{event_id}' SSE 'complete' event returned data wrapped as '[function_result]'. Gradio 5's queue API 'process_completed' event returns 'function_result' directly (no outer array). Both callers ('result[0]' for markers, 'result.forEach' for tags) expect the wrapper. Caused 'frames.filter is not a function' and 'result.forEach is not a function' on first test.
- Switched '_gradioCall()' to 'POST /gradio_api/queue/join' with 'fn_index' (0 = predict_tags, 1 = predict_markers) + random 'session_hash', SSE stream from '/gradio_api/queue/data'
- Parses 'data:' JSON lines for 'msg: "process_completed"' (Gradio 5 SSE v3 format)
- Wraps result as '[msg.output.data[0]]' to match old callers' expected shape
- Consolidated two URL constants into one 'STASHMARKER_API_BASE'
ONNX Space pins 'gradio>=5.0.0' (no minor lock). On container rebuild (cold start, cache eviction), pip resolves to latest Gradio 5.x. Old '/gradio_api/call/' endpoint was never correct for queue-enabled Gradio 5 it worked during testing due to whatever version/cache state the container had at that time.
Testing the new endpoint returns tags on inputs that would not appear previously, and the accuracy of the returned tags appears anecdotally to be more accurate than the previous non -ONNX endpoint. Also included an Agnets.md file in the event that a user wants to have an agent work with the API directly. this was sourced from [source](https://huggingface.co/spaces/cc1234/stashtag_onnx/agents.md)
There was a problem hiding this comment.
Tested on my machine on several clips and verified to be functional. I would recommend going to the ONNX url jsut to verify that the model is up and running as HuggingFace will spin the server down after a certain amount of time with no activity. This can cause the stashai functionality to be lost until the server is loaded. I was able to get new tags to appear that I was not getting returned to me with the old non-ONNX endpoint as well as the accuracy of the tags seemed more precise (anecdotally) than they did prior to the change. Your mileage may vary. Both the .js & .yaml need to be added to your code base for change to take effect, as well as a hard browser cache reload, and refreshing the plugins via the app interface. I would also recommend giving the application a hard restart as well as the browser
I would order it in this way: Plugin refresh in application interface plugins setting --> Hard application full restart --> clear browser cache and restart the browser then reconnect to your hosted endpoint. Hopefully this will be the final needed change to this fingers x crossed. Keep me posted with any further issues you may run into.
--P.S. the 'agents.md' file is not strictly necessary and was linked to on the models landing page for use if an agent wants to be integrated to work with the model. I included it in the off event someone would want integration with an agent of theirs. This can be included or not at the final verification and publishing stage, if we are able to get to that point.
This config file is meant for a separate agent that a user could choose to integrate with the ONNX model. This was linked directly on the ONNX landing page and can be found [here](https://huggingface.co/spaces/cc1234/stashtag_onnx/agents.md) this can be removed from the code base, or kept for additional integration possibilities in the future. It is not strictly necessary for any functionality of the base stashai plugin.
2 participants
Details
Post-Update
Some-Context
AI USAGE
Opencode was used to help me understand the syntax of the API requests to the Stashai ONNX Model endpoint, the GRADIO 5 documentation was fed into it, HF API Request documentation for JS was fed in, the old stashai.js was fed in, and the contents of my IDE was fed in, this was done to query when I had questions regarding the documentation and helping to troubleshoot non 200 OK Responses from the HF API endpoint. JavaScript is also fairly new to me so it was useful and time saving for me to get help with particular JS Syntax and plain English translations of functions and definitions contained within the original .JS file. The code was all tested and verified to be functioning properly and returning both Tags and Markers and successfully attaching them to the video file and adding those Tags/Markers to the DB properly. There may be issues still present. I am not an expert at web development or Javasript and have spent most of my time in programming writing Python and bash scripts for Linux, Like i stated though the script is working and returning responses from the correct ONNX API endpoint. If you review and see something is wrong don't hesitate to point it out. I will spend the time to fix it to the best of my abilities.