feat: async remote storage upload with tracking, QR indicator and env… - #1603
feat: async remote storage upload with tracking, QR indicator and env…#1603fmiccolis wants to merge 3 commits into
Conversation
- Environment & network tab: one QR code per IPv4 address encoding the photobooth URL on that network, to open it from another device - New remote storage queue tab showing per-file upload status, attempts, errors and timestamps (works with the existing auto-refresh toggle) - Add missing label for the remote storage log tab Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The build workflow runs "npm run build", which sorts en.json with localeCompare and then requires a clean working tree. The new remoteUpload* keys were placed after "rembglog" but localeCompare orders them after "remoteStorageTemplate", so the build modified the file and the status check failed. it.json is aligned for consistency. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hi, thanks for working on this. This PR is very similar in scope to my earlier PR #1463, which also implements asynchronous FTP/SFTP uploads with a persistent queue, retries and upload tracking. The main difference seems to be the architecture: #1603 uses a JSON queue processed through a background HTTP request, while #1463 uses SQLite and a dedicated Symfony/systemd worker. It may be worth comparing the approaches and seeing whether parts can be consolidated. |
|
Hi @flacoonb and sorry for not digging in the previous PRs before submitting mine. |
|
Hi @fmiccolis, no problem at all — it is easy to miss an older open PR, especially in a project with many ongoing contributions. I am glad you brought your approach forward, as the overlap gives us a good opportunity to compare both implementations and potentially combine the strongest parts. I chose SQLite because it provides a persistent transactional queue without requiring an external database server. It allows atomic job claiming, reliable recovery after crashes or restarts, and structured tracking of attempts, errors and filename mappings. The systemd worker keeps uploads fully independent from PHP and the web request lifecycle. This avoids request timeouts, session locks and occupied web-server workers during slow or unavailable FTP connections. It also gives automatic startup and restart on failure. The downside is additional installation complexity and a stronger dependency on Linux/systemd. Your HTTP-based approach is simpler and integrates more naturally into the existing Photobooth setup, so I can understand that choice. I agree that the remote-storage configuration should be simplified, and it may be useful to combine the strongest parts of both PRs. |
… debug tab
Prerequisites checklist
What is the purpose of this pull request? (put an "x" next to an item)
What changes did you make? (Give an overview)
This PR makes the remote storage (FTP/SFTP) upload asynchronous, so the transition
from capture to the result screen is no longer blocked by a slow uplink, and adds
upload tracking plus two new admin debug panel tabs.
Asynchronous upload with tracking
applyEffects.phpno longer uploads synchronously: files are enqueued in a persistentJSON queue (
var/run/remotestorage_queue.json, newRemoteStorageQueueService) and theresponse returns immediately.
api/remoteStorageUpload.php, which drains the queue in the background:ignore_user_abort(), single-drainer lock (flock),session_write_close()(critical:without it the PHP session lock would freeze the booth UI during uploads), priority for
the just-captured photo, and a circuit breaker after 2 consecutive failures so an
unreachable server does not pile up ~90s FTP timeouts.
pending/uploading/done/failed), attempts and last error are persisted.Retries happen on later drains (max 3 attempts); failed uploads can be listed/retried/
cleared via the new admin-protected
api/remoteStorageAdmin.php.upload); a small spinner next to it turns into a check/error icon, driven by a cheap
status-polling mode of the same endpoint (same pattern as
gallery.php?status).deletePhoto.phpcancels queued uploads and skips remote round-trips for photos thatwere never uploaded.
createWebpage()fixes/improvementsconfig.php→config.inc.php): before this, the early-returncondition could never be true, so the webpage template + config were re-uploaded on
every single photo.
file) instead of doing remote
fileExistsround-trips per photo.ftp.template_config_locationoption (defaultprivate/template/config.inc.php):an optional PHP file returning an array that is merged over the generated
config.inc.phpdefaults — the counterpart of the already-customizabletemplate_locationfor people using a custom remote gallery template. Also applied tothe
test/remote-storage-template.phppreview.New debug panel tabs
CIDR network (via
net_get_interfaces()), Wi-Fi SSID (best-effort viaiwgetid), andone QR code per IPv4 address encoding the photobooth URL on that network — so an
operator can quickly open the booth (or SSH/VNC into it) from another device.
timestamps), compatible with the existing auto-refresh toggle.
Behavior changes to be aware of
keying.show_all = falseare no longer uploaded to remote storage(previously they were uploaded and then deleted locally; the remote gallery now mirrors
local visibility).
faileduntil retried via the admin endpoint —deliberately not retried on every capture so an offline server cannot slow down an event.
useForQr+append_filenamethe QR deep link is now<gallery>/?photo=<file>instead of
<gallery>/images/<file>(see review notes below).Is there anything you'd like reviewers to focus on?
session_write_close()in the drain/status endpoints is load-bearing;please double-check I haven't missed a path that would hold the session while uploading.
a 1.5s linger re-check, a one-shot re-trigger from the status poller, and the next
capture's trigger), and stale
uploadingrecovery (entries reset after 10 min)./?photo=instead of/images/): the bundled gallery templatecurrently ignores the
photoquery parameter, so with the default template the QR opensthe gallery page rather than the raw image file. I use a custom template that supports
it. If you prefer, I can split this into a separate PR or extend the bundled template to
honor
?photo=.set_time_limit(0)cannot override server-level timeouts — leftover entries are simplypicked up by the next drain.
Tested on a live installation during a real event (FTP over a slow uplink): the result
screen now appears immediately, the queue drains in the background, and the
failure/recovery paths were exercised by taking the FTP server offline mid-event.
AI used to create this Pull Request?
Yes. This PR was developed with Claude Code (Anthropic). The AI wrote the initial
implementation (queue service, endpoints, frontend changes, debug tabs) based on a design
we discussed and I decided on; I reviewed every change and tested the full flow on a real
installation at a live event (successful uploads, failure/retry paths, burst captures,
offline-server recovery).
npm run buildandnpm run eslintwere run (clean), and alltouched PHP files were syntax-checked.