K2GO-386 feat(diskguard): app-side free-space guard that stops the box before ENOSPC - #544
Open
luisguzman-adfa wants to merge 4 commits into
Open
K2GO-386 feat(diskguard): app-side free-space guard that stops the box before ENOSPC#544luisguzman-adfa wants to merge 4 commits into
luisguzman-adfa wants to merge 4 commits into
Conversation
The disk-fill happens when the box proot dies (a relaunch/restore orphans a service that busy-loops); the in-box healer dies with it. An Android-side guard, independent of the box, is the only thing that can act — device-proven: only a full teardown (force-stop) stops the orphan; an in-box restart is unreachable and an in-proot kill does not reach it. - diskguard/domain/DiskGuardPolicy: pure rule. CRITICAL below a 1.5 GiB floor, set below StorageGuard's 2 GiB op-floor so only a headroom-less runaway trips it (legit ops keep >= 2 GiB). JVM unit-tested. - diskguard/DiskGuard: probe (StorageProbe) -> policy -> on CRITICAL reap the box, truncate the runaway log to reclaim the space, and warn the user (notification). - EnvironmentProcess.reapBox: full box teardown (kill the proot + reap the daemonised services by cmdline, same uid + SELinux domain) = the force-stop equivalent from inside the app. - WatchdogService: a 25 s poller runs the guard for the life of a protected (box-up) session; plus a DEBUG-only ACTION_DISK_GUARD_TEST hook to force a check with an injected floor (device-verify without filling ~58 GB). - strings_untranslated.xml: the guard's user alert, pending localization. Covers ANY runaway via the one common surface (free space), not just php-fpm.
…g files Code-review pass-2 fixes on the disk-guard reclaim step: - reclaimRunawayLog now recurses /var/log subdirectories, so a runaway writing to e.g. nginx/error.log is reclaimed, not just top-level files. - Only *.log files over 1 GiB are candidates (was any file over 256 MB), so a legitimate large file is never truncated, including when the DEBUG device-verify hook fires on a healthy box.
…ot relaunched Device testing (HD1901, 2026-09-04) proved a reap alone is undone: the ADFA-5343 reconciler relaunches the box proot within ~3s, so the runaway resumes. The guard now first sets the persisted user-intent lever to off (ServerLifecycleReconciler.setUserWantsOn(false)) so desired=DOWN and the reconciler will not relaunch, THEN reaps for immediacy (the reconciler's own graceful pdsm stop is ~40s, too slow while the disk fills). Reuses the one existing lever the server toggle owns rather than adding a second 'forced down' flag; the user re-enables the server after freeing space.
…g receiver The device-verify trigger lived on WatchdogService (exported=false), which adb am cannot reliably reach. Move it to DebugDiskGuardReceiver in src/debug (exported, reachable via am broadcast), mirroring DebugDeliveryReceiver, and drop the DEBUG action + hook from WatchdogService. The trigger never ships in release (src/debug source set) and no longer carries debug code in the production service.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an Android-side guard that watches free space while the box is up and, on a
critical reading, tears the box down and reclaims the runaway log — so a service
that busy-loops into /var/log can no longer fill the device to ENOSPC.
diskguard/domain/DiskGuardPolicy— pure rule (JVM-tested): free space below a1.5 GiB floor → CRITICAL; null/negative → UNKNOWN (fail-safe, never acts on a bad
read). The floor sits below StorageGuard's 2 GiB op-floor, so a normal op keeps
≥2 GiB and only a headroom-less runaway trips the guard.
diskguard/DiskGuard— on CRITICAL: set desired=DOWN via the reconciler, reap thebox, reclaim the runaway log, notify the user.
env/EnvironmentProcess.reapBox— kill the box proot + reap its services by/proc cmdline (same technique as the existing
reapEnvironmentHttpFront).WatchdogService— a single poller runsDiskGuard.checkevery 25 s for the lifeof a protected (box-up) session; started once per session, stopped on destroy.
strings_untranslated.xml— the guard's user alert (pending l10n).src/debug— an exportedDebugDiskGuardReceiverto trigger the guard from adb fordevice-verify; never ships in release.
Why
The disk-fill happens exactly when the box proot dies (a relaunch/restore orphans a
service that busy-loops), so the in-box healer is already dead — only an
outside-the-rootfs actor can stop it. Two points that shaped the design, both
device-proven:
relaunches the box in ~3 s and the runaway resumes. The guard first sets the one
persisted intent the server toggle already owns (
setUserWantsOn(false)) so thereconciler will not relaunch, then reaps for immediacy. No new "forced-down" flag —
one source for "should the box be up"; the user re-enables the server after freeing
space.
not just php-fpm.
Verification
Device-verified end to end (OnePlus HD1901, Android 15 / API 35): the guard set
desired=DOWN, reaped the box, the reconciler then held
intent=NOOP(box did notrelaunch), the runaway
.logwas truncated and the space reclaimed, and the alertposted.
DiskGuardPolicyunit tests green.Coordination
Barrier 2 of K2GO-386. Barrier 1 (stop php-fpm running idle in the rootfs build) ships
on
fix/K2GO-386-php-fpm-idle-off.