diff --git a/README.md b/README.md index a77277f..b0c5b4b 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ Hass-Cleaner is a Home Assistant App for inspecting storage, stale entities and registry relationships before anything is cleaned up. Safety, informed user choice and recovery come before reclaimed space. -> **Release status:** version 1.1.0 is published for controlled early use. Hass-Cleaner provides facts, advice, backup options and recovery guidance; the user makes the final cleanup decision. The HACS installation changes in this checkout are not yet released. +> **Release status:** version 1.1.1 is published for controlled early use, including HACS installation support. This checkout prepares **1.1.2**, not yet published. Hass-Cleaner provides facts, advice, backup options and recovery guidance; the user makes the final cleanup decision. ## English @@ -86,7 +86,13 @@ GitHub Actions builds the version in `hass_cleaner/config.yaml` for `amd64` and With HACS configured and Home Assistant 2026.9.0 or newer, add `https://github.com/dkwolf1/Hass-Cleaner` under **HACS → menu (⋮) → Custom repositories**, type **Integration**. Download **Hass-Cleaner Companion**, restart Home Assistant Core, then add it under **Settings → Devices & services**. Use the same release version as the app. -HACS updates the companion; the App Store updates the app. The ZIP remains available for manual installation. See [installation and migration instructions](docs/reference-checks.md#installation), including the publication prerequisite for this new HACS route. This repository is not in the default HACS catalog. +HACS updates the companion; the App Store updates the app. The ZIP remains available for manual installation. See [installation and migration instructions](docs/reference-checks.md#installation). This repository is not in the default HACS catalog. + +### Quarantine storage in 1.1.2 + +New operations atomically move files into `/homeassistant/.hass-cleaner-quarantine` (the Home Assistant configuration directory), retaining the original file rather than copying and unlinking it. This folder is excluded from scans. Cross-filesystem moves are rejected without a copy/delete fallback. Existing quarantine in app storage remains supported. + +Keep both the configuration folder and the app data in your backups: the recovery manifest stays in the app data. Do not manually remove the hidden quarantine folder. Moving files on the same filesystem does not free disk space; permanent deletion after expiry does. Stop active writers before cleanup; a later write may invalidate the stored checksum, requiring manual recovery instead of automatic restore. ### Local development and tests @@ -179,7 +185,9 @@ Versiegebonden wijzigingen staan in [CHANGELOG.md](hass_cleaner/CHANGELOG.md). G 4. Start de App en open de webinterface. 5. Voer eerst een scan uit en beoordeel het rapport voordat je een actie voorbereidt. -GitHub Actions bouwt de versie uit `hass_cleaner/config.yaml` voor `amd64` en `aarch64`. Na publicatie van de container kan Home Assistant die versie installeren of bijwerken. Versie 1.1.0 bevat [referentiecontrole en Reparaties](docs/reference-checks.md#nederlands-kort) via een aparte companion-integratie. Na publicatie van de HACS-ondersteuning kun je die via HACS installeren: voeg deze GitHub-repository toe als aangepaste repository, type **Integratie**. De ZIP blijft een alternatief. Praktijktests zijn nog nodig; de companion zit niet in de app-container. +GitHub Actions bouwt de versie uit `hass_cleaner/config.yaml` voor `amd64` en `aarch64`. Gepubliceerd: **1.1.1** met HACS-ondersteuning. Deze checkout bereidt **1.1.2** voor. Installeer de aparte [companion](docs/reference-checks.md#nederlands-kort) via HACS door deze repository toe te voegen als type **Integratie**. De ZIP blijft een alternatief; de companion zit niet in de app-container. + +Vanaf 1.1.2 worden nieuwe quarantainebestanden atomair naar `.hass-cleaner-quarantine` in de Home Assistant-configuratiemap verplaatst. Bestaande quarantaine blijft ondersteund. Bewaar zowel de configuratiemap als appdata in je back-up; verwijder de verborgen map niet handmatig. Verplaatsen maakt nog geen schijfruimte vrij. Verplaatsingen tussen bestandssystemen worden geblokkeerd. Stop actieve schrijvers vooraf; latere wijzigingen kunnen de checksum ongeldig maken en handmatig herstel vereisen. ### Lokaal ontwikkelen en testen diff --git a/ROADMAP.md b/ROADMAP.md index fbe4931..25802a5 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,6 +1,8 @@ # Hass-Cleaner Roadmap -[Nederlands](#nederlands) · [English](#english) +[English](#english) · [Nederlands](#nederlands) + +Current published release: **1.1.1**, including reference checks, Repairs and HACS support. This checkout prepares **1.1.2**: quarantine concurrency protection, reference diagnostics, list refresh fixes and documentation/translation corrections. All releases remain intended for controlled early testing, not a guarantee of safety. See [the changelog](hass_cleaner/CHANGELOG.md). ## Nederlands @@ -18,12 +20,15 @@ Veiligheid gaat vóór extra opruimwinst. Een release gaat pas door wanneer risi | 0.9.0 | Afgerond | Supervisor-back-upverificatie, hersteltest, bewaartermijn en expliciete verwijdering na verval. | | 0.9.1 | Afgerond | Toegestane back-upcontrole, hergebruik van back-upbewijs, bewuste optionele back-upkeuze en selecteerbare entitybeoordeling. | | 1.0.0 | Uitgebracht | Gebruikersgestuurde cleanup, registeruitvoering, persoonlijke inhoud, schone start en behoud van harde systeembescherming. | -| 1.0.1 | Voorbereid | Technische hardening: crashbestendige uitvoering, paginering, atomaire opslag, toegankelijkheid en betrouwbaar frontendcachebeleid. | -| 1.0.2 | Voorbereid | Herstel- en registerjournals, veilige configuratiesynchronisatie, begrensd scangeheugen, robuuste verzoekvalidatie en verbeterde tweetalige exports. | +| 1.0.1 | Uitgebracht | Technische hardening: crashbestendige uitvoering, paginering, atomaire opslag, toegankelijkheid en betrouwbaar frontendcachebeleid. | +| 1.0.2 | Uitgebracht | Herstel- en registerjournals, veilige configuratiesynchronisatie, begrensd scangeheugen, robuuste verzoekvalidatie en verbeterde tweetalige exports. | +| 1.1.0 | Uitgebracht | Companion met referentiecontrole en native Reparaties. | +| 1.1.1 | Uitgebracht | HACS-installatie en instructies. | +| 1.1.2 | Voorbereid | Atomaire quarantaine, diagnostiek, lijstverversing en documentatie/vertalingen. | ### Praktijkcontrole voor vroege testers -- 1.0.2 installeren of bijwerken via de echte GitHub/GHCR-workflow. +- De nieuwe release installeren of bijwerken via de echte GitHub/GHCR- en HACS-workflow. - Volledige back-up starten en voltooiing op Home Assistant OS verifiëren. - Zowel een veilige als een bewust gekozen reviewkandidaat naar quarantaine verplaatsen. - Hersteltest uitvoeren en hetzelfde bestand werkelijk terugplaatsen. @@ -35,7 +40,7 @@ Veiligheid gaat vóór extra opruimwinst. Een release gaat pas door wanneer risi - Minimaal enkele gebruikerstests uitvoeren met verschillende integraties en opslagprofielen. - Bekende problemen documenteren en alle releaseblokkerende fouten oplossen. -### 1.0.0 — Eerste stabiele versie +### 1.0.0 — Eerste publieke testversie - Gepubliceerd voor gecontroleerde vroege tests; meld onverwacht gedrag met het leesbare en technische scanrapport. - Stabiel quarantaine- en herstelcontract. @@ -49,7 +54,7 @@ Veiligheid gaat vóór extra opruimwinst. Een release gaat pas door wanneer risi - Betere opslagtrends en geschatte winst per integratie. - Optionele meldingen voor verlopen quarantaine en langdurige entityproblemen. - Uitbreiding naar extra talen via losse vertaalbestanden. -- Entityverwijdering uitsluitend onderzoeken als Home Assistant daarvoor een officiële, controleerbare en herstelbare API biedt. +- Registerverwijdering blijft een bewuste keuze; herstel vereist een Home Assistant-back-up. --- @@ -69,12 +74,15 @@ Safety takes priority over reclaimed space. A release proceeds only when risky s | 0.9.0 | Completed | Supervisor backup verification, restore testing, retention and explicit post-expiry deletion. | | 0.9.1 | Completed | Permitted backup verification, reusable evidence, an explicit optional backup choice and selectable entity reviews. | | 1.0.0 | Released | User-directed cleanup, registry execution, personal content, clean-start controls and hard core-system protection. | -| 1.0.1 | Prepared | Technical hardening: crash-safe execution, pagination, atomic storage, accessibility and reliable frontend caching. | -| 1.0.2 | Prepared | Recovery and registry journals, safe configuration synchronization, bounded scan memory, robust request validation and improved bilingual exports. | +| 1.0.1 | Released | Technical hardening: crash-safe execution, pagination, atomic storage, accessibility and reliable frontend caching. | +| 1.0.2 | Released | Recovery and registry journals, safe configuration synchronization, bounded scan memory, robust request validation and improved bilingual exports. | +| 1.1.0 | Released | Companion reference checks and native Repairs. | +| 1.1.1 | Released | HACS installation support and instructions. | +| 1.1.2 | Prepared | Atomic quarantine, diagnostics, list refresh and documentation/translations. | ### Practical checks for early testers -- Install or update 1.0.2 through the real GitHub/GHCR workflow. +- Install or update the new release through the real GitHub/GHCR and HACS workflows. - Start a full backup and verify completion on Home Assistant OS. - Move both a safe file and an explicitly accepted review candidate into quarantine. - Run the restore test and restore that file to its original location. @@ -86,7 +94,7 @@ Safety takes priority over reclaimed space. A release proceeds only when risky s - Run user tests across several integrations and storage profiles. - Document known issues and resolve every release-blocking defect. -### 1.0.0 — First stable release +### 1.0.0 — First public test release - Released for controlled early testing; report unexpected behaviour with the readable and technical scan reports. - Stable quarantine and recovery contract. @@ -100,4 +108,4 @@ Safety takes priority over reclaimed space. A release proceeds only when risky s - Better storage trends and estimated savings per integration. - Optional notifications for expired quarantine and persistent entity problems. - Additional languages through separate translation files. -- Consider entity deletion only if Home Assistant provides an official, verifiable and recoverable API. +- Keep registry deletion user-directed; recovery requires a Home Assistant backup. diff --git a/custom_components/hass_cleaner/__init__.py b/custom_components/hass_cleaner/__init__.py index 80365b1..b1aca46 100644 --- a/custom_components/hass_cleaner/__init__.py +++ b/custom_components/hass_cleaner/__init__.py @@ -3,6 +3,7 @@ import asyncio import logging +import traceback from datetime import timedelta import voluptuous as vol @@ -84,21 +85,29 @@ async def refresh(self, _now=None): self.report = {"schema_version": 1, "status": "starting", "summary": {}, "sources": [], "references": []} return try: + stage = "sources" sources = await collect_sources(hass) + stage = "registries" known = {"entity": set(er.async_get(hass).entities) | set(hass.states.async_entity_ids()), "entity_aliases": {e.id: e.entity_id for e in er.async_get(hass).entities.values()}, "device": set(dr.async_get(hass).devices), "area": set(ar.async_get(hass).areas), "action": {f"{domain}.{service}" for domain, services in hass.services.async_services().items() for service in services}} + stage = "statistics" known["statistic"] = await statistic_ids(hass) + stage = "analysis" report = await hass.async_add_executor_job(analyze, sources, known) if self.closed: return + stage = "repairs" self.reconcile(report) self.report = report except Exception as exc: if not self.report or self.report.get("status") != "unavailable": - _LOGGER.warning("Reference check could not complete (%s)", type(exc).__name__) + # Code locations only: no exception text, source lines or locals. + locations = " -> ".join(f"{frame.name}:{frame.lineno}" for frame in traceback.extract_tb(exc.__traceback__)) + _LOGGER.warning("Reference check failed: stage=%s type=%s locations=%s", stage, type(exc).__name__, locations) self.report = {"schema_version": 1, "status": "unavailable", "summary": {}, "sources": [], "references": [], + "reason": "check_failed", "stage": stage, "error": "Reference check failed; previous repair issues have been retained"} @callback diff --git a/custom_components/hass_cleaner/manifest.json b/custom_components/hass_cleaner/manifest.json index 03ed52a..e5ff8e7 100644 --- a/custom_components/hass_cleaner/manifest.json +++ b/custom_components/hass_cleaner/manifest.json @@ -9,5 +9,5 @@ "iot_class": "local_polling", "issue_tracker": "https://github.com/dkwolf1/Hass-Cleaner/issues", "requirements": [], - "version": "1.1.1" + "version": "1.1.2" } diff --git a/custom_components/hass_cleaner/references.py b/custom_components/hass_cleaner/references.py index dad2897..84f3cac 100644 --- a/custom_components/hass_cleaner/references.py +++ b/custom_components/hass_cleaner/references.py @@ -87,7 +87,10 @@ def walk(value, location="$", key="", depth=0, statistic_context=False): record["status"] = "partial" return if isinstance(value, dict): - statistic_context = statistic_context or (record["kind"] == "dashboard" and value.get("type") in {"statistic", "statistics-graph"}) + card_type = value.get("type") + if record["kind"] == "dashboard" and card_type is not None and not isinstance(card_type, str): + record["status"] = "partial" + statistic_context = statistic_context or (record["kind"] == "dashboard" and isinstance(card_type, str) and card_type in {"statistic", "statistics-graph"}) if key == "entities" and record["kind"] == "scene": for entity_id in value: add("entity", entity_id, f"{location}[{entity_id}]") @@ -140,8 +143,14 @@ def walk(value, location="$", key="", depth=0, statistic_context=False): if source.get("error") or not isinstance(source.get("config"), dict): record.update(status="unavailable", error=str(source.get("error") or "Configuration is unavailable")) else: - walk(source["config"]) - if record["dynamic"] or record["custom_cards"]: + start = len(references) + try: + walk(source["config"]) + except Exception: + # Never expose config values or resolve issues for failed sources. + del references[start:] + record.update(status="unavailable", error="Source analysis failed") + if record["status"] != "unavailable" and (record["dynamic"] or record["custom_cards"]): record["status"] = "partial" coverage.append(record) references.sort(key=lambda r: (r["source_id"], r["location"], r["target_id"])) diff --git a/docs/reference-checks.md b/docs/reference-checks.md index bc3ab23..572618b 100644 --- a/docs/reference-checks.md +++ b/docs/reference-checks.md @@ -1,6 +1,6 @@ # Reference checks and Home Assistant Repairs -Development preview for Hass-Cleaner 1.1.0. Keep a complete backup and test on a non-critical installation first. This feature does not provide Spook feature parity or prove that anything is safe to delete. +Reference checks were introduced in 1.1.0; HACS support shipped in 1.1.1. This checkout prepares the unreleased 1.1.2 fixes. Keep a complete backup and test on a non-critical installation first. This feature does not provide Spook feature parity or prove that anything is safe to delete. ## Two components @@ -26,7 +26,7 @@ HACS manages only `custom_components/hass_cleaner`, not the app container. Insta **Already installed from ZIP?** Back up first, then download the same integration through HACS and restart Core. Keep the existing integration entry; do not remove/re-add it or create a second nested folder. HACS may replace local edits to the integration files. -**Publishing note:** the HACS metadata must be merged to the default branch. Publish a new release containing these changes for the fully packaged installation route; existing release tags are not updated by a merge. Do not move an existing release tag. Installation through a real HACS instance still needs verification. +**Publishing note:** HACS support is available from release 1.1.1. Future changes need a new release; existing tags are not updated by a merge. Do not move an existing release tag. Validate each update in a real Home Assistant installation. ### Manual installation (without HACS) @@ -118,9 +118,9 @@ On a disposable Home Assistant 2026.9 instance: De zeven categorieën uit deel 1 zijn aangesloten, inclusief scènes, groepen, ondersteunde helpers, losse templates en energie/statistieken. Geldige historische/externe statistieken worden niet als verdwenen entiteit gemeld. Zonder Recorder is de statistiekcontrole expliciet onbekend. Niet alle aangepaste helpers of dynamische templates zijn interpreteerbaar; praktijktests blijven nodig voordat dit releaseklaar is. -- Met HACS en Home Assistant 2026.9.0 of nieuwer: voeg `https://github.com/dkwolf1/Hass-Cleaner` toe via **HACS → menu (⋮) → Aangepaste repositories**, type **Integratie**. Download **Hass-Cleaner Companion**, herstart Home Assistant Core en voeg de integratie toe via **Instellingen → Apparaten & diensten**. Kies dezelfde releaseversie als de app. De HACS-wijzigingen moeten eerst gepubliceerd zijn; de integratie staat niet in de standaardcatalogus. +- Met HACS en Home Assistant 2026.9.0 of nieuwer: voeg `https://github.com/dkwolf1/Hass-Cleaner` toe via **HACS → menu (⋮) → Aangepaste repositories**, type **Integratie**. Download **Hass-Cleaner Companion**, herstart Home Assistant Core en voeg de integratie toe via **Instellingen → Apparaten & diensten**. Kies dezelfde releaseversie als de app. HACS-ondersteuning is beschikbaar vanaf 1.1.1; de integratie staat niet in de standaardcatalogus. - De ZIP blijft een alternatief zonder HACS. Al handmatig geïnstalleerd? Maak een back-up, download via HACS en herstart Core; behoud de bestaande integratie. Werk voortaan de companion via HACS bij en de app via de App Store. - Start een nieuwe appscan. Onder **Entiteiten → Referentiecontrole** staan bronnen, doelen, exacte configuratiepaden en beperkte dekking. - Meldingen staan bij **Instellingen → Systeem → Reparaties**. Corrigeer zelf de configuratie; de companion controleert iedere vijf minuten. Er wordt niets automatisch hersteld of verwijderd. -- App en companion zijn afzonderlijke onderdelen en moeten afzonderlijk worden bijgewerkt. Versie 1.1.0 is nog een ontwikkelversie; praktijktests zijn nodig. +- App en companion zijn afzonderlijke onderdelen en moeten afzonderlijk worden bijgewerkt. 1.1.1 is gepubliceerd; deze checkout bereidt 1.1.2 voor. Praktijktests blijven nodig. - Geen verwijzingen gevonden betekent **niet** veilig verwijderen. Templates, blueprints, aangepaste kaarten en onleesbare bronnen kunnen gebruik verbergen. Maak een volledige back-up en controleer de gevolgen. diff --git a/hass_cleaner/CHANGELOG.md b/hass_cleaner/CHANGELOG.md index 474516a..6472517 100644 --- a/hass_cleaner/CHANGELOG.md +++ b/hass_cleaner/CHANGELOG.md @@ -1,5 +1,18 @@ # Changelog +## 1.1.2 — Unreleased + +- Use atomic, same-filesystem quarantine moves in protected configuration storage instead of copy/delete; retain compatibility with existing app-storage quarantine and preserve concurrent writes for recovery. +- Refresh the visible paginated list after a completed scan. +- Distinguish reference-bridge timeouts, connection failures, permission refusals and missing companion commands without exposing server error text. +- Align release documentation and complete the reviewed English entity-detail translations. + +- Fetch entity results when switching from the empty attention view to temporary signals. +- Handle non-scalar dashboard type fields and isolate unexpected source-analysis failures; failed sources remain explicitly unavailable and retain their Repairs. +- Report companion failure stages in the app and log code locations without exception messages or configuration values. +- Fill visible English translation gaps in group actions, bundle review and entity details. +- Live verification is still required: the reported TypeError did not include a traceback, so the reproduced dashboard defect is not yet confirmed as the cause on the affected installation. + ## 1.1.1 - Add HACS custom-repository metadata for the optional companion (Home Assistant 2026.9.0 minimum), with English and Dutch installation and migration guidance. diff --git a/hass_cleaner/DOCS.md b/hass_cleaner/DOCS.md index 7ce35d1..9f87d3e 100644 --- a/hass_cleaner/DOCS.md +++ b/hass_cleaner/DOCS.md @@ -4,6 +4,8 @@ Storage auditing and user-directed cleanup for Home Assistant OS. Handle with ca ## Getting started +Published release: **1.1.1**. This checkout prepares **1.1.2**. In 1.1.2, new quarantine files stay in the configuration directory under `.hass-cleaner-quarantine`, outside subsequent scans. Atomic moves preserve the original file; cross-filesystem moves are rejected. Existing app-storage quarantine remains supported. Back up both configuration and app data (which holds the manifest). Quarantine itself does not reclaim space. Do not remove the hidden folder manually; stop active writers before moving files. A changed checksum requires manual investigation, not forced deletion. + For optional automation/script/dashboard reference checks and native Repairs, install [Hass-Cleaner Companion](https://github.com/dkwolf1/Hass-Cleaner/blob/main/docs/reference-checks.md) separately. App 1.1.0 displays its scan snapshots; without it, the app explicitly reports that reference checks are unavailable. Existing cleanup features continue to work. 1. Start the app, enable **Show in sidebar**, and open its interface. diff --git a/hass_cleaner/config.yaml b/hass_cleaner/config.yaml index 43f0acb..918e07c 100644 --- a/hass_cleaner/config.yaml +++ b/hass_cleaner/config.yaml @@ -1,5 +1,5 @@ name: Hass-Cleaner -version: "1.1.1" +version: "1.1.2" slug: hass_cleaner description: Storage audit and controlled cleanup for Home Assistant OS url: https://github.com/dkwolf1/Hass-Cleaner diff --git a/hass_cleaner/hass_cleaner/__init__.py b/hass_cleaner/hass_cleaner/__init__.py index 7b68e38..9265426 100644 --- a/hass_cleaner/hass_cleaner/__init__.py +++ b/hass_cleaner/hass_cleaner/__init__.py @@ -1,3 +1,3 @@ """Hass-Cleaner Home Assistant App.""" -__version__ = "1.1.1" +__version__ = "1.1.2" diff --git a/hass_cleaner/hass_cleaner/policy.py b/hass_cleaner/hass_cleaner/policy.py index c7d5de9..a33ceb2 100644 --- a/hass_cleaner/hass_cleaner/policy.py +++ b/hass_cleaner/hass_cleaner/policy.py @@ -49,6 +49,8 @@ def classify(root: Path, path: Path, mode: int, modified: float, *, min_temp_age return Classification("symlink", RISK_PROTECTED, "Symbolische links worden nooit gevolgd", "none") if not stat.S_ISREG(mode): return None + if ".hass-cleaner-quarantine" in parts: + return Classification("quarantine_storage", RISK_PROTECTED, "Quarantine recovery data is protected", "none") if ".storage" in parts: return Classification("home_assistant_storage", RISK_PROTECTED, ".storage is absoluut beschermd", "none") if len(parts) == 1 and name in CORE_FILES: diff --git a/hass_cleaner/hass_cleaner/quarantine.py b/hass_cleaner/hass_cleaner/quarantine.py index b053586..c19994d 100644 --- a/hass_cleaner/hass_cleaner/quarantine.py +++ b/hass_cleaner/hass_cleaner/quarantine.py @@ -115,7 +115,9 @@ def _execute( scan_map = {item.id: item for item in scan.items} operation_id = uuid.uuid4().hex - operation_root = self.root / operation_id / "files" + # Keep the original inode: copying and then unlinking can lose writes + # made by another process between those two operations. + operation_root = self._contained_path(self.config_root, Path(".hass-cleaner-quarantine") / operation_id / "files", "Invalid quarantine storage path") prepared: list[tuple[Path, Path, dict[str, Any]]] = [] now = datetime.now(timezone.utc) @@ -133,6 +135,8 @@ def _execute( raise QuarantineError(f"Bestand is sinds de scan verdwenen: {item.path}") from exc if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode): raise QuarantineError(f"Bestandstype is sinds de scan gewijzigd: {item.path}") + if metadata.st_dev != self.config_root.stat().st_dev: + raise QuarantineError("Atomic quarantine across filesystems is not supported; no files were moved") expected_mtime = datetime.fromisoformat(item.modified_at.replace("Z", "+00:00")).timestamp() if metadata.st_size != item.size_bytes or abs(metadata.st_mtime - expected_mtime) >= 1: raise QuarantineError(f"Bestand is sinds de scan gewijzigd: {item.path}") @@ -168,25 +172,24 @@ def _execute( operation_id, scan.id, backup_token or backup_choice, requested_by, now, settings, records, "preparing", backup_choice, ) + operation["storage"] = "config" # The complete intent is durable before the first source file is touched. self._insert_operation(operation) try: for source, destination, record in prepared: destination.parent.mkdir(parents=True, exist_ok=True) - temporary = destination.with_name(destination.name + ".copying") - with source.open("rb") as input_file, temporary.open("xb") as output_file: - shutil.copyfileobj(input_file, output_file, 1024 * 1024) - output_file.flush() - os.fsync(output_file.fileno()) - source_hash = _sha256(source) - if _sha256(temporary) != source_hash: - temporary.unlink(missing_ok=True) - raise QuarantineError(f"Checksumcontrole mislukt voor {record['original_path']}") - temporary.replace(destination) - record.update({"sha256": source_hash, "status": "copied"}) - operation["status"] = "running" - self._replace_operation(operation) - source.unlink() + if destination.exists() or destination.is_symlink(): + raise QuarantineError("Quarantine destination already exists") + if not self._matches_regular_file(source, record["sha256"]): + raise QuarantineError("File changed during quarantine; scan again") + # No copy/unlink fallback (including EXDEV). A late writer + # retains the same inode in quarantine instead of losing data. + os.rename(source, destination) + _sync_directory(source.parent) + _sync_directory(destination.parent) + if not self._matches_regular_file(destination, record["sha256"]): + record["status"] = "recovery_required" + raise QuarantineError("File changed during quarantine; moved data has been preserved for recovery") record["status"] = "quarantined" self._replace_operation(operation) except Exception as exc: @@ -304,11 +307,12 @@ def clear_completed_history(self) -> int: or any(file.get("status") in retained_statuses for file in item.get("files", [])) ] removed = len(operations) - len(active) + roots = {item["id"]: self._operation_files_root(item["id"]).parent for item in operations} self._save(active) for operation in operations: if operation not in active: try: - operation_root = self._operation_root(str(operation.get("id", ""))) + operation_root = roots[operation["id"]] except QuarantineError: continue shutil.rmtree(operation_root, ignore_errors=True) @@ -350,7 +354,9 @@ def _load(self) -> list[dict[str, Any]]: try: value = json.loads(self.manifest_path.read_text(encoding="utf-8")) except FileNotFoundError as exc: - if self.root.exists() and any(path.is_dir() for path in self.root.iterdir()): + local = self.config_root / ".hass-cleaner-quarantine" + if ((self.root.exists() and any(path.is_dir() for path in self.root.iterdir())) + or (local.exists() and any(local.iterdir()))): raise QuarantineError("Quarantine manifest is missing; existing recovery data has been preserved") from exc return [] except (OSError, ValueError) as exc: @@ -364,6 +370,8 @@ def _load(self) -> list[dict[str, Any]]: or not isinstance(operation.get("files"), list) or not isinstance(operation.get("status"), str)): raise QuarantineError("Quarantine manifest has an invalid operation") + if operation.get("storage") not in (None, "config"): + raise QuarantineError("Unknown quarantine storage; recovery data preserved") ids.add(operation["id"]) file_ids = set() file_paths = set() @@ -451,7 +459,11 @@ def _reconcile_incomplete_operations(self) -> None: changed = True continue source_present = source.exists() or source.is_symlink() - if destination_ok and not source_present: + if operation.get("storage") == "config" and (destination.exists() or destination.is_symlink()): + # The original name may already have been recreated. Never + # delete either file when recovering an atomic move. + record["status"] = "quarantined" if destination_ok else "recovery_required" + elif destination_ok and not source_present: record["status"] = "quarantined" elif source_ok: if destination_ok: @@ -563,6 +575,9 @@ def _operation_root(self, operation_id: str) -> Path: return self._contained_path(self.root, Path(operation_id), "Ongeldig quarantaine-ID") def _operation_files_root(self, operation_id: str) -> Path: + operation = self._find_operation(operation_id) + if operation.get("storage") == "config": + return self._contained_path(self.config_root, Path(".hass-cleaner-quarantine") / operation_id / "files", "Invalid quarantine path") return self._contained_path(self._operation_root(operation_id), Path("files"), "Ongeldig quarantainepad") diff --git a/hass_cleaner/hass_cleaner/references.py b/hass_cleaner/hass_cleaner/references.py index 2a848a2..0967681 100644 --- a/hass_cleaner/hass_cleaner/references.py +++ b/hass_cleaner/hass_cleaner/references.py @@ -2,7 +2,11 @@ from __future__ import annotations import json +import logging import os +from websocket import WebSocketTimeoutException, WebSocketConnectionClosedException + +_LOGGER = logging.getLogger(__name__) def unavailable(reason="companion_unavailable"): @@ -11,7 +15,7 @@ def unavailable(reason="companion_unavailable"): def fetch_references(token, *, connect=None): - from .registry_audit import WEBSOCKET_URL, _receive_json, _receive_result + from .registry_audit import WEBSOCKET_URL, _receive_json, _receive_result, HomeAssistantApiError connection = None try: if connect is None: @@ -44,18 +48,31 @@ def fetch_references(token, *, connect=None): or not all(isinstance(item.get(key), str) for key in ("source_id", "source_kind", "source_name", "target_id", "location"))): return unavailable("invalid_report") - return {key: value for key, value in report.items() if key in { + result = {key: value for key, value in report.items() if key in { "schema_version", "status", "checked_at", "sources", "references", "summary", "limitations"}} + if report.get("reason") == "check_failed": + result["reason"] = "check_failed" + if report.get("stage") in ("sources", "registries", "statistics", "analysis", "repairs"): + result["stage"] = report["stage"] + return result + except (TimeoutError, WebSocketTimeoutException): + reason = "timeout" + except HomeAssistantApiError as exc: + reason = {"unauthorized": "permission_denied", "unknown_command": "companion_missing", + "not_loaded": "companion_not_loaded"}.get(exc.code if isinstance(exc.code, str) else "", "protocol_error") + except (OSError, WebSocketConnectionClosedException): + reason = "connection_failed" except Exception: - # Missing companion, permission refusal and connectivity failures never - # masquerade as a successful check or break the normal registry scan. - return unavailable() + reason = "unexpected_error" finally: if connection is not None: try: connection.close() except Exception: pass + # Fixed diagnostic codes only: never log token, server error text or URLs. + _LOGGER.warning("Reference bridge unavailable: %s", reason) + return unavailable(reason) def selection_review(audit, entity_ids=(), device_ids=()): diff --git a/hass_cleaner/hass_cleaner/registry_audit.py b/hass_cleaner/hass_cleaner/registry_audit.py index 02669af..91ca037 100644 --- a/hass_cleaner/hass_cleaner/registry_audit.py +++ b/hass_cleaner/hass_cleaner/registry_audit.py @@ -13,7 +13,9 @@ class HomeAssistantApiError(RuntimeError): - pass + def __init__(self, message, *, code=None): + super().__init__(message) + self.code = code @dataclass(frozen=True) @@ -532,7 +534,8 @@ def _receive_result(connection: Any, command_id: int) -> dict[str, Any]: if response.get("type") != "result" or not response.get("success"): error = response.get("error") message = error.get("message") if isinstance(error, dict) else None - raise HomeAssistantApiError(message or f"WebSocket-commando {command_id} mislukte") + raise HomeAssistantApiError(message or f"WebSocket-commando {command_id} mislukte", + code=error.get("code") if isinstance(error, dict) else None) return response diff --git a/hass_cleaner/hass_cleaner/scanner.py b/hass_cleaner/hass_cleaner/scanner.py index 6b98946..e861d00 100644 --- a/hass_cleaner/hass_cleaner/scanner.py +++ b/hass_cleaner/hass_cleaner/scanner.py @@ -96,6 +96,8 @@ def scan_tree(root: Path, settings: Settings, scan_id: str | None = None) -> Sca except (OSError, PermissionError, FileNotFoundError): continue if entry.is_dir(follow_symlinks=False): + if entry.name == ".hass-cleaner-quarantine": + continue stack.append(path) continue result.visited_files += 1 diff --git a/hass_cleaner/hass_cleaner/server.py b/hass_cleaner/hass_cleaner/server.py index b3c618f..caf1cd4 100644 --- a/hass_cleaner/hass_cleaner/server.py +++ b/hass_cleaner/hass_cleaner/server.py @@ -616,7 +616,8 @@ def _paged_scan_items(scan, kind: str, query: dict[str, list[str]]) -> dict[str, return {"items": items[offset:offset + limit], "total": len(items), "offset": offset, "source_counts": source_counts, "limit": limit, "has_more": offset + limit < len(items), "status": report.get("status", "unavailable"), "checked_at": report.get("checked_at"), - "summary": report.get("summary", {}), "sources": report.get("sources", [])} + "summary": report.get("summary", {}), "sources": report.get("sources", []), + "reason": report.get("reason"), "stage": report.get("stage")} if kind == "entities": all_items = list(scan.registry_audit.entity_workspace.get("items", [])) items = all_items diff --git a/hass_cleaner/tests/test_companion.py b/hass_cleaner/tests/test_companion.py index 3cc1fbf..efb55c2 100644 --- a/hass_cleaner/tests/test_companion.py +++ b/hass_cleaner/tests/test_companion.py @@ -117,7 +117,11 @@ async def test_waits_for_startup_and_failed_refresh_retains_issues(self): with self.assertLogs(self.companion._LOGGER, "WARNING") as logs: await self.monitor.refresh() self.assertNotIn("SECRET", str(logs.output)) + self.assertIn("stage=sources", str(logs.output)) + self.assertIn("locations=", str(logs.output)) self.assertEqual("unavailable", self.monitor.report["status"]) + self.assertEqual("sources", self.monitor.report["stage"]) + self.assertEqual("check_failed", self.monitor.report["reason"]) self.assertEqual(1, len(self.issues)) async def test_refresh_runs_analyzer_off_event_loop(self): diff --git a/hass_cleaner/tests/test_quarantine.py b/hass_cleaner/tests/test_quarantine.py index b02a49a..e4cbe2d 100644 --- a/hass_cleaner/tests/test_quarantine.py +++ b/hass_cleaner/tests/test_quarantine.py @@ -7,6 +7,7 @@ import unittest import hashlib import json +from unittest.mock import patch from dataclasses import replace from datetime import datetime, timezone from datetime import timedelta @@ -19,6 +20,63 @@ class QuarantineTests(unittest.TestCase): + def test_late_writer_keeps_original_inode_and_data(self): + with tempfile.TemporaryDirectory() as config_folder, tempfile.TemporaryDirectory() as data_folder: + source, scan, plan = self._fixture(Path(config_folder)) + manager = QuarantineManager(Path(config_folder), Path(data_folder)) + rename = os.rename + moved = [] + def late_writer(src, dst): + Path(src).write_bytes(b'late concurrent content') + rename(src, dst) + moved.append(Path(dst)) + Path(src).write_bytes(b'recreated source') + with patch('hass_cleaner.quarantine.os.rename', side_effect=late_writer): + with self.assertRaises(QuarantineError): + manager.execute(scan, Settings(), plan=plan, backup_token='ok', backup_valid=True, + backup_choice='verified', risk_acknowledged=False, + confirmation='QUARANTINE', requested_by='test') + self.assertEqual(b'late concurrent content', moved[0].read_bytes()) + self.assertEqual(b'recreated source', source.read_bytes()) + restarted = QuarantineManager(Path(config_folder), Path(data_folder)) + self.assertEqual('recovery_required', restarted.list()[0]['files'][0]['status']) + self.assertEqual(0, restarted.clear_completed_history()) + + def test_cross_filesystem_error_has_no_copy_fallback(self): + import errno + with tempfile.TemporaryDirectory() as config_folder, tempfile.TemporaryDirectory() as data_folder: + source, scan, plan = self._fixture(Path(config_folder)) + manager = QuarantineManager(Path(config_folder), Path(data_folder)) + with patch('hass_cleaner.quarantine.os.rename', side_effect=OSError(errno.EXDEV, 'cross-device')): + with self.assertRaises(QuarantineError): + manager.execute(scan, Settings(), plan=plan, backup_token='ok', backup_valid=True, + backup_choice='verified', risk_acknowledged=False, + confirmation='QUARANTINE', requested_by='test') + self.assertEqual(b'safe generated cache', source.read_bytes()) + + def test_atomic_move_recovery_preserves_recreated_source(self): + from hass_cleaner.scanner import scan_tree + with tempfile.TemporaryDirectory() as config_folder, tempfile.TemporaryDirectory() as data_folder: + source, scan, plan = self._fixture(Path(config_folder)) + manager = QuarantineManager(Path(config_folder), Path(data_folder)) + save = manager._replace_operation + def interrupted_commit(operation): + if operation['files'][0]['status'] == 'quarantined': + source.write_bytes(b'new file') + raise KeyboardInterrupt() + save(operation) + with patch.object(manager, '_replace_operation', side_effect=interrupted_commit): + with self.assertRaises(KeyboardInterrupt): + manager.execute(scan, Settings(), plan=plan, backup_token='ok', backup_valid=True, + backup_choice='verified', risk_acknowledged=False, + confirmation='QUARANTINE', requested_by='test') + restarted = QuarantineManager(Path(config_folder), Path(data_folder)) + op = restarted.list()[0] + self.assertEqual('quarantined', op['files'][0]['status']) + self.assertEqual(b'new file', source.read_bytes()) + self.assertTrue(restarted.test_restore(op['id'], 'file1')['passed']) + self.assertFalse(any('.hass-cleaner-quarantine' in item.path for item in scan_tree(Path(config_folder), Settings()).items)) + def _fixture(self, config: Path) -> tuple[Path, ScanResult, dict]: source = config / "custom_components" / "demo" / "__pycache__" / "demo.cpython-313.pyc" source.parent.mkdir(parents=True) diff --git a/hass_cleaner/tests/test_recovery_journals.py b/hass_cleaner/tests/test_recovery_journals.py index 1a1c6bb..ed508ab 100644 --- a/hass_cleaner/tests/test_recovery_journals.py +++ b/hass_cleaner/tests/test_recovery_journals.py @@ -86,7 +86,7 @@ def fail_commit(value): restarted = QuarantineManager(config, data) record = restarted.list()[0]["files"][0] self.assertEqual("restored", record["status"]) - self.assertFalse((restarted.root / operation["id"] / "files" / record["relative_path"]).exists()) + self.assertFalse((restarted._operation_files_root(operation["id"]) / record["relative_path"]).exists()) self.assertFalse(list(source.parent.glob("*.hass-cleaner-restore"))) def test_restore_intent_failure_keeps_source_in_quarantine(self): @@ -125,7 +125,7 @@ def test_corrupt_manifests_are_preserved_and_block_clear(self): with self.assertRaises(QuarantineError): restarted._insert_operation(operation) self.assertEqual(contents, manager.manifest_path.read_text(encoding="utf-8")) - self.assertTrue((manager.root / operation["id"]).exists()) + self.assertTrue((config / ".hass-cleaner-quarantine" / operation["id"]).exists()) def test_unreadable_manifest_is_not_empty(self): _, _, _, manager, _ = self.fixture() diff --git a/hass_cleaner/tests/test_references.py b/hass_cleaner/tests/test_references.py index 6824fc1..675b3f1 100644 --- a/hass_cleaner/tests/test_references.py +++ b/hass_cleaner/tests/test_references.py @@ -30,6 +30,45 @@ def source(config, kind="automation", source_id="automation.test"): class ReferenceTests(unittest.TestCase): + def test_bridge_distinguishes_transport_and_command_failures(self): + from websocket import WebSocketTimeoutException + for code, reason in [('unauthorized', 'permission_denied'), ('unknown_command', 'companion_missing'), ('not_loaded', 'companion_not_loaded')]: + connection = FakeConnection([{'type': 'auth_ok'}, {'id': 1, 'type': 'result', 'success': False, 'error': {'code': code, 'message': 'SECRET'}}]) + with self.assertLogs('hass_cleaner.references', 'WARNING') as logs: + result = fetch_references('SECRET TOKEN', connect=lambda *a, **k: connection) + self.assertEqual(reason, result['reason']) + self.assertNotIn('SECRET', str(logs.output) + str(result)) + for error, reason in [(WebSocketTimeoutException('SECRET'), 'timeout'), (OSError('SECRET'), 'connection_failed')]: + def fail(*args, **kwargs): + raise error + self.assertEqual(reason, fetch_references('test', connect=fail)['reason']) + + def test_dashboard_non_scalar_type_keeps_references(self): + for card_type in (["custom"], {"nested": True}): + report = analyzer.analyze([source({"type": card_type, "entity": "sensor.a"}, kind="dashboard")], {}) + self.assertEqual("partial", report["status"]) + self.assertEqual("sensor.a", report["references"][0]["target_id"]) + + def test_failed_source_does_not_break_other_sources(self): + class BrokenValue: + @property + def template(self): + raise TypeError("SECRET CONFIGURATION") + report = analyzer.analyze([source({"entity_id": "sensor.a", "bad": BrokenValue()}), + {"id": "script.ok", "kind": "script", "name": "OK", "config": {"entity_id": "sensor.b"}}], {}) + self.assertEqual("unavailable", report["sources"][0]["status"]) + self.assertEqual(["sensor.b"], [r["target_id"] for r in report["references"]]) + self.assertNotIn("SECRET", str(report)) + + def test_bridge_preserves_safe_failure_diagnostics(self): + report = {"schema_version": 1, "status": "unavailable", "sources": [], "references": [], "summary": {}, + "reason": "check_failed", "stage": "sources", "error": "SECRET"} + connection = FakeConnection([{"type": "auth_ok"}, {"type": "result", "id": 1, "success": True, "result": report}]) + result = fetch_references("test", connect=lambda *a, **k: connection) + self.assertEqual("sources", result["stage"]) + self.assertEqual("check_failed", result["reason"]) + self.assertNotIn("SECRET", str(result)) + def test_cleanup_rechecks_unchanged_changed_and_unavailable_references(self): snapshot = {"entities": [{"entity_id": "light.a"}]} audit = audit_registry_snapshot(snapshot) diff --git a/hass_cleaner/web/assets/app.js b/hass_cleaner/web/assets/app.js index efa2033..011aed3 100644 --- a/hass_cleaner/web/assets/app.js +++ b/hass_cleaner/web/assets/app.js @@ -59,7 +59,23 @@ function referenceRows(items) { function referenceHeader(report) { const date = report.checked_at ? new Date(report.checked_at).toLocaleString(interfaceLocale()) : "—"; - return `
${escapeHtml(referenceStatus(report.status))} · ${escapeHtml(date)}
${referenceWarning()}
`; + const stages = {sources: referenceText("reading sources", "bronnen lezen"), registries: referenceText("reading registries", "registers lezen"), statistics: referenceText("reading statistics", "statistieken lezen"), analysis: referenceText("analysing references", "verwijzingen analyseren"), repairs: referenceText("updating Repairs", "Reparaties bijwerken")}; + const reasons = { + check_failed: referenceText("Companion check failed. See Home Assistant logs.", "Companion-controle mislukt. Bekijk de Home Assistant-logboeken."), + timeout: referenceText("Companion request timed out. Check Core logs and scan again.", "Companion-verzoek verlopen. Controleer de Core-logboeken en scan opnieuw."), + permission_denied: referenceText("Home Assistant refused access to the companion.", "Home Assistant weigert toegang tot de companion."), + companion_missing: referenceText("Companion command is unavailable. Install the companion and restart Core.", "Companion-commando ontbreekt. Installeer de companion en herstart Core."), + companion_not_loaded: referenceText("Add the companion under Devices & services first.", "Voeg de companion eerst toe bij Apparaten & diensten."), + connection_failed: referenceText("Connection to Home Assistant failed.", "Verbinding met Home Assistant mislukt."), + authentication_failed: referenceText("Home Assistant authentication failed.", "Home Assistant-authenticatie mislukt."), + protocol_error: referenceText("Unexpected Home Assistant response. Check app and companion versions.", "Onverwacht Home Assistant-antwoord. Controleer de app- en companionversies."), + invalid_report: referenceText("Invalid companion report. Check app and companion versions.", "Ongeldig companionrapport. Controleer de app- en companionversies."), + unsupported_report: referenceText("Unsupported companion report version.", "Niet-ondersteunde companionrapportversie."), + unexpected_error: referenceText("Unexpected reference connection error. Check the app logs.", "Onverwachte fout in de referentieverbinding. Controleer het applogboek.") + }; + const diagnosis = reasons[report.reason] || ""; + const detail = diagnosis ? `${escapeHtml(diagnosis)}${stages[report.stage] ? ` (${escapeHtml(stages[report.stage])})` : ""}
` : ""; + return `${escapeHtml(referenceStatus(report.status))} · ${escapeHtml(date)}
${detail}${referenceWarning()}
`; } function renderReferenceOverview() { @@ -385,6 +401,10 @@ function finishScanSummary(scan, showCompletionToast = false) { if (persistenceErrors.length) showToast(persistenceErrors[0], true); $("#select-all-safe").disabled = !(state.guidance?.safe_recipes || []).length; loadScanHistory(); + const activeTab = $(".tab.active")?.dataset.tab; + if (activeTab === "entities") loadEntitiesPage(true); + if (activeTab === "registry") loadBundlesPage(true); + if (activeTab === "results") loadFilesPage(true); if (showCompletionToast) showToast("Veilige scan voltooid"); } @@ -729,7 +749,7 @@ function renderEntities() { $("#entity-show-watch").addEventListener("click", () => { $("#entity-status-filter").value = "watch"; $("#entity-group-filter").value = "integration"; - renderEntities(); + loadEntitiesPage(true); }); } else { list.innerHTML = '