hw: the S2 PLL lock check waits for the wrong condition - #340
Open
vrilutza wants to merge 2 commits into
Open
Conversation
The loop spins while S2_PLL_CMU_STATUS_LOCKED is set, but fthd_reg.h documents that bit as "1 = PLL locked", so it waits for the PLL to lose lock instead of to acquire it. Both of its outcomes are then reported backwards: PLL locks -> 10000 iterations, ~256 ms, "Failed to lock S2 PLL" PLL does not lock -> exits on the first read, "S2 PLL is locked after 10 us" Which one happens is decided by whether the PLL had settled by the time of the very first read, so the driver takes one of two paths at random. That matters because the error return leaves before putting the PLL into bypass, and the caller ignores it, so the DDR40 setup that follows runs against a clock configuration nothing intended. The 0xff00 mask is dropped with it: bit 15 is inside it, so it never selected anything. Measured on a MacBookPro14,1, 36 loads of each: master reports "Failed to lock" -- that is, the PLL had locked -- on 18 of 36, and skips bypass on those. With this patch, 0 of 36; the PLL reports locked after one or two polls, 10-20 us, with CMU_STATUS = 0xc902c902 and bit 15 set, and bypass is entered every time. Signed-off-by: Viorel Cernateanu <vrilutza@gmail.com>
fthd_hw_s2_pll_init() returns -EINVAL when the PLL does not lock, and on that path it returns before putting the PLL into bypass. The caller drops the return value and carries on into the DDR40 setup regardless, driving the DDR PHY from a clock the error path left half configured. Every other failure in fthd_hw_s2_init_ddr_controller_soc() aborts, so do the same here. With the previous patch the error path is not reached on this machine -- the PLL locks on every load -- so this changes nothing here. It matters where the PLL genuinely does not lock: refusing to load is better than continuing into the DDR40 setup without bypass. Signed-off-by: Viorel Cernateanu <vrilutza@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects around the S2 PLL lock check in
fthd_hw_s2_pll_init().The polling loop is inverted. It spins while
S2_PLL_CMU_STATUS_LOCKEDis set, butfthd_reg.hdocuments that bit as1 = PLL locked. So it waits for the PLL to lose lock, and both of its outcomes are reported backwards:Failed to lock S2 PLLS2 PLL is locked after 10 usWhich branch is taken depends only on whether the PLL had settled by the time of the very first read, so the driver picks one of two paths at random.
The error is dropped.
fthd_hw_s2_init_ddr_controller_soc()ignores the return value, and the error path leaves before putting the PLL into bypass — so the DDR40 setup that follows runs against a clock configuration nothing intended.Measured on a MacBookPro14,1, 36 loads of each, alternating master and patched within the same boot:
With the patch the PLL reports locked after one or two polls (10–20 µs),
CMU_STATUS = 0xc902c902, bit 15 set — the exit is on the right condition. Fisher exact, two-tailed:p = 4.4e-07.Why I went looking
This machine hard-hangs — screen frozen, no oops, nothing in
pstore, automatic reset — when the module is reloaded after the firmware has stopped responding. Two ingredients, both ordinary: a firmware timeout, which happens on its own (that is the premise of #332), and a module reload afterwards. I could reach the timeout reliably with an out-of-spec crop, which is what made it measurable.With a black box writing
/dev/kmsgto disk withfsyncon every line, the last thing before death is always inside the DDR40 VDL configuration — the code that runs right after the skipped bypass:On a healthy load the next line comes 0.02 ms later.
The 120 include 60 with
iommu.strict=1and 60 without. Fisher exact:p = 0.016.I would not claim that on its own — two incidents is thin, and this machine has a history of unexplained deaths. The inverted condition stands on its own regardless: it is wrong against the semantics in
fthd_reg.h, and it is what puts the driver on the no-bypass path in the first place.The second patch changes nothing on this machine, since the error path is unreachable once the loop is fixed. It matters where the PLL genuinely does not lock: refusing to load is better than continuing into the DDR40 setup without bypass.