From 2c1e63fcfda65ffdf2bf86fd71d26a8889ab9715 Mon Sep 17 00:00:00 2001 From: Jinjiang Tu Date: Tue, 31 Mar 2026 10:43:42 -0400 Subject: [PATCH] mm/huge_memory: fix folio isn't locked in softleaf_to_folio() [ Upstream commit 4c5e7f0fcd592801c9cc18f29f80fbee84eb8669 ] On arm64 server, we found folio that get from migration entry isn't locked in softleaf_to_folio(). This issue triggers when mTHP splitting and zap_nonpresent_ptes() races, and the root cause is lack of memory barrier in softleaf_to_folio(). The race is as follows: CPU0 CPU1 deferred_split_scan() zap_nonpresent_ptes() lock folio split_folio() unmap_folio() change ptes to migration entries __split_folio_to_order() softleaf_to_folio() set flags(including PG_locked) for tail pages folio = pfn_folio(softleaf_to_pfn(entry)) smp_wmb() VM_WARN_ON_ONCE(!folio_test_locked(folio)) prep_compound_page() for tail pages In __split_folio_to_order(), smp_wmb() guarantees page flags of tail pages are visible before the tail page becomes non-compound. smp_wmb() should be paired with smp_rmb() in softleaf_to_folio(), which is missed. As a result, if zap_nonpresent_ptes() accesses migration entry that stores tail pfn, softleaf_to_folio() may see the updated compound_head of tail page before page->flags. This issue will trigger VM_WARN_ON_ONCE() in pfn_swap_entry_folio() because of the race between folio split and zap_nonpresent_ptes() leading to a folio incorrectly undergoing modification without a folio lock being held. This is a BUG_ON() before commit 93976a20345b ("mm: eliminate further swapops predicates"), which in merged in v6.19-rc1. To fix it, add missing smp_rmb() if the softleaf entry is migration entry in softleaf_to_folio() and softleaf_to_page(). [tujinjiang@huawei.com: update function name and comments] Link: https://lkml.kernel.org/r/20260321075214.3305564-1-tujinjiang@huawei.com Link: https://lkml.kernel.org/r/20260319012541.4158561-1-tujinjiang@huawei.com Fixes: e9b61f19858a ("thp: reintroduce split_huge_page()") Signed-off-by: Jinjiang Tu Acked-by: David Hildenbrand (Arm) Reviewed-by: Lorenzo Stoakes (Oracle) Cc: Barry Song Cc: Kefeng Wang Cc: Liam Howlett Cc: Michal Hocko Cc: Mike Rapoport Cc: Nanyong Sun Cc: Ryan Roberts Cc: Suren Baghdasaryan Cc: Vlastimil Babka Cc: Signed-off-by: Andrew Morton [ adapted upstream leafops.h changes to swapops.h ] Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman Signed-off-by: Ulrich Hecht --- include/linux/swapops.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 28d572b7ea73..af5efbae4478 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -208,6 +208,12 @@ static inline unsigned long migration_entry_to_pfn(swp_entry_t entry) static inline struct page *migration_entry_to_page(swp_entry_t entry) { struct page *p = pfn_to_page(swp_offset(entry)); + /* + * Ensure we do not race with split, which might alter tail pages + * into new folios and thus result in observing an unlocked page. + * This matches the write barrier in __split_huge_page_tail(). + */ + smp_rmb(); /* * Any use of migration entries may only occur while the * corresponding page is locked