Added FAQs about TLB use

torvalds · roscheer · Feb 10, 2017 · Feb 12, 2017 · Feb 23, 2017 · Feb 23, 2017
commit f39e6b4eda694a7b3c46049330c51b470d4c19b3
diff --git a/README.md b/README.md
@@ -10,4 +10,10 @@ With so many unknowns, the first task is to get better estimates of the potentia
 FAQ
 
 Does not transparent huge pages (https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-memory-transhuge.html) already provide a good enough solution?
-THP is feature that causes some debate.Transparent huge pages abstracts the complexities of using larger pages from developers and system administrator, but its implementation also incurs additional overhead due to continuous scanning of pages that could be merged into larger pages (khugepaged kernel thread), and splitting the pages back to smaller pages in certain situations. THP also makes the Kernel code more complex. Moreover, THP is not suited for database workloads and currently only maps anonymous memory regions such as heap and stack space. THP is disabled by default in order to avoid the risk of increasing the memory footprint of applications without a guaranteed benefit.
+THP is feature that causes some debate.Transparent huge pages abstracts the complexities of using larger pages from developers and system administrator, but its implementation also incurs additional overhead due to continuous scanning of pages that could be merged into larger pages (khugepaged kernel thread), and splitting the pages back to smaller pages in certain situations. THP also makes the Kernel code more complex. Moreover, THP is not suited for database workloads and currently only maps anonymous memory regions such as heap and stack space. THP is currently disabled by default in order to avoid the risk of increasing the memory footprint of applications without a guaranteed benefit.
+
+Wouldn't using only 2MB pages overload the Huge TLB, causing more TLB misses?
+This can only be determined precisely performing tests with typical workloads. Indeed, on modern chips such as Intel's Kaby Lake (https://en.wikichip.org/wiki/intel/microarchitectures/kaby_lake) we have 128 entries for 4KB pages on the instruction TLB versus only 8 entries for the 2MB pages (the 4KB TLB is 16 times larger than the 2MB TLB!). For data TLB, we have 64 entries for 4KB pages, versus 32 entries for 2MB pages. While there are more entries for 4KB pages, each 2MB page replaces 512 4KB pages, so the overall number of TLB misses might still be reduced, even with the higher pressure incurred on the 2MB TLBs. We also need to take into account that page tables describing 2MB page mappings use only 3 page table layers, resulting in faster handling of each TLB miss.
+
+Wouldn't the 4KB TLBs become an idle (wasted!) resource?
+That is true. The TLBs are certainly one of the most expensive components of the CPUs. But if the resulting overall system performance actually increases, why would anyone care? Moreover, processors are evolving each year. If using 2MB pages as minimum memory allocation granularity proves to be worthwhile, future x86-64 processors may include flags in their control registers to reconfigure or repurpose the 4KB TLBs to handle 2MB pages (and then we would have even larger jumps in performance).