Ahhh, the fun with caching in the file system. Incidentally one of trickiest bugs I encountered also had to do with the Windows Cache Manager.
It was the interaction between the Cache Manager and the Memory Manager in managing the rare transient state ModifiedNoWrite of the cache pages when dealing with reentry IO read requests. The cache page status became Modified when its content was filled in from disk, but it's marked as NoWrite to avoid being flushed out by the Memory Manager. The physical page backing the cache page can't be reused since it's dirty (Modified) but the Memory Manager can't flush it out (NoWrite). Slowly over time as more pages are read, the system would run out of physical pages.
The Cache Manager was supposed to change the cache page status back to Standby after the read returned from lower layer. But with reentry IO read requests, it won't do it when upper layer IO request buffer passed straight down. The work around was to allocate a separate buffer to interact with the Cache Manager and copied the content back to the upper layer IO request buffer, incurring an extra copy.
At the end I intimately knew about the Windows Cache Manager and Memory Manager more than I needed to know.
It was the interaction between the Cache Manager and the Memory Manager in managing the rare transient state ModifiedNoWrite of the cache pages when dealing with reentry IO read requests. The cache page status became Modified when its content was filled in from disk, but it's marked as NoWrite to avoid being flushed out by the Memory Manager. The physical page backing the cache page can't be reused since it's dirty (Modified) but the Memory Manager can't flush it out (NoWrite). Slowly over time as more pages are read, the system would run out of physical pages.
The Cache Manager was supposed to change the cache page status back to Standby after the read returned from lower layer. But with reentry IO read requests, it won't do it when upper layer IO request buffer passed straight down. The work around was to allocate a separate buffer to interact with the Cache Manager and copied the content back to the upper layer IO request buffer, incurring an extra copy.
At the end I intimately knew about the Windows Cache Manager and Memory Manager more than I needed to know.