Rocket U2 | UniVerse & UniData

 View Only
  • 1.  Windows ReFS

    ROCKETEER
    Posted 30 days ago

    Any one using the Windows ReFS file system for either UniData or UniVerse? What's your experience performance wise?



    ------------------------------
    Christian Bristow
    TSE
    Rocket Software Inc
    Waltham MA US
    ------------------------------


  • 2.  RE: Windows ReFS

    Posted 10 days ago
    Edited by John Jenkins 10 days ago

    Christian,

    There are some references to ReFS being very memory intensive and using block-writes for metadata and parity - see https://norcliffe.wordpress.com/2019/08/15/extremely-slow-windows-server-with-refs/. It looks like Microsoft have made some effort to address some of the issues in patches and by adding new registry tunables - see https://learn.microsoft.com/en-gb/troubleshoot/windows-server/backup-and-storage/fix-heavy-memory-usage-refs .

    While not definitive, it suggests that a system using ReFS will need memory in proportion to the available disk churn and/or file volatility. With random-access files being generally heavily populated but updates sparsely distributed this suggests the memory load could be disproportionate compared to the volume of volatile data due to block caching in ReFS metadata and its extensive use of memory.

    It should be noted as well that ReFS uses checkpointing and depending upon details of the the implementation and ReFS buffer volatility, pauses in updates and/or disk access may occur at checkpoint intervals for the duration of a checkpoint. While for BATCH and STREAMED updates this may give better overall performance, for random access/interactive updates under interactive load occasional pauses are likely to be less acceptable.

    Additionally, while an independent flat multi-threaded I/O comparison of NTFS against ReFS gave a performance penalty of about 7% for ReFS. this figure should be treated with caution and may be a best case (registry tuning excepted). 

    Microsoft guidelines recommend that under high workload in random-access environments for example a triple-mirror be utilised -see https://learn.microsoft.com/en-us/windows-server/storage/refs/refs-overview

     Circumstantially this advice would seem to try and make up for notably slower random access performance under ReFS and the additional hardware is an attempt to close the gap to some degree. 

    Bottom line:

    There are enough questions around ReFS mamory demands and metadata/buffer management that a full benchmark would seem warranted under interactive and batch workloads at peak-busy times with the ability to tune the Microsoft newly-added tunables to coax best performance. Examination by others of ReFS performance seems to fall in line with this as a general recommendation.

    For reference, the registry entries are :

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
    Value Name: RefsEnableLargeWorkingSetTrim
    Set RefsEnableLargeWorkingSetTrim = 1
    Value Type: REG_DWORD

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
    Value Name: RefsNumberOfChunksToTrim
    Value Type: REG_DWORD
    DEFAULT (if not set or 0): 4

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem
    Value Name: RefsEnableInlineTrim
    Value Type: REG_DWORD
    Set RefsEnableInlineTrim = 1

    Benchmarking by others notes that ReFS with File Integrity ON changes the way that I/O requests are ordered - i.e. ReFS with FileIntegrity on changes the sequence of commands, changes requests and  I/O type, so it removes many random writes. ReFS in this case working similarly toa  Log Structured File System. As a result, there's a strong leap of IOPS, then IOPS fall, and the garbage collector is launched. Circumstantially the amount of residual free disk space  has been reported to be a singuler  issue in some tests. 

    Not definitive in any way I know - it raises more questions though they are the rigfht type of question I think. While it is possible to adopt a 'suck it and see' approach, caution should be exercised as performancer could vary markedly between peak-busy interactive and batch workloads. Third-party test results to date indicate significantly more memory would be indicated for ReFS disk management and Microsoft (untested) guidelines recommend mirroring - possibly  triple-mrroring. 

    Regards

    JJ



    ------------------------------
    John Jenkins
    Thame, Oxfordshire
    ------------------------------