Open-source Languages & Tools for z/OS

 View Only
Expand all | Collapse all

Handling very large git repositories on z/OS

  • 1.  Handling very large git repositories on z/OS

    Posted 04-21-2017 09:04

    If you attempt to clone or checkout a very large git repository (one with a great deal of history), you may get this message:

    bash-2.03$ git checkout --track -b v0.12-zos-ascii origin/v0.12-zos-ascii
    fatal: Out of memory? mmap failed: EDC5124I Too many open files. (errno2=0x07360344)
    bash-2.03$
    

    The problem is that git, by default, attempts to mmap the entire pack file (in the .git/objects/pack directory) into memory, and in this case that’s a very large file (about 235MB):

    bash-2.03$ ls -l .git/objects/pack
    total 494592
    -r--r--r--   1 jcallen  ussuser  7578432 Jan 26 11:23 pack-13c65cae0f429287bcff36df4d582d30125aa766.idx
    -r--r--r--   1 jcallen  ussuser  245499367 Jan 26 11:23 pack-13c65cae0f429287bcff36df4d582d30125aa766.pack
    bash-2.03$
    

    The z/OS port is 31-bit program, so that represents a sizable chunk of the entire 2GB address space.
    Fortunately, git provides some knobs to control memory use. A stackoverflow item provided the necessary clues. I used these commands to crank down the memory usage, and was then able to get the checkout to run:

    git config pack.packSizeLimit 20m
    git config core.packedGitWindowSize 16m
    git config core.packedGitLimit 32m
    git config pack.windowMemory 32m
    git config pack.thread 1
    git config pack.deltaCacheSize 1m


  • 2.  RE: Handling very large git repositories on z/OS

    Posted 04-28-2017 10:53

    Are you planning a 64-bit git release?