RocketCE for Power

 View Only
Back to discussions
Expand all | Collapse all

How to use Container Images of Pytorch from RocketCE v1.8.0

  • 1.  How to use Container Images of Pytorch from RocketCE v1.8.0

    ROCKETEER
    Posted 03-16-2023 02:52

    Overview 

    Pytorch is an open source machine learning framework that accelerates the path from research prototyping to production deployment. 

    RocketCE includes container images for Pytorch v1.13.0 and are based on the packages available in channel rocketce of anaconda.org. 

    In this version of RocketCE, the container images are available for both CPU and GPU. Container images for Pytorch are available for python 3.8, 3.9 and 3.10 versions and these container images are supported for both Power 9 and Power 10 Systems of IBM with linux-ppc64le.

    CPU Images

    CPU based images are available in rocketce/pytorch-cpu. These images are pulled using the following commands

      • docker pull quay.io/rocketce/pytorch-cpu

    Available tags

      • py38-rce180 - Based on Python 3.8
      • py39-rce180 - Based on Python 3.9
      • py310-rce180 - Based on Python 3.10

    GPU Images

    CPU based images are available in rocketce/pytorch. These images are pulled using the following commands

      • docker pull quay.io/rocketce/pytorch

    Available tags

      • py38-rce180 - Based on Python 3.8
      • py39-rce180 - Based on Python 3.9
      • py310-rce180 - Based on Python 3.10

    Instructions to run RocketCE container

    RocketCE container images can be run using the following commands

    CPU based

      • docker run -it --memory="<memory" --volume <path>:/shared-volume <imageid>

    GPU based

      • docker run --gpus <gpus> -it --memory="<memory>" --volume <path>:/shared-volume <imageid>
    • Argument
      • --gpus   - GPU devices to add to the container
      • --memory  - Memory limit. This can be provided based on the memory required to run the application
      • --volume, -v Bind mount a volume
    • Value
      • gpus - "all" to pass all GPUs, 1 to pass one GPU, etc
      • memory - "5g" represents that 5GB of memory is allocated for the container
      • path - Path of the folder to be mounted on container. This path will be available in container as "/shared-volume" folder

    For more information on docker run arguments, refer to this document.

    Container can be verified by the following command

    • CPU container
      • echo 'import torch; torch.cuda.is_available();' | python
    • GPU container
      • echo 'import torch; torch.cuda.is_available();' | python

    Important packages available as part of these container are

    • CPU container
      • pytorch-cpu            - 1.13.0
      • torchtext-base      - 0.14.0
      • torchvision-base  - 0.14.0
      • openblas                   - 0.3.21
      • numpy                         - 1.23.4

    • GPU container
      • pytorch                      - 1.13.0
      • torchtext-base      - 0.14.0
      • torchvision-base  - 0.14.0
      • cudatoolkit              - 11.4.4
      • openblas                   - 0.3.21
      • numpy                         - 1.23.4



    ------------------------------
    Rajesh Nukala
    Rocket Internal - All Brands
    ------------------------------