Rocket U2 | UniVerse & UniData

 View Only
Expand all | Collapse all

U2PE, Python and virtual environments

  • 1.  U2PE, Python and virtual environments

    Posted 12-07-2020 11:03
    Hello Everyone,  I have installed Rocket U2/Universe Personal Edition (12.1.1/Python 3.7.3) and I'm trying to get it to work a Python virtual environment and not being successful.  

    1. Is it possible to use virtual environments with Universe Python and if so,
    2. Are there any (or can one provide) instructions for doing so?

    Thanks,

    Dave Howell

    ------------------------------
    David Howell
    Market America
    ------------------------------


  • 2.  RE: U2PE, Python and virtual environments

    ROCKETEER
    Posted 12-07-2020 17:22
    David,

    For u2py:
    It is my belief that if it is possible, I would not recommend it.  ( Note that my statement is based on my knowledge of UniVerse and Python, and I have not tried to work around the issues )

    When you use u2py to access Python, or access Python from Universe, they share the process.  When setting which version of Python to use, you must set the path to Python in the .pyconfig file.  While I guess it might be possible to set it to one active Python Virtual environment, multiples could/would be problematic.

    for uopy:
    Should be possible since each Python Virtual environment would/could act as a different client.


    ------------------------------
    Michael Rajkowski
    Rocket Software
    ------------------------------



  • 3.  RE: U2PE, Python and virtual environments

    Posted 12-08-2020 10:43

    Hello Michael,  Thanks for the info.  I think I'll just stick with the standard UV implementation for PE as it relates to U2PY and pip install all my to-import packages there. 

     

    Regards.

     






  • 4.  RE: U2PE, Python and virtual environments

    ROCKETEER
    Posted 12-08-2020 11:57
    David,

    That is how I would handle it.

    Note that if you expect to setting up the environment several times, I would recommend using:

    $ pip freeze
    and

    $ pip install -r requirements.txt


    ------------------------------
    Michael Rajkowski
    Rocket Software
    ------------------------------



  • 5.  RE: U2PE, Python and virtual environments

    Posted 12-08-2020 12:58
    Roger that.






  • 6.  RE: U2PE, Python and virtual environments

    PARTNER
    Posted 12-08-2020 18:24
    You can also use the environment variable PYTHONUSERBASE to control where your python packages get installed.
    Having them in a separate directory structure can help to isolate your package requirements from the UV-supplied python installation.

    ------------------------------
    Gregor Scott
    gregors
    ------------------------------



  • 7.  RE: U2PE, Python and virtual environments

    PARTNER
    Posted 12-09-2020 19:09

    I have not explored using virtual environments with UV's python, and given how they work I think the only way you will find success is to activate the virtual environment before you launch uv so that UV inherits the active environment setup, including the PATH settings.

    I think you would also need to be mindful to use the uv-supplied python3 or pip3 commands to ensure the virtual environment you setup uses the UV-supplied python installation. Either an explicit path to the binary, or having the UV python bin directory at the front of your PATH environment variable should help with this.

    It is a pity that there is no uv-supplied command-line launcher for python or pip to ensure you run the correct installation. Something like uvpython or uvpip would go a long way to helping you out with this.



    ------------------------------
    Gregor Scott
    gregors
    ------------------------------



  • 8.  RE: U2PE, Python and virtual environments

    ROCKETEER
    Posted 12-10-2020 14:02
    Gregor,

    While I mentioned it may be possible to set an active virtual Python environment, I do not recommend it since it would require the environment to stay active, and would have issues if someone then wanted to use multiple virtual environments.

    As for uvpip and uvpython, I do not see the benefit, if you are setting up the virtual Python environment, you would be changing the PATH environment variable accordingly.

    This is the same step you would need to do if you had several version of Python on a system and wanted to work with one particular version over another.

    Note a quick search of the web reveals that there are several options people use to manage which Python versions:
    - Aliases
    - batch/scripts
    - providing the version when launching python or pip


    ------------------------------
    Michael Rajkowski
    Rocket Software
    ------------------------------



  • 9.  RE: U2PE, Python and virtual environments

    PARTNER
    Posted 02-22-2021 00:51
    Hi Michael,

    I am also not a fan of using virtual environments with UV.

    Using either the PYTHONUSERBASE or even PYTHONPATH environment variables can be used to great effect to isolate the python module installation away from the python installation.

    I mentioned the uvpython and uvpip commands simply because the location of the python installation used by UV is configurable.
    it is not possible to totally rely on the UV python installation path being /usr/uv/python, so it is somewhat dangerous to use a hard-coded path of /usr/uv/python/bin/python3 to access the python3 binary from the python installation used by uv.

    The /usr/uv/.pyconfig file identifies the the path of the python installation:

    root@h113200# cat .pyconfig
    PYHOME=/usr/uv/python
    PYLIB=/usr/uv/python/lib/libpython3.7m.so
    root@h113200#

    So to accurately get the path to the python3 binary of the python installation path being used by UV you need to do some extra effort:

    IFS='=' read -r var UVPYPATH <<<$(grep PYHOME $(cat /.uvhome)/.pyconfig)
    ${UVPYPATH}/bin/python3

    Whilst this will work it is not an easy thing to remember each time you want to launch the UV python binary.
    So wrapping the above into scripts to correctly launch the uv python and pip commands would be very helpful.

    It would also open up the ability use use the uopy python module to interact with the UV database from outside of the UV shell, which is a new option for anyone looking to use DevOps or to automate linux processing using UV data.

    ------------------------------
    Gregor Scott
    Software Architect
    Pentana Solutions
    Mount Waverley VIC Australia
    ------------------------------



  • 10.  RE: U2PE, Python and virtual environments

    ROCKETEER
    Posted 02-22-2021 12:07
    Gregor,

    I see where a simple script can be created to do what you ask.

    i.e.
    [root@dentrpy bin]# cat uvpython
    UVHOME=`cat /.uvhome`
    export UVHOME
    . $UVHOME/.pyconfig
    export PYHOME
    $PYHOME/bin/python3
    ---

    Note that this is a very simple example I threw together, and it does not handle passing arguments.  There is also no checking that the Python path is set up correctly and/or that there is a python3 executable in the bin.

    Is setting up an alias in the .profile not an option?
    i.e.
    alias uvpython=$PYHOME/bin/python3


    ------------------------------
    Michael Rajkowski
    Rocket Software
    ------------------------------



  • 11.  RE: U2PE, Python and virtual environments

    PARTNER
    Posted 02-22-2021 18:40

    Yes, an alias is an option.
    They certainly can automate the steps needed to resolve the correct path to the python binary.
    The downside is that it has to be applied during the user session to establish it in the first place.

    We setup our user logins to have the UV bin dir in the user's path, so having a uvpython command within the UV bin directory makes sense for us.

    The current script I have for uvpython is below, and caters for the custom_env setup as well as properly setting the LD_LIBRARY_PATH environment variable.
    The end result is that python run form uvpython will have the same operating environment as python launched from within the UV shell.
    It has been written for Bash on RHEL servers.

    (This forum interface does not appear to offer better options for presenting code  - the PREFORMATTED paragraph setting looks like it just removes line feeds so I did not apply it here. Instead I hand modified the HTML with <pre></pre> tags)

    ----------------

    #!/bin/bash
    ##############################################################################
    # uvpython - Script to locate the current uv-based python installation and
    # forward all command line arguments to the uv python3 binary
    #
    # uvpython - Version 1.0.5 Date 20-August-2020
    #
    # (c) Copyright Pentana Solutions 2019,2020. All Rights Reserved.
    ##############################################################################

    # Finds the appropriate path to the uv-integrated python
    # then execs the python3 command, passing existing command line params

    # get our UV installation directory
    if [ ! -f /.uvhome ];then
    echo "Missing /.uvhome - cannot find python installation."
    exit 1
    fi
    UV_instdir=$(cat /.uvhome)
    # Check we have a value, and it is a path
    if [ -z "${UV_instdir}" ];then
    echo "Empty /.uvhome - cannot find python installation."
    exit 1
    elif [ ! -d "${UV_instdir}" ];then
    echo "${UV_instdir} is not found - cannot find python installation."
    exit 1
    fi

    # get the configured python path
    if [ ! -f ${UV_instdir}/.pyconfig ];then
    echo "Missing ${UV_instdir}/.pyconfig - cannot find python installation"
    exit 1
    fi
    # now source this config file to get variables set
    . ${UV_instdir}/.pyconfig

    # This *SHOULD* have populated 2 environment variables:
    # PYHOME & PYLIB

    # Check we have an environment variable, and it is a path
    if [ -z "${PYHOME}" ]; then
    echo "Missing PYHOME from ${UV_instdir}/.pyconfig - cannot find python installation"
    exit 1
    elif [ ! -d "${PYHOME}" ];then
    echo "Directory ${PYHOME} is not found - cannot find python installation."
    exit 1
    fi

    # build our python binary path
    # hard-code python3, for now
    UVPYBIN=${PYHOME}/bin/python3
    if [ ! -f "${UVPYBIN}" ]; then
    echo "Missing python3 from ${PYHOME}/bin - cannot find python installation"
    exit 1
    elif [ ! -x "${UVPYBIN}" ];then
    echo "${UVPYBIN} is not executable - cannot find python installation"
    exit 1
    fi

    # Ensure we have an appropriate LD_LIBRARY_PATH for uv+python
    # Logic sourced from /usr/uv/bin/uv bash script
    if [ -z "${LD_LIBRARY_PATH}" ];then
    # Likely that we are NOT being launched from within a bash shell
    # launched by the "SH" TCL command - most likely that we are root
    # Start with the bin directory in our UV install directory
    # and the lib path in the uv-configured python directory
    LD_LIBRARY_PATH="${UV_instdir}/bin:${PYHOME}/lib"
    # on linux we also prefix it with /usr/lib64
    if [[ ! ${LD_LIBRARY_PATH} == *"/usr/lib64"* ]]; then
    LD_LIBRARY_PATH=/usr/lib64:${LD_LIBRARY_PATH}
    fi
    fi

    # Check for custom variables (ie. PS1 redefined)
    if [ -f "${UV_instdir}/bin/custom_env" ];then
    # Source the customized variables (should not be a shell script)
    . ${UV_instdir}/bin/custom_env
    fi
    # Make sure we have LIBRARY_PATH assigned
    # LIBRARY_PATH is used when python installs modules and needs to compile+link C code
    if [ -z "${LIBRARY_PATH}" ];then
    LIBRARY_PATH=${LD_LIBRARY_PATH}
    fi

    # now run the python binary, passing all our command line arguments, and
    # setup the LD_LIBRARY_PATH and LIBRARY_PATH env variables for the exec'd binary
    exec env LD_LIBRARY_PATH=${LD_LIBRARY_PATH} LIBRARY_PATH=${LIBRARY_PATH} ${UVPYBIN} "$@"
    # we don't come back from this.

    ----------------



    ------------------------------
    Gregor Scott
    Software Architect
    Pentana Solutions
    Mount Waverley VIC Australia
    ------------------------------



  • 12.  RE: U2PE, Python and virtual environments

    ROCKETEER
    Posted 02-22-2021 18:55
    Gregor,

    I do like the script you wrote, and I am glad it works for you. 

    I will need to discuss this and other options with engineering since there are pros and cons to each solution, and there may not be one that works for everyone.

    As for the issue with the code the forum, it looked fine to me.


    ------------------------------
    Michael Rajkowski
    Rocket Software
    ------------------------------



  • 13.  RE: U2PE, Python and virtual environments

    ROCKETEER
    Posted 02-23-2021 07:53
    Edited by David Andrews 02-23-2021 08:19
    Hi Forum members. I wanted to point out we do have a way to share code on the site.  It doesn't appear this is the universal symbol for code (;-) so some people have missed it. Give it a try when you have the need! Dave



    ------------------------------
    Dave Andrews
    Head of Customer Engagement
    Rocket Software
    ------------------------------