52
Yes, you should use a Python venv in a container like docker (www.bitecode.dev)
submitted 2 years ago by [M] to c/python@programming.dev
41 comments fedilink hide all child comments
 

via https://framapiaf.org/@bitecode/113810446556732210

all 41 comments

sorted by: hot top controversial new old
[–] 21 points 2 years ago (2 children)

Soon, you won't have a choice because major distros are adopting PEP 668. This will make pip install fail in the default system Python and show an error telling you to use a virtual environment.

Well, if this is true then why bother convincing people ;)

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 11 points 2 years ago (5 children)

    So ... if I want to use a python module like, for example, mcstatus in a live shell for convenience I first need to create a venv, activate it, install the package and then use it? And then either have dozens of venvs somewhere or remake them every time?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] 3 points 2 years ago (1 child)

    I am not sure what you mean. Once you created a venv you can always reuse it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 7 points 2 years ago (3 children)

    Yes, but it has to be somewhere. I don't want dozens of venv dirs in my homedir.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 3 points 2 years ago*

    This article is about Python venvs using Docker. That I wouldn't want to pollute the base installation on my local machine should be clear.

    But you can just create a venv and install everything in there, no need to create dozens of venvs if that's what you want.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 2 years ago (2 children)

    Then create one venv for everything

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 3 points 2 years ago* (last edited 2 years ago)

    the one venv to rule them all is not a viable solution.

    Some packages cause problems, one tactic is to isolate the problem package into a separate venv.

    For example

    .venv/ -- main venv for the dev setup

    .doc/.venv - Sphinx docs (now minimum py310)

    .wth for e.g. package restview which has out of date dependencies.

    Each venv has its respective requirements files. Some associated with a dependency or optional dependency. The ones that aren't are pin files.

    Lets say there are easily a total of 20 requirements and constraints (pin files).

    This mess is in a freak'n nasty multi level hierarchy.

    Now imagine you are the author maintainer of 10 packages. Can you be confident one package requirements won't conflict with other packages requirements?

    Almost forgot

    these packages are no longer maintained:

    pip-tools

    pip-requirements-parser

    ... scary

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 years ago (1 child)

    Can you create venvs inside venvs? That sounds like stuff is going to break tbh.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (1 child)

    Why would you want a venv "inside" a venv? What would that mean?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    Well, if you want to have Pip-installed tools available generally (e.g. until distros started screwing it up, pip was the best way to install CMake), the suggestion was to have a venv for the user that would be activated in your .bashrc or whatever.

    I think that would work, but then what happens if you want to use a project-level venv, which is really what they're designed for? If you create and activate a venv when you already have one activated does it all work sensibly? My guess would be that it doesn't.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago

    Oh! Hmm. That's a good question and I really don't know. So in other words (this is just how I'm organizing the thoughts in my own head, probably includes some misunderstandings so feel free to correct any you notice) - your "system Python" is really an activated venv specified in your user config in some way, and the question is what happens when you deliberately try to then activate a distinct project venv, which Python executable and collection of installed libraries is invoked when doing stuff with it active?

    On the one hand I've never considered that and it's probably a mistake to make too many assumptions about how Python (and its instrumentation, pip etc. included) are interacting with the OS. Because I know fuck all about that, when I really think about it lol. On the other hand, one of the things I find pleasant about Python is that usually much more informed and thoughtful people than myself have chosen among several ways of dealing with whatever situation I'm thinking about, and have decided on a sensible default. But yep, idk. I originally just thought you misunderstood the idea of a venv lol, to my happy surprise, nope!

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 years ago (1 child)

    Yes

    Whats the alternative you are advocating for?!

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
    • [–] 9 points 2 years ago (3 children)

    I hate this hand-holding. Certainly use venvs for dev projects but allow system-wide installations for those that want it. OSS has always been about giving you enough rope to hang yourself.

  • permalink
  • fedilink
  • source
  • hideshow 6 child comments
    •
  • [–] 2 points 2 years ago* (last edited 2 years ago) (1 child)

    What really annoys me is they purposely broke per-user and local installation. Fine, system wise installation isn't a good idea when it's already managed by another package manager, but user installation is my domain.

    The reason they did this is because a package installed by the user can be active when a system tool is called and break the system tool. The distro developers went "Oh, we should force all user code into venvs so that our code is safe".

    Completely and utterly backwards. The protected code needs to be inside the defensive wall. The user should be allowed to do anything in the knowledge that they can't inadvertently change the OS. When a system tool is called it should only have system libraries on it's Python Path.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago (2 children)

    You still have the option to choose not to use a venv and risk breaking your user space.

    The changes make this harder to do it by accident by encouraging use of a venv. Part of the problem is that pip install --user is not exactly in the user space and may in fact break system packages, and as you wrote, the user shouldn't be able to inadvertently change the OS.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 2 points 2 years ago*

    So the problem here is that you can inject code into a system python process because they run with the user's python install location on their path.

    They've fixed the wrong "root cause".

  • permalink
  • fedilink
  • source
  • parent
    • [–] 7 points 2 years ago (2 children)

    You already are forced to use a venv, but I fucking hate pip and some projects don't work in venv I don't know why it just doesn't and it sucks

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
  • [–] 7 points 2 years ago (1 child)

    That's the thing, if everybody is forced to use a venv, those projects will either fix their shit or lose all of their userbase.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    So these package maintainers are harboring magical charms and voodoo dolls which us lowly plebs just don't know about?

    If these guys are so awesome, shouldn't we be plying them with coke and blow and keep 'em working resolving our dependency resolution issues?

    They do have the secret sauce and just holding it back from the rest of us

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago (1 child)

    I'm sorry but... what?!?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago* (last edited 2 years ago)

    Would the coke and blow happen for that guy?

    Makes sense if i'm that guy

    This question is about Python package funding. If world+dog no longer stresses over pip dependency resolution isn't this not extremely valuable? So how to go about getting that package permanently funded. No bs dangling a tiny carrot and insisting on strict justice (reporting milestones ...). Then funding only happens for large projects.

    Question on package funding is very legitimate. Have a list of packages that are no longer maintained cuz funding never happened.

    Can subsist on crocodile tears. It's a guilty pleasure.

    Meaning, if package funding never ever happens, and all that ever happens is never ending articles/threads on Python devs whining about dependency resolution, i'm going to feed that.

    Personally not suffering from dependency resolution stress. Everyone else does.

    If the available solutions were sufficient there would be no more articles with comment sections filled with war stories, tales of carnage, and loss.

    ... always comes down to that one guy.

    Solve the Python author maintainer funding issue!

    Then and only then will i market the package that specifically targeted towards resolving pip dependency resolution issues for package (and app) maintainers.

  • permalink
  • fedilink
  • source
  • parent
    • [–] 5 points 2 years ago* (last edited 2 years ago) (2 children)

    System-wide installation as it was implemented should stay in the past. I like pixi's (Conda alternative) approach here, where each system dependency lives in its own virtual bubble, so recreating and porting this software is a breeze.

    But if all you use can stay in a venv, just use one.

  • permalink
  • fedilink
  • source
  • hideshow 4 child comments
    •
    • [–] 3 points 2 years ago (1 child)

    @norambna Good points! πŸ‘πŸ» especially since conflict resolution in PIP sucks and it’ll happily install incompatible packages

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 1 point 2 years ago

    pip is great! It lets ya know when there are dependency conflicts.

    Up to us to learn how to deal with resolving dependency conflicts.

    There are those who attend the whining parade down main street.

    There are the very very few who write a package to resolve these issues.

  • permalink
  • fedilink
  • source
  • parent
    • [–] 2 points 2 years ago

    I'd you have full ci/cd then it's unnecessary

  • permalink
  • fedilink
  • source
    •
    [–] 2 points 2 years ago (1 child)

    How will this affect command-line tools like azure-cli installed in a container image that can be installed with pip? Will we be forced to append the venv to $PATH?

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 2 points 2 years ago* (last edited 2 years ago)

    We need AA meetings

    Hello!

    My name is Billy Joe Jim Bob

    Hello Billy!

    I haven't had a dependency conflict for the past 3 hours. The sleeping problems haven't gone away. As i feel my eye lids drupe, keep thinking about each of my packages and imagining where will the next unresolvable dependency conflicts emerge.

    Then i wake up covered in sweat.

    Can't keep going on like this. Thank you for listening

    Thank you for sharing Billy!

  • permalink
  • fedilink
  • source
  • parent