I’ve been going down a path in astronomy and cosmology that is utilizing more and more python processes.
I started using the anaconda python system that is installed as user. Anaconda is a huge repo of python files in it’s own environment. Over a few years, i have moved to smaller but similar user systems and I am now using miniforge python with half a dozen environments.
As an experiment, I then started using python system installs while working in some astropy tutorials. I found this very successful. I also think openSUSE repos have probably all the python files that anaconda has. That surprised me.
pip install for niche files (those not in system) does not run for the system python. Zypper does seem to have all the python files I have been needing but the naming convention is different, i.e., python313-XXXX vs just XXX in miniforge. I am not sure how to add a python file to system that is outside of zypper knowledge.
All the major ides (geany, pycharm, and spyder can be configured to work from system or anaconda environments with some tinkering. Image plot displays do look better to me using the anaconda type system as opposed to system python vbut that is subjective.
As a far from expert in python, I am wondering over the long term what are the advantages or not of using anaconda python vs system python. I am into cosmology/space stuff now which does include some data analysis.
Anyone with any thoughts from a more experienced viewpoint? I thought this was more of a chat topic than technical issue.
In my experience, either isolated in a Docker container (I’ve a few Python programs that I run that way) or in a venv configuration really seems to be the way to go. I’ve used anaconda a couple of times, but venv seems to be much easier. The only exception I’d make to using a venv is if the script doesn’t really need anything that’s not in the standard python libraries in the standard TW repos.
The basic idea of venv really is to be able to build a functional environment based on the dependencies needed for any specific set of python code by pulling in the specific versions you need.
It keeps the dependencies clean, and makes it feasible to move stuff between systems really easily.
As for anaconda, if a program calls for it, I’d use it. If it doesn’t, I’d stick to using venv with a base installed set of system python libraries. But unless the scripts are really limited to what’s in the system python packages, I’d use venv.
I have been working studying tuts using astropy which is in the standard system python using geany ide. One tut, though, calls for use of “acstools” which is a specialized Hubble tele code. I can’t add it to system python through zypper as it does not exist there. pip will not install it into system. I am looking at creating venv in system and using pipx to install it.
Of course, I could also use anaconda (really miniforge) to also create an env to add acstools to.
Your experience says the system venv is less hassle than the anaconda route. By the way, I have found that anaconda being in the PATH has created gliches in use of some other programs as they are using anaconda modules unknowingly when they should be using system modules. These problems were very difficult to diagnose. I now deactive anaconda to avoid this problem.
i will move ahead using openSUSE system python and set up venv for my ongoing stuff.
The important thing about using venv is to use it per project rather than a global configuration in your userspace. The reason to do it per project is to handle requirements that are specific to each project - if you have something that requires, for example, numpy 2.4.6, and something else that requires nyumpy 2.4.5, they probably would conflict in the same venv and cause problems.
So having the venv as an isolation layer for the python install used for each project makes the most sense.
The biggest thing to remember is to activate the venv enviornment before starting to use it. You’ll know if you haven’t done that, because stuff won’t work. It also modifies the system prompt so you know which venv you’re in, which is helpful.
If it are only a few packages you could consider building them in your OpenSuse build home project, add that as repo and install them from your home repo.
I’m worried about root disk space if I set up many venv. I went through one effort to increase root system (/) partition size. Can you put venv storage space in a user directory?
What packages does venv put into each environment? Is, e.g., a numpy put into each venv that uses it? If the venv uses big data files can these be stored on user space?
I see that my anaconda (really miniforge) is currently at 26 gb in user space but I have basically unlimited user space so that is unimportant. If 2/3 of that goes into / by transferring to system venv, I would be starting to get worried.
That’s how venv works. It doesn’t use space outside your home directory (or wherever you define the venv space).
The point of venv is that it’s self-contained and doesn’t use the system python (the python executables are linked to the system python, but all of the site_packages stuff is local to the project).
I’m starting to get onboard with venv. I see the venv is in user; not system / which was a concern. Working through tutorial. Will make an astropy venv and get my geany code pointed to it for a test drive.