Charm venv shadows system python dependencies
Metadata
Current evaluation
No evaluation has been recorded for this issue yet.
Issue body
### Bug Description
When running subprocesses within the charm code, the Juju Python virtual environment `/var/lib/juju/agents/unit-landscape-server-0/charm/venv/` is included in the system’s PYTHONPATH. This causes conflicts, as we need to use the system Python and its dependencies, not those from the Juju environment.
For example this bug https://bugs.launchpad.net/landscape-charm/+bug/2076926, installing a later version of the pydantic package via apt doesn't do anything because the version from the Juju charm virtual environment takes precedence. A similar issue occurred with the distutils package, where the system version was overshadowed.
### To Reproduce
We used this workaround to make sure the system Python paths are used by removing Juju's virtual environment from PYTHONPATH before invoking subprocesses:
```python
def get_modified_env_vars():
"""
Because the python path gets munged by the juju env, this grabs the current
env vars and returns a copy with the juju env removed from the python paths
"""
env_vars = os.environ.copy()
logging.info("Fixing python paths")
new_paths = [path for path in sys.path if "juju" not in path]
env_vars["PYTHONPATH"] = ":".join(new_paths)
return env_vars
subprocess.check_call(call, env=get_modified_env_vars())
```
### Environment
22.04
juju --version
3.5.3-genericlinux-amd64
### charmcraft.yaml
```yaml
N/a
```
### Relevant log output
```shell
2024-08-13 08:25:19 WARNING unit.landscape-server/0.db-relation-changed logger.go:60 from canonical.landscape.model.main.oidc import (
2024-08-13 08:25:19 WARNING unit.landscape-server/0.db-relation-changed logger.go:60 File "/opt/canonical/landscape/canonical/landscape/model/main/oidc.py", line 6, in <module>
2024-08-13 08:25:19 WARNING unit.landscape-server/0.db-relation-changed logger.go:60 from pydantic import AnyUrl, BaseModel, Field, model_validator
2024-08-13 08:25:19 WARNING unit.landscape-server/0.db-relation-changed logger.go:60 ImportError: cannot import name 'model_validator' from 'pydantic' (/var/lib/juju/agents/unit-landscape-server-0/charm/venv/pydantic/__init__.py)
2024-08-13 08:25:19 ERROR unit.landscape-server/0.juju-log server.go:325 db:8: Landscape Server schema update failed with return code 1
```
Evaluation history
No evaluation history available.