Bug description

For docker images tagged 5.0.0 and 5.0.0-dev I've encountered an issue when extending these to include OAuth and ClickHouse drivers. Well, any python extending actually breaks.

This isn't obvious until a container is started based on the extended image. The extra python modules fails to be imported because they are not found. This is because pip and python is pointing to different environments in these images. For reference, the image 4.1.3-dev has both pip and python pointing to the same environment. The container stops with "ModuleNotFoundError: No module named 'authlib'", in my case. I see that this error has been spotted in earlier issues as well. Perhaps it is this rogue environment that was the culprit back then as well?

python is pointing towards a python environment placed in /app/.venv/ that also seems to be first in PATH. pip is pointing to /usr/local/bin/pip.

A workaround for me to get past this was to remove the /app/.venv/ when extending the docker image.

Screenshots/recordings

No response

Superset version

5.0.0

Python version

3.10

Node version

I don't know

Browser

Not applicable

Additional context

ModuleNotFoundError: No module named 'authlib'

Checklist

  • [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
  • [x] I have searched the GitHub issue tracker and did find similar bug reports but they were closed without finding the root cause.
  • [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Comment From: dosubot[bot]

Hi @jesperbagge! I'm Dosu and I’m helping the superset team.

This is a known breaking change in Superset 5.0.0 Docker images. The image now creates a Python virtual environment at /app/.venv using uv venv /app/.venv and sets the PATH so that /app/.venv/bin is first, making this venv the default for python and pip commands. However, the package manager switched from pip to uv, and uv is installed system-wide, not inside /app/.venv. If you install Python packages using the system pip or outside /app/.venv, they won't be available at runtime, leading to ModuleNotFoundError for packages like authlib or database drivers (detailsUPDATING.mdDockerfile).

To extend the image and install additional Python packages, use either uv pip install <package> or /app/.venv/bin/pip install <package> in your Dockerfile or bootstrap script. This ensures the packages are installed in the environment Superset actually uses. Removing /app/.venv works as a workaround but is not recommended unless you have a specific need to bypass the venv.

There's also a known issue where the superset package is installed both in /app/superset and /app/.venv/lib/python3.10/site-packages/superset, which can cause confusion and bloat, but removing /app/superset does not break functionality (reference).

For more context and ongoing discussion, see this issuethis discussion, and this PR updating the docs.

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X

Comment From: user1500177

I also faced the same error ,what to do , usually I never used to install the authlib via docker file

Comment From: jesperbagge

Without fully understanding what uv brings to the table I did what @dosu suggested and now have a working image. Neat!

FROM apache/superset:5.0.0-dev
USER root
COPY requirements.txt .
RUN uv pip install --no-cache-dir -r requirements.txt
USER superset
WORKDIR /app

Comment From: dosubot[bot]

Glad to hear that worked! For context, uv is a modern Python package manager that aims to be much faster than pip and handles dependency resolution efficiently. In Superset 5.0.0 Docker images, uv is used to create and manage the /app/.venv virtual environment, and running uv pip install ensures your packages end up in the right place for Superset to use them. If you want more details on uv or the new Docker setup, let me know!

To reply, just mention @dosu.


How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other  Chat with Dosu Join Discord Share on X