Bug description
For docker images tagged 5.0.0
and 5.0.0-dev
I've encountered an issue when extending these to include OAuth and ClickHouse drivers. Well, any python extending actually breaks.
This isn't obvious until a container is started based on the extended image. The extra python modules fails to be imported because they are not found. This is because pip
and python
is pointing to different environments in these images. For reference, the image 4.1.3-dev
has both pip
and python
pointing to the same environment. The container stops with "ModuleNotFoundError: No module named 'authlib'", in my case. I see that this error has been spotted in earlier issues as well. Perhaps it is this rogue environment that was the culprit back then as well?
python
is pointing towards a python environment placed in /app/.venv/
that also seems to be first in PATH
.
pip
is pointing to /usr/local/bin/pip
.
A workaround for me to get past this was to remove the /app/.venv/
when extending the docker image.
Screenshots/recordings
No response
Superset version
5.0.0
Python version
3.10
Node version
I don't know
Browser
Not applicable
Additional context
ModuleNotFoundError: No module named 'authlib'
Checklist
- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and did find similar bug reports but they were closed without finding the root cause.
- [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
Comment From: dosubot[bot]
Hi @jesperbagge! I'm Dosu and I’m helping the superset team.
This is a known breaking change in Superset 5.0.0 Docker images. The image now creates a Python virtual environment at /app/.venv
using uv venv /app/.venv
and sets the PATH so that /app/.venv/bin
is first, making this venv the default for python
and pip
commands. However, the package manager switched from pip
to uv
, and uv
is installed system-wide, not inside /app/.venv
. If you install Python packages using the system pip
or outside /app/.venv
, they won't be available at runtime, leading to ModuleNotFoundError
for packages like authlib
or database drivers (details, UPDATING.md, Dockerfile).
To extend the image and install additional Python packages, use either uv pip install <package>
or /app/.venv/bin/pip install <package>
in your Dockerfile or bootstrap script. This ensures the packages are installed in the environment Superset actually uses. Removing /app/.venv
works as a workaround but is not recommended unless you have a specific need to bypass the venv.
There's also a known issue where the superset
package is installed both in /app/superset
and /app/.venv/lib/python3.10/site-packages/superset
, which can cause confusion and bloat, but removing /app/superset
does not break functionality (reference).
For more context and ongoing discussion, see this issue, this discussion, and this PR updating the docs.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: user1500177
I also faced the same error ,what to do , usually I never used to install the authlib via docker file
Comment From: jesperbagge
Without fully understanding what uv
brings to the table I did what @dosu suggested and now have a working image. Neat!
FROM apache/superset:5.0.0-dev
USER root
COPY requirements.txt .
RUN uv pip install --no-cache-dir -r requirements.txt
USER superset
WORKDIR /app
Comment From: dosubot[bot]
Glad to hear that worked! For context, uv
is a modern Python package manager that aims to be much faster than pip
and handles dependency resolution efficiently. In Superset 5.0.0 Docker images, uv
is used to create and manage the /app/.venv
virtual environment, and running uv pip install
ensures your packages end up in the right place for Superset to use them. If you want more details on uv
or the new Docker setup, let me know!
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other