Bug description

During upgrading to new 5.0.0rc1 version is not possible due to No module named 'psycopg2'.

Error: superset | File "/app/.venv/lib/python3.10/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 811, in dbapi superset | import psycopg2 superset | ModuleNotFoundError: No module named 'psycopg2'

docker-compose.yml:
version: '3.8' services: superset: image: apache/superset:5.0.0rc1 container_name: superset ports: - "8088:8088" environment: - SECRET_KEY=123 - SUPERSET_CONFIG_PATH=/app/superset_config.py - EXTRA_PIP_PACKAGES=psycopg2-binary volumes: - ./superset_config.py:/app/superset_config.py command: > sh -c " pip install psycopg2-binary && superset db upgrade && superset init && gunicorn -w 2 --timeout 60 -b 0.0.0.0:8088 'superset.app:create_app()' " extra_hosts: - "host.docker.internal:host-gateway"

I installed the package in my venv and also outside (on system level) but error still present.

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.11

Node version

16

Browser

Not applicable

Additional context

No response

Checklist

  • [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
  • [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
  • [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Comment From: dosubot[bot]

Hey @kenyodenes! I'm here to help you with any bugs, questions, or contributions you have for Superset. Let's squash those bugs together while we wait for a human to jump in!

To resolve the "No module named 'psycopg2'" error when upgrading to Apache Superset version 5.0.0rc1 using Docker Compose, you should use the apache/superset:latest-dev image tag instead of apache/superset:5.0.0rc1. The base superset:latest image no longer includes PostgreSQL drivers by default [1].

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

Comment From: kenyodenes

Hello @dosu.

the lates-dev does not pull the 5.0. Still keep the 4.1 version. I would like to use 5.0

Comment From: dosubot[bot]

To use Apache Superset version 5.0.0rc1 with Docker Compose, you need to specify the version tag directly in your Docker Compose configuration. You can do this by setting the TAG environment variable to 5.0.0rc1 before running Docker Compose, or by modifying the configuration file to use the specific version tag.

Here's how you can do it:

  1. Set the TAG environment variable in your shell before running Docker Compose:
export TAG=5.0.0rc1
docker-compose up
  1. Alternatively, modify your docker-compose.yml file to directly specify the version:
services:
  superset:
    image: apache/superset:5.0.0rc1
    ...

This will ensure that Docker Compose pulls the specified version 5.0.0rc1 of Apache Superset [1].

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

Comment From: kenyodenes

@dosu, but that was my original complaint. It is not working due to above mentioned error.

Comment From: dosubot[bot]

To resolve the "No module named 'psycopg2'" error with Apache Superset version 5.0.0rc1, you can try using the apache/superset:latest-dev image, which includes the necessary PostgreSQL drivers. If you specifically need version 5.0.0rc1 and the latest-dev tag doesn't work for you, you can manually install psycopg2-binary in your Docker setup.

Here's a workaround you can try:

  1. Modify your docker-compose.yml to include the EXTRA_PIP_PACKAGES environment variable:
services:
  superset:
    image: apache/superset:5.0.0rc1
    environment:
      - EXTRA_PIP_PACKAGES=psycopg2-binary
    ...
  1. Ensure that your command in the docker-compose.yml includes the installation of psycopg2-binary:
command: >
  sh -c "
  pip install psycopg2-binary &&
  superset db upgrade &&
  superset init &&
  gunicorn -w 2 --timeout 60 -b 0.0.0.0:8088 'superset.app:create_app()'
  "

This setup should help you include the psycopg2-binary package during the container startup, resolving the import error.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

Comment From: hainenber

@kenyodenes Superset v5 introduces uv as Python package manager instead of usual pip. You can install psycopg2-binary with uv pip install psycopg2-binary.

Comment From: kenyodenes

@hainenber , Many Thanks. That was the missing information.

Comment From: rc452860

Is this problem present in all release versions? I encountered the same issue on 4.1.1 as well.

Comment From: yudinikita

Not sure if this is the right solution, but it helped me to add the line: psycopg2-binary>=2.9.0 to the file superset/docker/requirements-local.txt

Comment From: sfirke

Is this problem present in all release versions? I encountered the same issue on 4.1.1 as well.

4.1.0 is the first version that didn't bundle the Postgres driver, I believe.