I setup superset to authenticate the users with keycloak as IdP. I'm able to connect to superset after validating a login/pwd on keycloak but I cannot access the users list page (/users/list) as an Admin.

How to reproduce the bug

Setup superset as described in "Additional context". Connect to superset through keycloak as an Admin Click on the menu Settings > List Users

Expected results

See the users list

Actual results

I'm back on the welcome page (/superset/welcome/) with a message saying "Access is Denied".

Screenshots

n/a

Environment

(please complete the following information):

  • browser type and version: Brave Version 1.37.109 Chromium: 100.0.4896.60 (Build officiel) (64 bits)

  • superset version: superset version

    Superset 1.4.2

  • python version: python --version Python 3.8.12
  • node.js version: node -v n/a
  • any feature flags active: ALERT_REPORTS

Checklist

Make sure to follow these steps before submitting your issue - thank you!

  • [X] I have checked the superset logs for python stacktraces and included it here as text if there are any. No stacktrace
  • [X] I have reproduced the issue with at least the latest released version of superset.
  • [X] I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

I have a keycloak setup with the url: https://auth.mydomain.com/ I create a realm named "demo" and a user on this realm. I add a client named "superset" (client ID) on this realm with a Client Protocol "openid-connect" and a Root URL "https://superset.demo.mydomain.com/"

My superset is accessible with the url: https://superset.demo.mydomain.com/

I use the Docker version of superset deployed on a Docker Swarm cluster. I use Traefik to route the HTTP requests to the containers (superset and keycloak).

To setup superset with keycloak, I follow these posts: - https://github.com/apache/superset/discussions/13915 - https://stackoverflow.com/questions/54010314/using-keycloakopenid-connect-with-apache-superset/54024394#54024394

But I modify some files so I describe below my whole configuration.

Content of /app/docker/requirements-local.txt:

clickhouse-driver>=0.2.0
clickhouse-sqlalchemy>=0.1.6,<0.2.0
mysql-connector-python
flask-oidc==1.3.0

The first 2 packages are used to connect to my clickhouse database. The third one is used to have the superset database on MySQL (instead of postgres). Only the fourth one is related to keycloak to enable OIDC (OpenID Connect).

Content of /app/pythonpath/superset_config.py:

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#   http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#
# This file is included in the final Docker image and SHOULD be overridden when
# deploying the image to prod. Settings configured here are intended for use in local
# development environments. Also note that superset_config_docker.py is imported
# as a final step as a means to override "defaults" configured here
#
import logging
import os
from datetime import timedelta
from typing import Optional

from cachelib.file import FileSystemCache
from celery.schedules import crontab

logger = logging.getLogger()


def get_env_variable(var_name: str, default: Optional[str] = None) -> str:
    """Get the environment variable or raise exception."""
    try:
        return os.environ[var_name]
    except KeyError:
        if default is not None:
            return default
        else:
            error_msg = "The environment variable {} was missing, abort...".format(
                var_name
            )
            raise EnvironmentError(error_msg)


DATABASE_DIALECT = get_env_variable("DATABASE_DIALECT")
DATABASE_USER = get_env_variable("DATABASE_USER")
DATABASE_PASSWORD = get_env_variable("DATABASE_PASSWORD")
DATABASE_HOST = get_env_variable("DATABASE_HOST")
DATABASE_PORT = get_env_variable("DATABASE_PORT")
DATABASE_DB = get_env_variable("DATABASE_DB")

# The SQLAlchemy connection string.
SQLALCHEMY_DATABASE_URI = "%s://%s:%s@%s:%s/%s" % (
    DATABASE_DIALECT,
    DATABASE_USER,
    DATABASE_PASSWORD,
    DATABASE_HOST,
    DATABASE_PORT,
    DATABASE_DB,
)

REDIS_HOST = get_env_variable("REDIS_HOST")
REDIS_PORT = get_env_variable("REDIS_PORT")
REDIS_CELERY_DB = get_env_variable("REDIS_CELERY_DB", "0")
REDIS_RESULTS_DB = get_env_variable("REDIS_RESULTS_DB", "1")

##### Change from original file
#RESULTS_BACKEND = FileSystemCache("/app/superset_home/sqllab")
# Put results on Redis not on a file that is not shared among container on Swarm stack
from cachelib.redis import RedisCache
RESULTS_BACKEND = RedisCache(host=REDIS_HOST, port=REDIS_PORT, key_prefix='superset_results')
##### End of change

class CeleryConfig(object):
    BROKER_URL = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
    CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
    CELERY_RESULT_BACKEND = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
    CELERYD_LOG_LEVEL = "DEBUG"
    CELERYD_PREFETCH_MULTIPLIER = 1
    CELERY_ACKS_LATE = False
    CELERYBEAT_SCHEDULE = {
        "reports.scheduler": {
            "task": "reports.scheduler",
            "schedule": crontab(minute="*", hour="*"),
        },
        "reports.prune_log": {
            "task": "reports.prune_log",
            "schedule": crontab(minute=10, hour=0),
        },
    }


CELERY_CONFIG = CeleryConfig

FEATURE_FLAGS = {"ALERT_REPORTS": True}
ALERT_REPORTS_NOTIFICATION_DRY_RUN = True
WEBDRIVER_BASEURL = "http://superset:8088/"
# The base URL for the email report hyperlinks.
WEBDRIVER_BASEURL_USER_FRIENDLY = WEBDRIVER_BASEURL

SQLLAB_CTAS_NO_LIMIT = True

#
# Optionally import superset_config_docker.py (which will have been included on
# the PYTHONPATH) in order to allow for local settings to be overridden
#
try:
    import superset_config_docker
    from superset_config_docker import *  # noqa

    logger.info(
        f"Loaded your Docker configuration at " f"[{superset_config_docker.__file__}]"
    )
except ImportError:
    logger.info("Using default Docker config...")

Content of /app/pythonpath/superset_config_docker.py:

import os
from typing import Optional

def get_env_variable(var_name: str, default: Optional[str] = None) -> str:
    """Get the environment variable or raise exception."""
    try:
        return os.environ[var_name]
    except KeyError:
        if default is not None:
            return default
        else:
            error_msg = "The environment variable {} was missing, abort...".format(
                var_name
            )
            raise EnvironmentError(error_msg)


# The allowed translation for you app
LANGUAGES = {
    "en": {"flag": "us", "name": "English"},
    #"es": {"flag": "es", "name": "Spanish"},
    #"it": {"flag": "it", "name": "Italian"},
    "fr": {"flag": "fr", "name": "French"},
    #"zh": {"flag": "cn", "name": "Chinese"},
    #"ja": {"flag": "jp", "name": "Japanese"},
    #"de": {"flag": "de", "name": "German"},
    #"pt": {"flag": "pt", "name": "Portuguese"},
    #"pt_BR": {"flag": "br", "name": "Brazilian Portuguese"},
    #"ru": {"flag": "ru", "name": "Russian"},
    #"ko": {"flag": "kr", "name": "Korean"},
    #"sk": {"flag": "sk", "name": "Slovak"},
    #"sl": {"flag": "si", "name": "Slovenian"},
}

ENABLE_PROXY_FIX = True

#---------------------------KEYCLOACK ----------------------------
# See: https://github.com/apache/superset/discussions/13915
# See: https://stackoverflow.com/questions/54010314/using-keycloakopenid-connect-with-apache-superset/54024394#54024394

from keycloak_security_manager  import  OIDCSecurityManager
from flask_appbuilder.security.manager import AUTH_OID

OIDC_ENABLE = get_env_variable("OIDC_ENABLE", 'False')

if OIDC_ENABLE == 'True':
    AUTH_TYPE = AUTH_OID
    SECRET_KEY = get_env_variable("SECRET_KEY", 'ChangeThisKeyPlease')
    OIDC_CLIENT_SECRETS = get_env_variable("OIDC_CLIENT_SECRETS", '/app/pythonpath/client_secret.json')
    OIDC_ID_TOKEN_COOKIE_SECURE = False
    OIDC_REQUIRE_VERIFIED_EMAIL = False
    OIDC_OPENID_REALM = get_env_variable("OIDC_OPENID_REALM")
    OIDC_INTROSPECTION_AUTH_METHOD = 'client_secret_post'
    CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
    AUTH_USER_REGISTRATION = True
    AUTH_USER_REGISTRATION_ROLE = get_env_variable("AUTH_USER_REGISTRATION_ROLE", 'Admin')
#--------------------------------------------------------------

Content of /app/pythonpath/keycloak_security_manager.py:

from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging
from flask import redirect, request


class OIDCSecurityManager(SupersetSecurityManager):

    def __init__(self, appbuilder):
        super(OIDCSecurityManager, self).__init__(appbuilder)
        if self.auth_type == AUTH_OID:
            self.oid = OpenIDConnect(self.appbuilder.get_app)
        self.authoidview = AuthOIDCView

class AuthOIDCView(AuthOIDView):

    @expose('/login/', methods=['GET', 'POST'])
    def login(self, flag=True):
        sm = self.appbuilder.sm
        oidc = sm.oid

        @self.appbuilder.sm.oid.require_login
        def handle_login():
            user = sm.auth_user_oid(oidc.user_getfield('email'))

            if user is None:
                info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
                user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'),
                                   info.get('email'), sm.find_role('Admin'))

            login_user(user, remember=False)
            return redirect(self.appbuilder.get_url_for_index)

        return handle_login()

    @expose('/logout/', methods=['GET', 'POST'])
    def logout(self):
        oidc = self.appbuilder.sm.oid

        oidc.logout()
        super(AuthOIDCView, self).logout()
        redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login

        return redirect(
            oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))

Content of /app/pythonpath/client_secret.json:

{
  "web": {
      "issuer": "https://auth.mydomain.com/realms/demo",
      "auth_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/auth",
      "client_id": "superset",
      "client_secret": "<Client Secret>",
      "redirect_uris": [
          "https://superset.demo.mydomain.com/*"
      ],
      "userinfo_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/userinfo",
      "token_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/token",
      "token_introspection_uri": "https://auth.mydomain.com/realms/demo/protocol/openid-connect/token/introspect"
  }
}

The environment variables I pass to my Docker Swarm service:

FLASK_ENV=production
SUPERSET_ENV=production
SUPERSET_LOAD_EXAMPLES=no
CYPRESS_CONFIG=false
SUPERSET_PORT=8088
OIDC_ENABLE=True
OIDC_OPENID_REALM=demo

NOTA: I remove my environment variables related to mysql and redis.

Thanks for your help. Tell me if some information is missing.

Comment From: fedepad

Questions which I think are helpful for debugging: - What roles does your user on IDP has? - Is Admin an actual role in the IDP or not? If not, do you have the mapping from IDP roles to FAB roles somewhere? - Does the role of the admin user you are using to login is the one you expect in the superset/FAB backend db? Check which role this user has just by looking at the backend db (MySQL in your case, since you cannot check the users list from UI...) These are just to start with, first sanity checks.

Comment From: PatBriPerso

Thanks @fedepad.

In fact, I change the role of my user with the superset UI and give him the Admin role. Here is how I do that: - I start my stack with OIDC_ENABLE=False so I can connect with the default Admin account (stored in the DB) and I change its password - I update my stack with OIDC_ENABLE=True and connect with a keycloak account. A new user was created in superset DB - I change back to OIDC_ENABLE=False, connect with the default Admin account and change the role of my keycloak user to Admin - Finally, I change again to OIDC_ENABLE=True, connect with my keycloak account and have the access issue (to the users list page)

I have no specific roles for my keycloak user. I think he has the default keycloak roles but I do not know if those roles are sent to superset. I think roles on keycloak and roles on superset are not related (but I'm not sure).

Comment From: rahul149386

@fedepad @PatBriPerso I created same user in keycloak and superset eventhough im able to see None user inevent log and througing timeout error any help regarding this. Seeing below

Comment From: rahul149386

@fedepad @PatBriPerso I created same user in keycloak and superset eventhough im able to see None user inevent log and througing timeout error any help regarding this. Seeing below

Comment From: debimishra89

Can we do role mapping between keycloak and superset with AUTH_TYPE=AUTH_OID?? Or is it only available with AUTH_OAUTH

Comment From: rusackas

@PatBriPerso are you still facing this issue, or should we close it?

Comment From: resulraveendran

Hi,

I got the same issue when i integrate with Zitadel is there any solution for this.

Comment From: leezj-cbm

Having the same issue as OP. Unable to access User List as Admin.

Unable to access User Info as Admin, Alpha and Gamma. Symptoms similar to described as above.

Any advise given will be appreciated.

Comment From: alexvaut

I confirm I have the same issue, if I disable OIDC auth, then I can access user list and user info. If I enable it then I get an "Access Denied". Superset version: 4.1.1 keycloak version: 26.0.5 I use some kind of role mapping between the roles in keycloak and the roles in superset. My logic is implemented in my CUSTOM_SECURITY_MANAGER.

It used to work well with superset version 2.1.0.

Comment From: ZAZmaster

I confirm I have the same issue. Does anyone have a temporary solution?

Comment From: JacksonSx

We had the same problem with Superset 4.1.1 in Docker Swarm, after executing superset init the problem was solved. Used AUTH_TYPE = AUTH_OAUTH for integration with Keycloak.

Comment From: plant123x

I have the same issue with superset version 4.1.1. Did any one resolve the issue?