Bug description

Hey Superset team!

Since I've upgraded from Superset v3.0.0 to v3.1.0, I have stopped being able to successfully import dashboards that contain datasets that specifically use a Trino datasource, with the new feature of Enable row expansion in schemas enabled. Additionally figured out that if I modify the respective database yaml that is shipped out in the zip export, and manually remove the extra.schema_options key and values, importing works. This mitigates the problem for the time being, but does not solve it.

The source of the problem seems to lie in the Superset importing tool not take into account this new Trino feature.

I will provide evidence and more context below.

Looking forward for your response and happy to clarify any more details.

How to reproduce the bug

Go to Dashboards
Bulk select dashboards that contain charts that use a Trino database with Enable row expansion in schemas enabled
Export dashboards
Import dashboards

Screenshots/recordings

Superset stacktrace:

Command failed validation
2024-02-06 10:56:35,825:INFO:superset.commands.dashboard.importers.dispatcher:Command failed validation
CommandException
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 127, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 121, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1454, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 255, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 108, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/dashboards/api.py", line 1193, in import_
    command.run()
  File "/app/superset/commands/dashboard/importers/dispatcher.py", line 64, in run
    raise exc
  File "/app/superset/commands/dashboard/importers/dispatcher.py", line 57, in run
    command.run()
  File "/app/superset/commands/importers/v1/__init__.py", line 73, in run
    self.validate()
  File "/app/superset/commands/importers/v1/__init__.py", line 111, in validate
    raise CommandInvalidError(
superset.commands.exceptions.CommandInvalidError: Error importing dashboard
2024-02-06 10:56:35,826:WARNING:superset.views.base:CommandException
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/usr/local/lib/python3.9/site-packages/flask_appbuilder/security/decorators.py", line 95, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/views/base_api.py", line 127, in wraps
    raise ex
  File "/app/superset/views/base_api.py", line 121, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
  File "/app/superset/utils/core.py", line 1454, in time_function
    response = func(*args, **kwargs)
  File "/app/superset/utils/log.py", line 255, in wrapper
    value = f(*args, **kwargs)
  File "/app/superset/views/base_api.py", line 108, in wraps
    return f(self, *args, **kwargs)
  File "/app/superset/dashboards/api.py", line 1193, in import_
    command.run()
  File "/app/superset/commands/dashboard/importers/dispatcher.py", line 64, in run
    raise exc
  File "/app/superset/commands/dashboard/importers/dispatcher.py", line 57, in run
    command.run()
  File "/app/superset/commands/importers/v1/__init__.py", line 73, in run
    self.validate()
  File "/app/superset/commands/importers/v1/__init__.py", line 111, in validate
    raise CommandInvalidError(
superset.commands.exceptions.CommandInvalidError: Error importing dashboard

Screenshots

Superset error Screenshot_20240206_110928

Browser debug Screenshot_20240206_111352

Trino Database connection Screenshot_20240206_110907

Trino Database exported YAML Screenshot_20240206_111702

Superset version

3.1.0

Python version

3.9

Node version

Not applicable

Browser

Firefox

Additional context

I have the following feature flags enabled:

FEATURE_FLAGS = {
      "ALERT_REPORTS": True,
      "DASHBOARD_RBAC": True,
      "ENABLE_TEMPLATE_PROCESSING": True,
    }

There are no customization made on Superset, but the container image is packaged by me in order to include necessary dependencies like the Trino connector. The Superset is deployed in Kubernetes using the official Superset helm chart.

About the data source I am querying, I've already provided details on it on the post.

Checklist

[X] I have searched Superset docs and Slack and didn't find a solution to my problem.
[X] I have searched the GitHub issue tracker and didn't find a similar bug report.
[X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Comment From: michael-s-molina

Nice issue description @ruifpedro! Thank you.

I'll tag some folks with more knowledge about that feature: @betodealmeida @eschutho @rusackas

Comment From: ilndinesh

Hello @ruifpedro thank you for detailed explanation. It helped us as well.

Comment From: lokeshwar9

Hello @ruifpedro , Your detailed explanation helped us figure out that checking the "enable row expansion" is causing the dashboard import error in the same instance and different instance, we fixed it by writing code to stop schema options keys from going into the export zip. we noticed that we are unable to uncheck the "enable row expansion" using GUI and we used curl to uncheck it. Many Thanks once again and hope this helps others as well. Good day

Comment From: marituhone

@ruifpedro do you manage to solve the issue?i get the same issue how can i fix it?

Comment From: ruifpedro

Hello @ruifpedro , Your detailed explanation helped us figure out that checking the "enable row expansion" is causing the dashboard import error in the same instance and different instance, we fixed it by writing code to stop schema options keys from going into the export zip. we noticed that we are unable to uncheck the "enable row expansion" using GUI and we used curl to uncheck it. Many Thanks once again and hope this helps others as well. Good day

I've missed your comment! Yes that it's the case, I'm also unable to uncheck the "enable row expansion" option.

Comment From: ruifpedro

@ruifpedro do you manage to solve the issue?i get the same issue how can i fix it?

As my first post states, I've managed to circumvent the problem by simply, exporting the dashboard, opening up the zip and modifying the respective database yaml to manually remove the extra.schema_options key and values. Zip the contents and then you have a valid set of assets that can be imported.

Comment From: JustGui

Hey guys ! It's still an issue on 4.0.2; is a fix planned for it ?

Comment From: rusackas

Hey guys ! It's still an issue on 4.0.2; is a fix planned for it ?

Welcomed, yes, but planned... not so much. Unless @yousoph is aware of this and planning to get it in a Preset sprint, I doubt we'll see a fix soon. Contributions/PRs more than welcomed if you'd like to tackle this one!

Comment From: SuDaYu2019

I also encountered this issue (4.1.1). In the dataset settings, I added a parameter template (jinja), which resulted in the creation of template_params. This affected the import of dashboards and databases (export was not impacted).

Comment From: eschutho

@geido can we put up a bounty for this?

Comment From: qleroy

This is a fix https://github.com/apache/superset/pull/32657 ?

Comment From: rusackas

Can anyone cofirm or deny if https://github.com/apache/superset/pull/32657 resolved this?

Comment From: rusackas

Or https://github.com/apache/superset/pull/33547 for that matter (you'd have to pull it and test locally)

Apache Superset Error importing dashboard

Bug description

How to reproduce the bug

Screenshots/recordings

Screenshots

Superset version

Python version

Node version

Browser

Additional context

Checklist