Bug description
Cannot accesse 'Query history' page, and get empty list with this error:
the error comes from sql_parse.py. check the logs below.
How to reproduce the bug
Minimum sample:
- Start superset with following docker-compose.yml
version: "3.7"
services:
superset:
image: apachesuperset.docker.scarf.sh/apache/superset:3.1.2
volumes:
- superset_home:/app/superset_home
environment:
SUPERSET_SECRET_KEY: secretkey
SUPERSET_DB_URI: sqlite:///superset_home/superset.db?check_same_thread=false
SUPERSET_CONFIG_PATH: /app/superset_home/superset_config.py
ports:
- 8088:8088
command:
- "sh"
- "-c"
- |-
# allow sqlite for easier reproduce
echo 'PREVENT_UNSAFE_DB_CONNECTIONS=False' > /app/superset_home/superset_config.py
# init database
superset db upgrade
# init admin user
superset fab create-admin --username admin --firstname Superset --lastname Admin --email admin@superset.com --password admin
# init roles
superset init
# run server
/usr/bin/run-server.sh
volumes:
superset_home:
external: false
- Go localhost:8088, login as admin/admin
- Add one database, can just use 'sqlite:///superset_home/superset.db?check_same_thread=false'
- Goto SQL->SQLLab, run some queries.
- Goto SQL->Query hisory, and the error appears.
Screenshots/recordings
Superset version
3.1.2
Python version
3.10
Node version
Not applicable
Browser
Chrome
Additional context
tried both 3.1.2 and 4.0.0, and got same error with slightly different line numbers.
logs from 3.1.2:
Traceback (most recent call last):
File "/app/superset/views/base.py", line 252, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 487, in get_list_headless
duration, response = time_function(super().get_list_headless, **kwargs)
File "/app/superset/utils/core.py", line 1463, in time_function
response = func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/api/__init__.py", line 1613, in get_list_headless
response[API_RESULT_RES_KEY] = list_model_schema.dump(lst, many=True)
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 557, in dump
result = self._serialize(processed_obj, many=many)
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 519, in _serialize
return [
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 520, in <listcomp>
self._serialize(d, many=False)
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 525, in _serialize
value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
File "/usr/local/lib/python3.10/site-packages/marshmallow/fields.py", line 344, in serialize
return self._serialize(value, attr, obj, **kwargs)
File "/usr/local/lib/python3.10/site-packages/marshmallow/fields.py", line 1991, in _serialize
return self._serialize_method(obj)
File "/app/superset/queries/schemas.py", line 76, in get_sql_tables
return obj.sql_tables
File "/app/superset/models/sql_lab.py", line 75, in sql_tables
extract_tables_from_jinja_sql(
File "/app/superset/sql_parse.py", line 1110, in extract_tables_from_jinja_sql
| ParsedQuery(
File "/app/superset/sql_parse.py", line 280, in __init__
self._parsed = sqlparse.parse(self.stripped())
File "/app/superset/sql_parse.py", line 517, in stripped
return self.sql.strip(" \t\r\n;")
AttributeError: 'Template' object has no attribute 'strip'
Checklist
- [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [X] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
Comment From: rusackas
Hmmm... can't reproduce this one so far. Are you using any particular DB, or doing anything fancy with jinja?
Comment From: AutumnSun1996
The problem first appears with our test env, which use mysql:8. But I can reproduce it using sqlite, with reproduce steps above, so I think it's related to superset itself. Are u using the same docker-compose.yml? The image digest is sha256:0106a05a9e612df6c5972cc28e4712ff4a9ab2972665bad384aff6e5ef3112c8
Comment From: AutumnSun1996
and there is no use or modify with jinja, just some simple queries. BTW, I'm now force cast 'self.sql' to str inside the 'ParsedQuery.stripped' method, as a workaround to fix the http api. not sure if it's the right way, but it worked for my case.
Comment From: dagostinelli
I have this same problem. I experienced it during an upgrade. Old superset was based on an older Github commit. New superset is based on whatever is currently in Dockerhub. I backed up the database (Postgres 10) and restored it to the new database (Postgres 10) I performed a superset db upgrade followed by a superset init. I get that exact error in the query history page.
2024-04-24 01:30:07,715:DEBUG:superset.sql_parse:Parsing with sqlparse statement: Template(body=[Output(nodes=[TemplateData(data='SELECT * from t_page\nlimit 10')])])
2024-04-24 01:30:07,715:ERROR:superset.views.base:'Template' object has no attribute 'strip'
Traceback (most recent call last):
File "/app/superset/views/base.py", line 252, in wraps
return f(self, *args, **kwargs)
File "/app/superset/views/base_api.py", line 487, in get_list_headless
duration, response = time_function(super().get_list_headless, **kwargs)
File "/app/superset/utils/core.py", line 1463, in time_function
response = func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/flask_appbuilder/api/__init__.py", line 1613, in get_list_headless
response[API_RESULT_RES_KEY] = list_model_schema.dump(lst, many=True)
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 557, in dump
result = self._serialize(processed_obj, many=many)
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 519, in _serialize
return [
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 520, in <listcomp>
self._serialize(d, many=False)
File "/usr/local/lib/python3.10/site-packages/marshmallow/schema.py", line 525, in _serialize
value = field_obj.serialize(attr_name, obj, accessor=self.get_attribute)
File "/usr/local/lib/python3.10/site-packages/marshmallow/fields.py", line 344, in serialize
return self._serialize(value, attr, obj, **kwargs)
File "/usr/local/lib/python3.10/site-packages/marshmallow/fields.py", line 1991, in _serialize
return self._serialize_method(obj)
File "/app/superset/queries/schemas.py", line 76, in get_sql_tables
return obj.sql_tables
File "/app/superset/models/sql_lab.py", line 75, in sql_tables
extract_tables_from_jinja_sql(
File "/app/superset/sql_parse.py", line 1110, in extract_tables_from_jinja_sql
| ParsedQuery(
File "/app/superset/sql_parse.py", line 280, in __init__
self._parsed = sqlparse.parse(self.stripped())
File "/app/superset/sql_parse.py", line 517, in stripped
return self.sql.strip(" \t\r\n;")
AttributeError: 'Template' object has no attribute 'strip'
Here's a screenshot:
Comment From: Umkqq
I am experiencing the same issue. Using PostgreSQL version 12.18 and docker deployment based on latest superset image. Deployed from scratch and using empty database.
Issue is reproduced only when data exists to show on query history page.
Comment From: dagostinelli
I patched the file with this attached patch and I was able to get it working. I think this is a hack, but it's got me past the problem.
filename: sql_parse.patch
diff --git a/sql_parse.py b/sql_parse.py
index 62a2457..ceba521 100644
--- a/sql_parse.py
+++ b/sql_parse.py
@@ -514,7 +514,7 @@ class ParsedQuery:
return self._parsed[0].get_type() == "UNKNOWN"
def stripped(self) -> str:
- return self.sql.strip(" \t\r\n;")
+ return str(self.sql).strip(" \t\r\n;")
def strip_comments(self) -> str:
return sqlparse.format(self.stripped(), strip_comments=True)
I applied it as follows:
cd /app/superset/ &&patch sql_parse.py < sql_parse.patch
Comment From: Umkqq
I patched the file with this attached patch and I was able to get it working. I think this is a hack, but it's got me past the problem.
filename: sql_parse.patch
``` diff --git a/sql_parse.py b/sql_parse.py index 62a2457..ceba521 100644 --- a/sql_parse.py +++ b/sql_parse.py @@ -514,7 +514,7 @@ class ParsedQuery: return self._parsed[0].get_type() == "UNKNOWN"
def stripped(self) -> str:
- return self.sql.strip(" \t\r\n;")
return str(self.sql).strip(" \t\r\n;")
def strip_comments(self) -> str: return sqlparse.format(self.stripped(), strip_comments=True) ```
I applied it as follows:
cd /app/superset/ &&patch sql_parse.py < sql_parse.patch
That helped. Thanks a lot!
Comment From: rusackas
Pinging @michael-s-molina @dpgaspar in case they see a better way to provide a generalized fix/migration for others getting stuck. Otherwise, I hope the change above resolves this issue enough for now. 🤞
Comment From: gthomas-strike
We are encountering the same issue with a deployment using the latest 0.12.9 helm chart version
Comment From: ichux
I patched the file with this attached patch and I was able to get it working. I think this is a hack, but it's got me past the problem.
filename: sql_parse.patch
``` diff --git a/sql_parse.py b/sql_parse.py index 62a2457..ceba521 100644 --- a/sql_parse.py +++ b/sql_parse.py @@ -514,7 +514,7 @@ class ParsedQuery: return self._parsed[0].get_type() == "UNKNOWN"
def stripped(self) -> str:
- return self.sql.strip(" \t\r\n;")
return str(self.sql).strip(" \t\r\n;")
def strip_comments(self) -> str: return sqlparse.format(self.stripped(), strip_comments=True) ```
I applied it as follows:
cd /app/superset/ &&patch sql_parse.py < sql_parse.patch
This also worked for us but we just linked an edited version back into the container. Thanks
Comment From: toniphan21
I think this PR will also fix this problem.
Comment From: vivian-rook
This error still appears in an upgrade from superset 3.1.1 (helm chart 0.12.7) to 4.0.1 (helm chart 0.12.11)
Comment From: Habeeb556
Hopefully we can merge that soon as the issue still exists in the new version 4.0.2, but the sql_parse.patch provided by @dagostinelli has resolved it.
Comment From: Aaron-ML
We are seeing this issue on 4.0.2 also from a fresh deployment.
Comment From: shengbinxu
I patched the file with this attached patch and I was able to get it working. I think this is a hack, but it's got me past the problem. filename: sql_parse.patch ``` diff --git a/sql_parse.py b/sql_parse.py index 62a2457..ceba521 100644 --- a/sql_parse.py +++ b/sql_parse.py @@ -514,7 +514,7 @@ class ParsedQuery: return self._parsed[0].get_type() == "UNKNOWN"
def stripped(self) -> str:
- return self.sql.strip(" \t\r\n;")
return str(self.sql).strip(" \t\r\n;")
def strip_comments(self) -> str: return sqlparse.format(self.stripped(), strip_comments=True) ```
I applied it as follows:
cd /app/superset/ &&patch sql_parse.py < sql_parse.patchThis also worked for us but we just linked an edited version back into the container. Thanks
If you installed Superset using Docker Compose, you can follow these steps:
- Use the command
docker exec -it 5e7f2e42f256 bashto enter the container. - Modify the code as per the instructions above.
- Run
docker restart 5e7f2e42f256to restart the container.
Comment From: adangadang
bug
Comment From: rusackas
This has been silent for quite a > 6 months. Is this still an issue anyone's facing?
Comment From: rusackas
@Habeeb556 does your thumbs down mean it's no longer an issue, or you're upset I asked?
SqlParse is now gone from the codebase, as of a day or so ago. If anyone can test on current master it would be appreciated.