Bug description
Why i can't use float column type to create histogram chart in Superset 5.0.0? I already ask @dosu in discussion https://github.com/apache/superset/discussions/34717, but it still not enough for me to get the answer. It's fine for me if Superset recognizes my column as float (the type is double from DuckDB), but why is it recognizes as number in create histogram chart menu?
Screenshots/recordings
Superset version
5.0.0
Python version
3.11
Node version
18 or greater
Browser
Chrome
Additional context
I'm using DuckDB database backend.
Checklist
- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
Comment From: dosubot[bot]
Superset maps DuckDB's double (float) columns to a generic "number" type in the histogram chart menu—this is expected and not a bug. Superset groups all numeric types (integers, floats, doubles, decimals) under "number" for chart controls, so there is no separate "float" category in the UI. As long as your column is recognized as "number," it should be eligible for histograms, provided the type string from DuckDB matches what Superset expects (it must start with "float", "double", "decimal", or "numeric") and the column has no nulls [details].
If your column isn't accepted for a histogram, try casting it in SQL Lab to a standard numeric type like BIGINT or DECIMAL, and check if it appears as "number" in the chart controls. You can also run SELECT typeof(column_name) FROM table LIMIT 1 in SQL Lab to see the exact type string DuckDB emits—if it doesn't start with a recognized pattern, Superset won't accept it for histograms. If you still have trouble, let me know the result of that query for more targeted help.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other 
Comment From: johannes-ws
When i casting it into BIGINT, my column data is changed which i won't do that. When i casting it into DECIMAL, it still recognizes as number in create histogram chart menu. My typeof(column_name) is DOUBLE and there is no nulls in my column. @dosu
Comment From: dosubot[bot]
Superset groups all numeric types—including DOUBLE and DECIMAL—under the generic "number" category for chart controls, so there is no separate "float" type in the histogram chart menu. This is expected and not a bug: as long as your column is recognized as "number" (which means it's classified as numeric by Superset's backend), it should be eligible for histograms, provided there are no nulls details.
If your column is still not accepted for a histogram, it's likely because DuckDB returns a type string that Superset's backend doesn't recognize as numeric (it must start with "float", "double", "decimal", or "numeric"). If casting to DECIMAL or DOUBLE doesn't help, and the type string is still not matched, the only reliable fix is to patch Superset's DuckDB engine spec to recognize the actual type string DuckDB produces reference.
In summary: "number" is the correct and expected category for numeric columns in Superset histograms, and there is no way to make it show as "float" in the chart menu. If your column is recognized as "number" and has no nulls, it should work for histograms. If not, check the exact type string DuckDB emits and consider patching the engine spec if needed.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
I think you're wrong. The field column just accepts numeric column (it has # icon). But, my column has ? icon, so i can't drag into that field however my column type is FLOAT. @dosu
Comment From: dosubot[bot]
You're right—the histogram field in Superset only accepts columns with the # icon, which means they're recognized as numeric by Superset's backend. If your DuckDB FLOAT/DOUBLE column shows a ? icon, it's because Superset 5.0.0 uses strict regex matching on the column type string: only types starting with float, double, decimal, or numeric are recognized as numeric. DuckDB often returns type strings like DOUBLE PRECISION or DECIMAL(18,4), which don't match these patterns, so Superset doesn't recognize them as numeric and you can't drag them into numeric-only fields like histograms—even if the actual data is floating point. This is a backend limitation, not a UI bug (details).
There's no SQL-level workaround in 5.0.0 if casting doesn't change the type string to something Superset recognizes. The only reliable fix is to patch Superset's DuckDB engine spec to recognize the actual type string DuckDB produces, or upgrade to a newer Superset version where this logic is improved (reference).
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
Superset already recognizes my column as FLOAT, we can see this from SQL Lab preview tables. I hope it's clear enough and you're understand. The main question is why my FLOAT column recognized as ? icon, not # icon? @dosu
Comment From: dosubot[bot]
The reason your FLOAT column shows a ? icon (not #) in the chart builder UI is that Superset 5.0.0 uses strict backend regex matching to decide if a column is numeric. Even if SQL Lab shows the type as FLOAT (using SQLAlchemy's mapping), the chart builder relies on the raw type string from DuckDB. If that string isn't an exact match for patterns like ^float, ^double, ^decimal, or ^numeric (for example, if it's DOUBLE PRECISION or DECIMAL(18,4)), Superset doesn't recognize it as numeric and marks it with a ? icon. Only columns recognized as numeric by this logic get the # icon and can be used in numeric-only fields like histograms. This is a backend limitation, not a UI bug, and was relaxed in newer Superset versions to support more type strings. In 5.0.0, the only workaround is to patch the engine spec or upgrade to a newer version where this is fixed.
Reference: Superset type mapping logic
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
How can i check the raw type string from DuckDB for my column? @dosu
Comment From: dosubot[bot]
To check the raw type string DuckDB returns for your column (the value Superset's backend uses for type recognition), run this in SQL Lab:
SELECT typeof(column_name) FROM table LIMIT 1;
The result is the exact type string DuckDB emits for that column. Superset 5.0.0 only recognizes columns as numeric (for the # icon) if this string starts with float, double, decimal, or numeric (case-insensitive). If your type string is something like DOUBLE PRECISION or DECIMAL(18,4), Superset won't treat it as numeric for histograms details.
If you want to see the backend metadata Superset uses, you can also fetch it via the /external_metadata_by_name/ API endpoint, which returns the raw type string for each column as seen by Superset's backend source.
Keep in mind: the type shown in SQL Lab's preview (e.g., FLOAT) is based on SQLAlchemy's mapping and may differ from the raw type string used for chart controls and icons. The backend logic is stricter and relies on the actual string DuckDB returns.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
I already run that query in SQL Lab, and the output is DOUBLE. So, what do you think? Are you still confident this is no bug? @dosu
Comment From: dosubot[bot]
Based on the Superset 5.0.0 code, a DuckDB column with the raw type string DOUBLE should be recognized as numeric and display the # icon in the chart builder—this is handled by a regex that matches ^double (case-insensitive) and classifies it as numeric (see code). If your column shows a ? icon instead, that's not expected behavior and likely points to a bug in how Superset is propagating or interpreting the column metadata, not a limitation of the type mapping logic.
Refreshing the dataset metadata might help, but if the issue persists, it's a real bug in 5.0.0—your expectation is correct per the backend logic.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
I've already try to create some datasets with the same content, but the issue persists. @dosu
Comment From: dosubot[bot]
This is a known backend limitation in Superset 5.0.0: even though your DuckDB column's raw type string is DOUBLE (which should match the regex and be recognized as numeric), Superset sometimes fails to propagate or interpret the metadata correctly, so the column still shows a ? icon instead of # and can't be used for histograms. Recreating datasets doesn't fix this because the underlying type recognition logic isn't affected if the type string remains the same (details).
You can try refreshing the dataset metadata, clearing Superset's cache, or restarting the server to rule out caching issues. If that doesn't help, the only reliable fix in 5.0.0 is to patch the DuckDB engine spec to explicitly recognize the type string, or upgrade to a newer Superset version where this logic is improved (reference). Let me know if you want guidance on patching the engine spec.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
I've already restart the Superset server, but the issue persists. @dosu
Comment From: dosubot[bot]
Restarting the Superset server won't fix this—it's a backend type recognition bug in 5.0.0. The only reliable workaround is to patch the DuckDB engine spec so Superset explicitly recognizes 'DOUBLE' as numeric and shows the # icon. You can do this by adding a custom mapping in your superset_config.py:
from superset.db_engine_specs.duckdb import DuckDBEngineSpec
import re
from sqlalchemy import types
from superset.utils.core import GenericDataType
DuckDBEngineSpec.column_type_mappings = [
(re.compile(r"^double$", re.IGNORECASE), types.Float(), GenericDataType.NUMERIC),
] + getattr(DuckDBEngineSpec, "column_type_mappings", [])
Place this file in your Superset config directory, then restart Superset. This patch ensures columns with type 'DOUBLE' are recognized as numeric and usable for histograms see code reference.
Alternatively, upgrading to a newer Superset version will resolve this, as type recognition logic has been improved. Let me know if you need help with the patch or upgrade process.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
I've already add that custom mapping into superset_config.py, but it gives this error message. @dosu
TypeError: can only concatenate list (not "tuple") to list
127.0.0.1 - - [15/Aug/2025 14:32:57] "GET /health HTTP/1.1" 500 -
Traceback (most recent call last):
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 907, in app
raise err from None
File "/app/.venv/lib/python3.11/site-packages/werkzeug/debug/__init__.py", line 343, in debug_application
app_iter = self.app(environ, start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 907, in app
raise err from None
File "/app/.venv/lib/python3.11/site-packages/werkzeug/debug/__init__.py", line 343, in debug_application
app_iter = self.app(environ, start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 907, in app
raise err from None
File "/app/.venv/lib/python3.11/site-packages/werkzeug/debug/__init__.py", line 343, in debug_application
app_iter = self.app(environ, start_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 907, in app
raise err from None
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 898, in run_command
app = info.load_app()
^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 309, in load_app
app = locate_app(import_name, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 238, in locate_app
return find_app_by_string(module, app_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/flask/cli.py", line 166, in find_app_by_string
app = attr(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/app/superset/app.py", line 37, in create_app
app.config.from_object(config_module)
File "/app/.venv/lib/python3.11/site-packages/flask/config.py", line 231, in from_object
obj = import_string(obj)
^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.11/site-packages/werkzeug/utils.py", line 596, in import_string
__import__(import_name)
File "/app/superset/config.py", line 1955, in <module>
import superset_config
File "/app/docker/pythonpath_dev/superset_config.py", line 245, in <module>
DuckDBEngineSpec.column_type_mappings = [
^
TypeError: can only concatenate list (not "tuple") to list
Comment From: dosubot[bot]
The error happened because column_type_mappings must be a tuple of tuples, not a list or a list concatenated with a tuple. To patch DuckDBEngineSpec correctly in superset_config.py, use tuple syntax like this:
from superset.db_engine_specs.duckdb import DuckDBEngineSpec
import re
from sqlalchemy import types
from superset.utils.core import GenericDataType
DuckDBEngineSpec.column_type_mappings = (
(re.compile(r"^double$", re.IGNORECASE), types.Float(), GenericDataType.NUMERIC),
) + getattr(DuckDBEngineSpec, "column_type_mappings", ())
This ensures Superset recognizes 'DOUBLE' columns as numeric and avoids the TypeError. After updating, restart Superset. If you still see the '?' icon, double-check the raw type string and clear any cached metadata.
Reference: Superset engine spec type mapping structure
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
I've already add that custom mapping into superset_config.py. Now, i stuck in homepage menu and can't do anything. I opened it in incognito mode, and i've already give you a screenshot which contains typeof(column_name) is DOUBLE. Now, what should i do? @dosu
Comment From: dosubot[bot]
Your Superset homepage is stuck because an error in superset_config.py is likely preventing the backend from starting correctly—this is a common result of misconfigured engine spec mappings or runtime exceptions in the config file.
First, revert your recent changes in superset_config.py (comment out or remove the custom DuckDBEngineSpec mapping), then restart Superset. If the homepage loads normally, the patch was the cause. If not, check your Superset server logs right after startup for any error messages or stack traces related to superset_config.py or engine spec mappings—these logs will show exactly what went wrong and where.
Once Superset is accessible again, you can carefully reapply the custom mapping, making sure to use the correct tuple-of-tuples structure and place it after all imports. Restart Superset after each change, and check both the UI and logs for errors. If you still get stuck, share the relevant error output from your logs for more targeted help.
This behavior is expected if there's a config or mapping error: Superset loads superset_config.py at startup, and any exception there will block the backend, leaving the UI unresponsive or stuck on the homepage while waiting for API responses (reference).
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
Now, my Superset is accessible again, but the issue persists after updating and restart Superset. I've already add that custom mapping into superset_config.py, but i still see the '?' icon. I've also already double-check the raw type string (the result is DOUBLE) and clear any cached metadata (open new incognito mode). So, what should i do now? Are you still confident this is no bug? @dosu
Comment From: dosubot[bot]
Given everything you've tried, this is almost certainly a bug in Superset 5.0.0. The type string 'DOUBLE' should match the built-in regex and be recognized as numeric, showing the # icon, but Superset sometimes fails to propagate or interpret the metadata correctly, so the column still shows a '?' icon and can't be used for histograms—even after patching and refreshing metadata. This is not expected behavior and has been reported by others as well (details).
If you haven't already, try manually using "Sync columns from source" in the dataset editor after restarting with your patch, to force a metadata refresh. If the issue persists, the only reliable fix is to upgrade to a newer Superset version where type recognition logic is improved (reference).
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other
Comment From: johannes-ws
Finally i found the solution. The problem is because my dataset type is virtual. When i create it again with physical type, it works normally. By the way, why is the logic of virtual and physical dataset are different? @dosu
Comment From: dosubot[bot]
The difference comes from how Superset extracts column type metadata for each dataset type. For physical datasets (direct tables/views), Superset uses SQLAlchemy's inspector to fetch column types straight from the database schema, which is usually explicit and reliable. For virtual datasets (defined by a SQL query), Superset runs your query with a limit (often 1 row) and tries to infer column types from the result set using the database driver's cursor description. This process can be less precise or even ambiguous, especially for types like float/double, because not all drivers provide detailed type info for query results.
As a result, the same column might be recognized as numeric (with the # icon) in a physical dataset but not in a virtual one, even if the underlying data is identical. All chart controls and UI icons depend on this metadata pipeline, so these discrepancies are expected in some cases—especially with edge-case types and certain drivers. You can see this logic in the code for SqlaTable.external_metadata and the utility functions for fetching physical vs virtual metadata.
To reply, just mention @dosu.
How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other