Screenshot
Description
In the Apache Superset web interface, I have set D3Format for my data, which displays correctly on the web page. However, when I generate a report for a chart and specify text content, the data in the received email loses its formatting.
Design input
I hope the following formatted content can also be realized in the text format of emails. .1s (12345.432 => 10k) .3s (12345.432 => 12.3k) .4r (12345.432 => 12350) "+, (12345.432 => +12,345) " $,.2f (12345.432 => $12,345.43) Duration in ms (66000 => 1m 6s) Duration in ms (1.40008 => 1ms 400µs 80ns) %d/%m/%Y | 14/01/2019 %m/%d/%Y | 01/14/2019 %Y-%m-%d | 2019-01-14 %d-%m-%Y %H:%M:%S | 14-01-2019 01:32:10 %H:%M:%S | 01:32:10
Comment From: rusackas
Hmm... I don't really use the text-based reports, so I'll CC @yousoph here for more context/info. Did they ever support these formatters, or is it just the raw data from the database? The date formatters are picked up by the various visualizations to use as they need to... if we need to add support for formatters in text-based reports where it didn't exist before, then that seems more like a new feature request.
Comment From: liangliangGit
Hmm... I don't really use the text-based reports, so I'll CC @yousoph here for more context/info. Did they ever support these formatters, or is it just the raw data from the database? The date formatters are picked up by the various visualizations to use as they need to... if we need to add support for formatters in text-based reports where it didn't exist before, then that seems more like a new feature request.
Yes, the data is retrieved from the database, and some formats have been implemented while others haven't. Here's a summary of my findings: ,d (12345.432 => 12,345) NG .1s (12345.432 => 10k) NG .3s (12345.432 => 12.3k) NG ,.1% (12345.432 => 1,234,543.2%) OK .2% (12345.432 => 1234543.20%) OK .3% (12345.432 => 1234543.200%) OK .4r (12345.432 => 12350) NG ,.1f (12345.432 => 12,345.4) OK ,.2f (12345.432 => 12,345.43) OK ,.3f (12345.432 => 12,345.432) OK "+, (12345.432 => +12,345) " NG $,.2f (12345.432 => $12,345.43) NG Duration in ms (66000 => 1m 6s) NG Duration in ms (1.40008 => 1ms 400µs 80ns) NG %d/%m/%Y | 14/01/2019 NG %m/%d/%Y | 01/14/2019 NG %Y-%m-%d | 2019-01-14 NG %Y-%m-%d %H:%M:%S | 2019-01-14 01:32:10 OK %d-%m-%Y %H:%M:%S | 14-01-2019 01:32:10 NG %H:%M:%S | 01:32:10 NG Prefix ¥ (JPY) NG Suffix ¥ (JPY) NG
Comment From: rusackas
Oh, interesting... I didn't know it was a mixed bag... I'm not sure why some are being picked up, but not others.
CC @eschutho in case you happen to know which gear isn't meshing.
Comment From: liangliangGit
Below is a section of Python backend formatting code (found within the 'post_processing.py' file). It is deemed accurate if the Python format method aligns exactly with the d3format syntax, for instance, when using .1%.
def table(
df: pd.DataFrame,
form_data: dict[str, Any],
datasource: Optional[ # pylint: disable=unused-argument
Union["BaseDatasource", "Query"]
] = None,
) -> pd.DataFrame:
"""
Table.
"""
# apply `d3NumberFormat` to columns, if present
column_config = form_data.get("column_config", {})
for column, config in column_config.items():
if "d3NumberFormat" in config:
format_ = "{:" + config["d3NumberFormat"] + "}"
try:
df[column] = df[column].apply(format_.format)
except Exception: # pylint: disable=broad-except
# if we can't format the column for any reason, send as is
pass
return df
Comment From: rusackas
This issue is at risk of going stale. Meanwhile pinging @kgabryje who looked at number formatters of all sorts not so long ago, and @yousoph / @Vitor-Avila to see if this is on their collective radar at all.
Comment From: eschutho
It seems that this is a bug. We'll be looking into adding D3 formatting for exports. It's possible it may fix this as a side effect.
Comment From: Vitor-Avila
I heard this in the past from users as well (looking for the ability to export the formatted data from Superset) but it's not on my todo for now.
Comment From: rusackas
I'm assuming this is still the case, but if anyone can validate this is still the case in 4.1.x/5.0.0, that would be appreciated.