Bug description

Bug Description

Hello. When using the Deck.gl Multiple Layers chart, the geometry on the map is not updated after the underlying dataset changes.

  1. I create a dataset in Superset that returns latitude and longitude.
  2. I use this dataset to create a Deck.gl Scatter chart.

  3. I then create a Deck.gl Multiple Layers chart that includes the Scatter chart from step (2). At this point, both the Deck.gl Scatter chart and the Deck.gl Multiple Layers chart correctly display points on the map. However, after I change the dataset so that it returns different records, the Scatter chart updates correctly while the Deck.gl Multiple Layers chart does not update—it continues to display the old points.

  4. OS: Ubuntu 22.04.5

  5. Superset version: 4.1.1
  6. DB: PostgreSQL

Detailed Steps to Reproduce

  1. Create a dataset (test-dataset) with the following SQL (this initially returns 2 records):
SELECT *
FROM (VALUES (36.54, 136.97, 1),
             (36.70, 137.10, 2)
      ) AS t(lat, lng, id);

  1. From test-dataset, create a Deck.gl Scatter Map chart (test-scatter). You will see two points plotted on the map.

  2. Create a Deck.gl Multiple Layers chart (test-multi-layer) that includes:

  3. Dataset: test-dataset

  4. Deck.gl Chart: test-scatter At this stage, both charts (Scatter and Multiple Layers) properly display two points on the map.

  5. Update test-dataset so that the SQL now returns only 1 record:

SELECT *
FROM (VALUES (36.54, 136.97, 1),
             (36.70, 137.10, 2)
      ) AS t(lat, lng, id)
WHERE id = 1;

After this change, the Scatter chart updates correctly and displays only 1 point. However, the Deck.gl Multiple Layers chart still shows the original 2 points and does not update.

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.9

Node version

16

Browser

Chrome

Additional context

No response

Checklist

  • [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
  • [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
  • [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Comment From: dosubot[bot]

Hey @devkenji! 👋 I'm here to help you with bugs, questions, and becoming a contributor.

The issue you're experiencing with the Deck.gl Multiple Layers chart not updating after a dataset change might be related to the caching mechanism in Superset. Superset uses caching to improve performance, and this can sometimes lead to charts not reflecting the latest data changes immediately.

Here are a few things you can check or try:

  1. Cache Settings: Verify if the caching settings for your chart are causing the old data to persist. You can check the cache_timeout setting for your chart and adjust it if necessary. The default cache timeout might be causing the data to remain cached for a longer period than desired [1].

  2. Cache Warm-Up: Superset has a cache warm-up feature that can refresh the cache for charts. Ensure that this feature is configured correctly to update the cache at appropriate intervals [2].

  3. Force Refresh: Try forcing a refresh of the chart to see if it updates with the new data. This can sometimes be done by clearing the cache manually or by using any available options in the Superset UI to refresh the chart.

If these steps do not resolve the issue, it might be necessary to investigate further into the specific configuration of the Deck.gl Multiple Layers chart and how it handles data updates in your Superset setup.

To continue talking to Dosu, mention @dosu.

Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

Comment From: devkenji

Thank you. I tried steps 1, 2, and 3, but none of them worked. I also modified the cache settings in the GUI and in superset_config.py, but that didn't help. I'm using Redis for caching, and even manually clearing the Redis cache had no effect.

Comment From: rusackas

Hmm... I was suspecting a caching issue as well, so I'm not sure. I confess, I don't use the multi-layer plugin often. I wonder if @Mattc1221 @jansule @villebro have any insights here.

Comment From: devkenji

I also think the cache function is the cause. Deck.gl Multi Layer Chart is a very good feature, but unfortunately it cannot be used when data changes dynamically due to this phenomenon.

Comment From: rusackas

Are you able to shorten the cache life on both underlying charts, so each updates more often? I'm not sure whether or not that would effectively solve your use case.