Apache Superset [SIP-177] (WIP) Proposal for SQL Lab Extensions

[SIP-177] Proposal for SQL Lab Extensions

Motivation Proposed change    The vision    Architectural principles    High-level architecture    Extension project structure    Extension metadata    Frontend contribution types       Views       Commands       Menus    Interacting with the host    Dynamic module loading    Deploying an extension    Lifecycle and management    Development mode    Versioning    Security implications and responsibilities    What this means for Superset's built-in features    Real extension example New or changed public interfaces New dependencies Migration plan and compatibility    Key sub-projects Downsides and risks Rejected alternatives Future work Special thanks

Motivation

Superset’s evolution and adoption have highlighted the limitations of a monolithic architecture. As the platform’s user base has grown, so have the demands for advanced features, custom workflows, and integrations. Many organizations require capabilities such as query explainers, optimizers, or natural language support, but the absence of a robust extension model has often forced teams to fork the codebase or make invasive modifications. This leads to significant maintenance overhead, fragmentation, and a barrier to broader community contributions.

This SIP is a direct follow-up to SIP-151 and builds on the direction established in previous discussions/proposals. The primary motivation is to establish a foundation for extensibility in SQL Lab by defining clear patterns and constructs for extensions. The objective at this stage is not to fully implement all possible APIs or extension points, but rather to continue toward a flexible architecture (using SQL lab as a starting point) and a set of guiding principles that the community can iterate on and expand over time. This approach encourages experimentation and shared ownership, allowing extensions to evolve organically as new needs and use cases emerge. While the initial focus is on SQL Lab, the intention is to reuse these patterns and APIs across other modules in Superset, such as dashboards, charts, and connectors, to promote consistency and extensibility throughout the platform.

During the course of this work, we decided to change the terminology from "Plugins" to "Extensions." This change reflects a more accurate description of the intended architecture and philosophy. While "plugin" often implies a narrowly scoped add-on or isolated feature, "extension" better conveys the idea of enhancing and expanding the core capabilities of Superset in a modular and integrated way. The term "extension" also aligns with broader industry conventions, emphasizing the goal of providing well-defined, versioned points for the community to build upon and extend the platform’s functionality.

The design of this extension architecture draws inspiration from the successful model established by VS Code Extensions. VS Code has demonstrated how a thoughtfully designed extension architecture can empower a broad community to innovate, share, and maintain enhancements without burdening the core maintainers or fragmenting the platform. By adopting similar principles—such as clear contribution points, versioned APIs, and a modular approach—this SIP aims to create an environment where the Superset community can collaboratively define and expand the platform’s capabilities over time.

Proposed change

The vision

Our proposal is to refactor SQL Lab to act as a host for extensions, allowing developers to create and share custom features that enhance the core functionality of the platform. While this SIP focuses on SQL Lab as the initial host for extensions, the intention is to introduce the foundational framework for extensibility across Superset as a whole. The patterns, APIs, and mechanisms described here are designed to be reusable and extensible, paving the way for future expansion to other modules such as dashboards and charts. The following prototype illustrates the vision:

Extension authors will be able to contribute new UI elements, commands, and menu options using well-defined APIs, while the host application manages the full lifecycle of these extensions. The architecture will support customization of five key areas within SQL Lab: the left-side panel, right-side panel, editor panel, bottom panels, and the status bar. As part of this effort, the built-in left-side panel will be refactored in accordance with SIP-111, modernizing data exploration and improving the overall user experience.

Examples of possible SQL Lab extensions include panels that display rewritten SQL for virtual query layers, integrations with external query explainers or optimizers, custom autocomplete providers, natural language to SQL assistants, or tools that visualize database schemas and execution plans. Extensions could also add new menu actions, context-aware commands, or even replace the default SQL editor with an alternative such as Monaco Editor, as explored in recent prototypes. This flexibility will allow organizations and the community to tailor SQL Lab to their specific workflows and analytical needs.

Architectural principles

Realizing this vision requires a strong architectural foundation. To ensure the resulting system is robust, maintainable, and adaptable, we have defined a set of architectural principles that will guide the design and implementation of the changes. These principles serve as the basis for all technical decisions and help create an environment where extensions can be developed safely and predictably, while minimizing technical debt and fragmentation.

The architectural principles guiding this proposal include:

1. Lean core: Superset’s core should remain as minimal as possible, with many features and capabilities delegated to extensions. Wherever possible, built-in features should be implemented using the same APIs and extension mechanisms available to external extension authors. This approach reduces maintenance burden and complexity in the core application, encourages modularity, and allows the community to innovate and iterate on features independently of the main codebase.
2. Explicit contribution points: All extension points must be clearly defined and documented, so extension authors know exactly where and how they can interact with the host system. Each extension must also declare its capabilities in a metadata file, enabling the host to manage the extension lifecycle and provide a consistent user experience.
3. Versioned and stable APIs: Public interfaces for extensions should be versioned and follow semantic versioning, allowing for safe evolution and backward compatibility.
4. Lazy loading and activation: Extensions should be loaded and activated only when needed, minimizing performance overhead and resource consumption.
5. Composability and reuse: The architecture should encourage the reuse of extension points and patterns across different modules, promoting consistency and reducing duplication.
6. Community-driven evolution: The system should be designed to evolve based on real-world feedback and contributions, allowing new extension points and capabilities to be added as needs emerge.

High-level architecture

It is important to note that this SIP is not intended to address every aspect of the architecture in a single document. Rather, it should be seen as a chapter in an ongoing book: it establishes foundational concepts and direction, while recognizing that many important topics remain to be explored. Items not covered here are listed under the Future Work section and are expected to be addressed in subsequent SIPs as the architecture and community needs evolve.

The following diagram illustrates the overall architecture, emphasizing the relationships between the host application, extensions, and the supporting packages that will be developed to enable this ecosystem.

On the frontend side, as a result of discussions in [SIP-169] Proposal for Extracting and Publishing Superset Core UI Components, the @apache-superset/core package will be created to provide all the essential building blocks for both the host application and extensions, including shared UI components, utility functions, APIs, and type definitions. By centralizing these resources, extensions and built-in features can use the same APIs and components, ensuring consistency, type safety, and a seamless user experience across the Superset ecosystem. This package will be versioned to support safe evolution of the platform while maintaining compatibility for both internal and external features.

On the backend side, the apache-superset-core package will expose key classes and APIs needed by extensions that provide backend functionality such as extending Superset's API, providing database connectors, or customizing the security manager. It will contain dependencies on critical libraries like FAB, SQLAlchemy, etc and like @apache-superset/core, it will be versioned to ensure compatibility and stability.

The apache-superset-extensions-cli package will provide a comprehensive set of CLI commands for extension development, including tools for code generation, building, and packaging extensions. These commands streamline the development workflow, making it easier for developers to scaffold new projects, build and bundle their code, and distribute extensions efficiently. By standardizing these processes, the CLI helps ensure that extensions are built in a consistent manner, remain compatible with evolving versions of Superset, and adhere to best practices across the ecosystem.

Extension projects might have references to all packages, depending on their specific needs. For example, an extension that provides a custom SQL editor might depend on the apache-superset-core package for backend functionality, while also using the @apache-superset/core package for UI components and type definitions. This modular approach allows extension authors to choose the dependencies that best suit their needs, while also promoting consistency and reusability across the ecosystem.

The host application (Superset) will depend on core packages and is responsible for providing the implementation of the APIs defined in them. It will expose a new endpoint (/api/v1/extensions) for extension registration and management, and a dedicated UI for managing extensions. Registered extensions will be stored in the metadata database in a table called extensions, which will contain information such as the extension's name, version, author, contributed features, exposed modules, relevant metadata and the built frontend and/or backend code.

Extension project structure

The apache-superset-extensions-cli package provides a command-line interface (CLI) that streamlines the extension development workflow. It offers the following commands:

superset-extensions init: Generates the initial folder structure and scaffolds a new extension project.

superset-extensions build: Builds extension assets.

superset-extensions bundle: Packages the extension into a .supx file.

superset-extensions dev: Automatically rebuilds the extension as files change.

When creating a new extension with superset-extensions init <extension-name>, the CLI generates a standardized folder structure:

dataset_references/
├── extension.json
├── frontend/
│   ├── src/
│   ├── webpack.config.js
│   ├── tsconfig.json
│   └── package.json
├── backend/
│   ├── src/
│        └── dataset_references/
│   ├── tests/
│   ├── pyproject.toml
│   └── requirements.txt
├── dist/
│   ├── manifest.json
│   ├── frontend
│        └── dist/
│             ├── remoteEntry.d7a9225d042e4ccb6354.js
│             └── 900.038b20cdff6d49cfa8d9.js
│   └── backend
│        └── dataset_references/
│             ├── __init__.py
│             ├── api.py
│             └── entrypoint.py
├── dataset_references-1.0.0.supx
└── README.md

The extension.json file serves as the declared metadata for the extension, containing the extension's name, version, author, description, and a list of capabilities. This file is essential for the host application to understand how to load and manage the extension.

The frontend directory contains the source code for the frontend components of the extension, including React components, styles, and assets. The webpack.config.js file is used to configure Webpack for building the frontend code, while the tsconfig.json file defines the TypeScript configuration for the project. The package.json file specifies the dependencies and scripts for building and testing the frontend code.

The backend directory contains the source code for the backend components of the extension, including Python modules, tests, and configuration files. The pyproject.toml file is used to define the Python package and its dependencies, while the requirements.txt file lists the required Python packages for the extension. The src folder contains the functional backend source files, tests directory contains unit tests for the backend code, ensuring that the extension behaves as expected and meets the defined requirements.

The dist directory is built when running the build or dev command, and contains the files that will be included in the bundle. The manifest.json file contains critical metadata about the extension, including the majority of the contents of the extension.json file, but also other build-time information, like the name of the built Webpack Module Federation remote entry file. The files in the dist directory will be zipped into the final .supx file. Although this file is technically a zip archive, the .supx extension makes it clear that it is a Superset extension package and follows a specific file layout. This packaged file can be distributed and installed in Superset instances.

The README.md file provides documentation and instructions for using the extension, including how to install, configure, and use its functionality.

Extension metadata

The extension.json file contains all metadata necessary for the host application to understand and manage the extension:

{
  "name": "dataset_references",
  "version": "1.0.0",
  "frontend": {
    "contributions": {
      "views": {
        "sqllab.panels": [
          {
            "id": "dataset_references.main",
            "name": "Dataset references"
          }
        ]
      }
    },
    "moduleFederation": {
      "exposes": ["./index"]
    }
  },
  "backend": {
    "entryPoints": ["dataset_references.entrypoint"],
    "files": ["backend/src/dataset_references/**/*.py"]
  },
}

The contributions section declares how the extension extends Superset's functionality through views, commands, menus, and other contribution types. The backend section specifies entry points and files to include in the bundle.

Frontend contribution types

To facilitate the development of extensions, we will define a set of well-defined contribution types that extensions can implement. These contribution types will serve as the building blocks for extensions, allowing them to interact with the host application and provide new functionality. The initial set of contribution types will include:

Views

Extensions can add new views or panels to the host application, such as custom SQL Lab panels, dashboards, or other UI components. Each view is registered with a unique ID and can be activated or deactivated as needed. Contribution areas are uniquely identified (e.g., sqllab.panels for SQL Lab panels), enabling seamless integration into specific parts of the application.

"views": {
  "sqllab.panels": [
    {
      "id": "dataset_references.main",
      "name": "Table references"
    }
  ]
},

Commands

Extensions can define custom commands that can be executed within the host application, such as context-aware actions or menu options. Each command can specify properties like a unique command identifier, an icon, a title, and a description. These commands can be invoked by users through menus, keyboard shortcuts, or other UI elements, enabling extensions to add rich, interactive functionality to Superset.

"commands": [
  {
    "command": "extension1.copy_query",
    "icon": "CopyOutlined",
    "title": "Copy Query",
    "description": "Copy the current query to clipboard"
  },
]

Menus

Extensions can contribute new menu items or context menus to the host application, providing users with additional actions and options. Each menu item can specify properties such as the target view, the command to execute, its placement (primary, secondary, or context), and conditions for when it should be displayed. Menu contribution areas are uniquely identified (e.g., sqllab.editor for the SQL Lab editor), allowing extensions to seamlessly integrate their functionality into specific menus and workflows within Superset.

"menus": {
  "sqllab.editor": {
    "primary": [
      {
        "view": "builtin.editor",
        "command": "extension1.copy_query"
      }
    ],
    "secondary": [
      {
        "view": "builtin.editor",
        "command": "extension1.prettify"
      }
    ],
    "context": [
      {
        "view": "builtin.editor",
        "command": "extension1.clear"
      },
      {
        "view": "builtin.editor",
    "command": "extension1.refresh"
      }
    ]
  },
}

Interacting with the host

Extensions interact with Superset through well-defined, versioned APIs provided by the @apache-superset/core (frontend) and apache-superset-core (backend) packages. These APIs are designed to be stable, discoverable, and consistent for both built-in and external extensions.

Frontend APIs (via @apache-superset/core):

The frontend extension APIs in Superset are organized into logical namespaces such as authentication, commands, extensions, sqlLab, and others. Each namespace groups related functionality, making it easy for extension authors to discover and use the APIs relevant to their needs. For example, the sqlLab namespace provides events and methods specific to SQL Lab, allowing extensions to react to user actions and interact with the SQL Lab environment:

export const getCurrentTab: () => Tab | undefined;

export const getDatabases: () => Database[];

export const getTabs: () => Tab[];

export const onDidChangeEditorContent: Event<string>;

export const onDidClosePanel: Event<Panel>;

export const onDidChangeActivePanel: Event<Panel>;

export const onDidChangeTabTitle: Event<string>;

export const onDidQueryRun: Event<Editor>;

export const onDidQueryStop: Event<Editor>;

The following code demonstrates more examples of the existing frontend APIs:

import { core, commands, sqlLab, authentication, Button } from '@apache-superset/core';
import MyPanel from './MyPanel';

export function activate(context) {
  // Register a new panel (view) in SQL Lab and use shared UI components in your extension's React code
  const panelDisposable = core.registerView('my_extension.panel', <MyPanel><Button/></MyPanel>);

  // Register a custom command
  const commandDisposable = commands.registerCommand('my_extension.copy_query', {
    title: 'Copy Query',
    execute: () => {
      // Command logic here
    },
  });

  // Listen for query run events in SQL Lab
  const eventDisposable = sqlLab.onDidQueryRun(editor => {
    // Handle query execution event
  });

  // Access a CSRF token for secure API requests
  authentication.getCSRFToken().then(token => {
    // Use token as needed
  });

  // Add all disposables for automatic cleanup on deactivation
  context.subscriptions.push(panelDisposable, commandDisposable, eventDisposable);
}

Backend APIs (via apache-superset-core):

Backend APIs follow a similar pattern, providing access to Superset's models, sessions, and query capabilities. Extensions can register REST API endpoints, access the metadata database, and interact with Superset's core functionality.

Extension endpoints are registered under a dedicated /extensions namespace to avoid conflicting with built-in endpoints and also because they don't share the same version constraints. By grouping all extension endpoints under /extensions, Superset establishes a clear boundary between core and extension functionality, making it easier to manage, document, and secure both types of APIs.

from superset_core.api import rest_api, models, query
from .api import DatasetReferencesAPI

# Register a new extension REST API
rest_api.add_extension_api(DatasetReferencesAPI)

# Access Superset models with simple queries that filter out entities that
# the user doesn't have access to
databases = models.get_databases(id=database_id)
if not databases:
    return self.response_404()

database = databases[0]

# Perform complex queries using SQLAlchemy BaseQuery, also filtering
# out inaccessible entities
session = models.get_session()
db_model = models.get_database_model())
database_query = session.query(db_model.database_name.ilike("%abc%")
databases_containing_abc = models.get_databases(query)

# Bypass security model for highly custom use cases
session = models.get_session()
db_model = models.get_database_model())
all_databases_containg_abc = session.query(db_model.database_name.ilike("%abc%").all()

In the future, we plan to expand the backend APIs to support configuring security models, database engines, SQL Alchemy dialects, etc.

Dynamic module loading

The extension architecture leverages Webpack's Module Federation to enable dynamic loading of frontend assets at runtime. This sophisticated mechanism involves several key concepts:

Module Federation allows extensions to be built and deployed independently while sharing dependencies with the host application. Extensions expose their entry points through the federation configuration:

new ModuleFederationPlugin({
  name: 'my_extension',
  filename: 'remoteEntry.[contenthash].js',
  exposes: {
    './index': './src/index.tsx',
  },
  externalsType: 'window',
  externals: {
    '@apache-superset/core': 'superset',
  },
  shared: {
    react: { singleton: true },
    'react-dom': { singleton: true },
    'antd-v5': { singleton: true }
  }
})

externals and externalsType ensure that extensions use the host's implementation of shared packages rather than bundling their own copies.

shared dependencies prevent duplication of common libraries like React and Ant Design avoiding version conflicts and reducing bundle size.

This configuration tells Webpack that when the extension imports from @apache-superset/core, it should resolve to window.superset at runtime, where the host application provides the actual implementation. The following diagram illustrates how this works in practice:

During extension registration, the host application fetches the remote entry file and dynamically loads the extension's modules without requiring a rebuild or restart of Superset.

On the host application side, the @apache-superset/core package will be mapped to the corresponding implementations during bootstrap in the setupExtensionsAPI function.

import * as supersetCore from '@apache-superset/core';
import {
  authentication,
  core,
  commands,
  environment,
  extensions,
  sqlLab,
} from 'src/extensions';

export default function setupExtensionsAPI() {
  window.superset = {
    ...supersetCore,
    authentication,
    core,
    commands,
    environment,
    extensions,
    sqlLab,
  };
}

Deploying an extension

Once an extension has been developed, the deployment process involves packaging and uploading it to the host application.

Packaging is handled by the superset-extensions bundle command, which:

1. Builds frontend assets using Webpack (with Module Federation configuration).
2. Collects backend Python source files and all necessary resources.
3. Generates a manifest.json with build-time metadata, including the contents of extension.json and references to built assets.
4. Packages everything into a .supx file (a zip archive with a specific structure required by Superset).

Uploading is accomplished through Superset's REST API at /api/v1/extensions/import/. The endpoint accepts the .supx file as form data and processes it by:

1. Extracting and validating the extension metadata and manifest.
2. Storing extension assets in the metadata database for dynamic loading.
3. Registering the extension in the metadata database, including its name, version, author, and capabilities.
4. Automatically activating the extension, making it immediately available for use and management via the Superset UI or API.

This API-driven approach enables automated deployment workflows and simplifies extension management for administrators. Extensions can be uploaded through the Swagger UI, programmatically via scripts, or through the management interface:

https://github.com/user-attachments/assets/98b16cdd-8ec5-4812-9d5e-9915badd8f0d

Lifecycle and management

Superset will manage the full lifecycle of extensions, including activation, deactivation, and cleanup. The lifecycle is designed to ensure that extensions can safely register resources and reliably clean them up when no longer needed.

Frontend lifecycle

• When an extension is activated, its activate(context) function is called. The extension should register all event listeners, commands, views, and other contributions using the provided context, and add any disposables to context.disposables.
• When the extension is deactivated (e.g., disabled or uninstalled), Superset automatically calls dispose() on all items in context.disposables, ensuring that event listeners, commands, and UI contributions are removed and memory leaks are avoided.

Backend lifecycle

• Backend entry points, which can add REST API endpoints or execute arbitrary backend code, are eagerly evaluated during startup. Other backend code is lazily loaded when needed, ensuring minimal startup latency.
• In the future, we plan to leverage mechanisms like Redis pub/sub (already an optional dependency of Superset) to dynamically manage the lifecycle of extensions’ backend functionality. This will ensure that all running instances of the Superset backend have the same available extensions without requiring a restart. This will be addressed in a follow-up SIP.

The proof-of-concept (POC) code for this SIP already implements a management module where administrators can upload, delete, enable/disable, and inspect the manifest for installed extensions via the Superset UI, making extension operations straightforward. These operations are currently supported dynamically for frontend extensions. For backend extensions, dynamic upload, deletion, and enable/disable are planned for a future iteration; at present, changes to backend extensions (such as uploading, deleting, or enabling/disabling) still require a server restart.

https://github.com/user-attachments/assets/4eb7064b-3290-4e4c-b88b-52d8d1c11245

Development mode

Development mode accelerates extension development by letting developers see changes in Superset quickly, without the need for repeated packaging and uploading. To enable development mode, set the LOCAL_EXTENSIONS configuration in your superset_config.py:

LOCAL_EXTENSIONS = [
    "/path/to/your/extension1",
    "/path/to/your/extension2",
]

This instructs Superset to load and serve extensions directly from disk, so you can iterate quickly. Running superset-extensions dev watches for file changes and rebuilds assets automatically, while the Webpack development server (started separately with npm run dev-server) serves updated files as soon as they're modified. This enables immediate feedback for React components, styles, and other frontend code. Changes to backend files are also detected automatically and immediately synced, ensuring that both frontend and backend updates are reflected in your development environment.

Example output when running in development mode:

superset-extensions dev

⚙️  Building frontend assets…
✅ Frontend rebuilt
✅ Backend files synced
✅ Manifest updated
👀 Watching for changes in: /dataset_references/frontend, /dataset_references/backend

Versioning

All core packages published to NPM and PyPI—including @apache-superset/core, apache-superset-core, and apache-superset-extensions-cli—will follow semantic versioning (semver). This means that any breaking changes to public APIs, interfaces, or extension points will result in a major version bump, while new features and non-breaking changes will be released as minor or patch updates.

During the initial development phase, packages will remain under version 0.x, allowing for rapid iteration and the introduction of breaking changes as the architecture evolves based on real-world feedback. Once the APIs and extension points are considered stable, we will release version 1.0.0 and begin enforcing strict server practices.

To minimize disruption, breaking changes will be carefully planned and communicated in advance. We will provide clear migration guides and changelogs to help extension authors and core contributors adapt to new versions. Where possible, we will deprecate APIs before removing them, giving developers time to transition to newer interfaces.

Security implications and responsibilities

By default, extensions are disabled and must be explicitly enabled by setting the ENABLE_EXTENSIONS feature flag. Built-in extensions are included as part of the Superset codebase and are held to the same security standards and review processes as the rest of the application.

For external extensions, administrators are responsible for evaluating and verifying the security of any extensions they choose to install, just as they would when installing third-party NPM or PyPI packages. At this stage, all extensions run in the same context as the host application, without additional sandboxing. This means that external extensions can impact the security and performance of a Superset environment in the same way as any other installed dependency.

We plan to introduce an optional sandboxed execution model for extensions in the future (as part of an additional SIP). Until then, administrators should exercise caution and follow best practices when selecting and deploying third-party extensions. A directory of known Superset extensions may be maintained in a means similar to this page on the wiki. We also discussed the possibility of introducing a shared registry for vetted extensions but decided to leave it out of the initial scope of the project. We might introduce a registry at a later stage depending on the evolution of extensions created by the community.

Any performance or security vulnerabilities introduced by external extensions should be reported directly to the extension author, not as Superset vulnerabilities. Any security concerns regarding built-in extensions (included in Superset’s monorepo) should be reported to the Superset Security mailing list for triage and resolution by maintainers.

What this means for Superset's built-in features

Transitioning to a well-defined, versioned API model has significant implications for Superset’s built-in features. By exposing stable, public APIs for core functionality, we enable both extensions and internal modules to interact with the application in a consistent and predictable way. This approach brings several key benefits:

• Unified API Surface: All important features of Superset will be accessible through documented, versioned APIs. This not only empowers extension authors but also ensures that built-in features use the same mechanisms, validating the APIs through real-world usage and making it easier to replace or enhance individual features over time.
• Dogfooding and Replaceability: By building Superset’s own features using the same APIs available to extensions, we ensure that these APIs are robust, flexible, and well-tested. This also means that any built-in feature can potentially be replaced or extended by a third-party extension, increasing modularity and adaptability.
• Versioned and Stable Contracts: Public APIs will be versioned and follow semantic versioning, providing stability for both internal and external consumers. This stability is critical for long-term maintainability, but it also means that extra care must be taken to avoid breaking changes and to provide clear migration paths when changes are necessary.
• Improved Inter-Module Communication: With clearly defined APIs and a command-based architecture, modules and extensions can communicate through explicit interfaces rather than relying on direct Redux store access or tightly coupled state management. This decouples modules, reduces the risk of unintended side effects, and makes the codebase easier to reason about and maintain.
• Facilitated Refactoring and Evolution: As the application evolves, having a stable API layer allows for internal refactoring and optimization without breaking consumers. This makes it easier to modernize or optimize internal implementations while preserving compatibility.
• Clearer Documentation and Onboarding: A public, versioned API surface makes it easier to document and onboard new contributors, both for core development and for extension authors.

Overall, this shift represents a move toward a more modular, maintainable, and extensible architecture, where both built-in features and extensions are first-class citizens, and where the boundaries between core and community-driven innovation are minimized.

Real extension example

The proof-of-concept implementation, available at https://github.com/apache/superset/pull/31934, includes a real extension example called Dataset references, which demonstrates the architecture and APIs in action. This extension adds a custom SQL Lab panel that, for any SQL query, displays:

• Tables Referenced in the SQL Query: Helping users discover the origin of the data, especially for long or complex queries.
• Owners of Each Referenced Table: Allowing users to quickly identify who to contact for access permissions or troubleshooting.
• Last Available Partition: Indicating the most recent partition for each table, which is important for understanding data completeness and freshness.
• Estimated Row Count for Each Table: Helping users grasp the scale of the data and the implications of joins in their queries.

https://github.com/user-attachments/assets/d94ad586-d853-4381-87d1-47573df2b613

New or Changed Public Interfaces

The new extension architecture introduces several important public interfaces that define how extensions are described, managed, and integrated with Superset. The following table summarizes the key public interfaces introduced or updated as part of this proposal:

Name	Usage	Used By
extension.json	Production	Extension authors, Host
/api/v1/extensions	Production	Admins, Host, Extensions
ENABLE_EXTENSIONS feature flag	Production	Admins, Host
CLI commands	Development	Extension authors

New dependencies

The extension architecture also introduces several new dependencies that are essential for supporting extension development, testing, packaging, and documentation. The following table summarizes the key dependencies added as part of this proposal:

Name	Usage	Used By
@apache-superset/core	Production	Extensions, Host
apache-superset-core	Production	Extensions, Host
apache-superset-extensions-cli	Development	Extension authors
Developer portal	Development	All developers

Describe any npm/PyPI packages that are required. Are they actively maintained? What are their licenses?

Migration Plan and Compatibility

The migration to the new extension architecture will be incremental and community-driven, with a focus on delivering value at each stage. Rather than attempting a disruptive, all-at-once rewrite, we will gradually introduce new APIs, packages, and extension points, allowing both core maintainers and extension authors to adopt the new model at their own pace. This approach ensures ongoing stability for existing users while enabling rapid innovation and feedback from the community.

The initial proof of concept (POC) implements only a subset of the envisioned APIs and extension points. As we move forward, we invite the community to participate in identifying gaps, proposing improvements, and helping to complete the API surface. All new @apache-superset packages and host API implementations will be developed iteratively and kept under version 0.x until they are considered stable. This allows for rapid iteration and breaking changes as we refine the architecture based on real-world usage and feedback.

A key principle of this migration is to deliver value continuously, not just at the end. As new APIs, extension points, and shared components are introduced, we will encourage the creation of real extensions—even if they initially rely on incomplete or evolving APIs, or have access to only a subset of UI elements and features. Building and shipping extensions early in the process will help validate the architecture, surface practical requirements, and provide immediate benefits to users and developers. This iterative approach ensures that the community can start taking advantage of the new extension model right away, while also providing valuable feedback to guide the ongoing evolution of the platform.

Key sub-projects

Sub-Project	Description
@apache-superset/core (npm)	Migrate vetted UI components and utilities from existing packages into this new package. Deprecate legacy @superset-ui packages and only maintain @apache-superset/core. Evolve and document APIs for interacting with the host application, and refactor built-in features to consume these APIs, validating them through real-world usage.
Apache-superset-core (PyPI)	Evolve backend APIs to support extension integration, and refactor built-in backend features to use these new APIs. This will help validate the backend extension model and ensure a consistent developer experience across frontend and backend.
Apache-superset-extensions-cli (PyPI)	Enhance the CLI to streamline extension development, including improved scaffolding, packaging, and publishing workflows. This will make it easier for developers to create, test, and distribute extensions.
SQL Lab Host refactor	Incrementally refactor SQL Lab to align with the new extension vision, replacing existing features with built-in extensions where appropriate. Some candidates include refactoring the database visualizer and the SQL editor to be built-in extensions that can be replaced. This will serve as a reference implementation and help uncover practical challenges in the migration process.
Developer Portal	As new features and APIs are introduced, update the developer portal with comprehensive documentation, guides, and best practices to support extension authors and core contributors.

This migration plan is designed to be collaborative and adaptable, with multiple teams able to contribute to different sub-projects in parallel. By delivering improvements incrementally and validating each step through real-world adoption, we aim to minimize risk, maximize community engagement, and ensure a smooth transition to a more modular, extensible, and maintainable Superset architecture.

Downsides and risks

While the proposed extension architecture offers many benefits, it also introduces some risks and challenges that need to be carefully managed:

• Migration Overhead: Refactoring core functionalities into extensions can add overhead, requiring extra abstraction and coordination. This may temporarily slow development but is necessary for long-term modularity and maintainability.
• Performance Impact: The introduction of a dynamic loading mechanism using Webpack Module Federation may introduce some performance overhead, especially during initial load times. However, this is expected to be mitigated by caching and optimization strategies.
• Risk of Low Community Adoption: The success of this architecture depends on community adoption and participation. If extension authors do not embrace the new model, we may not achieve the desired modularity and extensibility. We will need to provide clear documentation, examples, and support to encourage adoption and help developers transition to the new architecture.
• Initial Instability: While we will follow semantic versioning for public APIs, the initial phase of development will involve rapid iteration and breaking changes. This may lead to instability in the early stages, requiring careful management of versioning and migration paths to ensure a smooth transition for extension authors and users.

Rejected alternatives

Several alternatives were considered and ultimately rejected:

• Allowing arbitrary code overrides or patching of core files was deemed too risky and difficult to maintain.
• Supporting unrestricted extension points was rejected in favor of a model that only allows well-defined, versioned contribution types. This approach ensures that extensions can be developed and maintained in a predictable and secure manner, without compromising the integrity of the core application.
• Supporting backend extensions only through PyPI/pip, rather than as part of the extension bundle, was considered. This approach was rejected because most advanced use cases require both frontend and backend to be delivered together, so we now support unified extension bundles containing both.
• We considered having an engine configuration similar to VSCode about version compatibility but decided to use the core packages version for this end. Once the core APIs stabilize, we expect breaking changes in the API to be less frequent than in the main Superset application. This means that extension authors will only need to review breaking changes in the core APIs when a new major version is introduced, rather than every time a major Superset version is released.
• Continuing to extend Superset’s current system of registries/extensions, since they only add complexity and maintenance to the monorepo.

Future work

The extension architecture lays the groundwork for a more modular and extensible Superset, but there are several areas that will require further development and community collaboration as the ecosystem matures. The following initiatives are planned for future iterations and follow-up SIPs:

• We plan to write a follow-up SIP to introduce a new Editor contribution type to replace or enhance the built-in editors within Superset, such as those used for SQL, HTML, JSON, and other content types. This enables extension authors to introduce alternative editing experiences—such as integrating advanced code editors like Monaco or CodeMirror, adding custom syntax highlighting, or providing domain-specific editing tools—tailored to the needs of their users.
• We plan to write a follow-up SIP to introduce a new Chat contribution type to integrate large language models (LLMs) into SQL Lab, enabling advanced natural language features such as query assistance, data exploration, or workflow automation.
• Expand extension support to additional Superset modules, including charts, dashboards, database connectors, etc.
• Define how extensions will store and manage data, both locally and remotely, ensuring that extensions can maintain state, configuration, and user data in a secure, scalable, and reliable manner. We plan to draw inspiration from established systems like the Data Storage functions provided by VS Code, adapting best practices to fit Superset’s architecture and security requirements.
• Enable theming to be contributed through extensions by introducing a new contribution type, allowing developers to create and share custom themes that can be easily integrated into Superset and applied across both core features and extensions.
• Develop and implement JS/DOM sandboxing for frontend extensions, enabling support for non-vetted extensions contributed by anyone in the community. This will open the door to a broader ecosystem of extensions and lay the foundation for a potential extension marketplace, where users can safely discover, share, and install community-contributed features.
• Introduce new extension points and contribution types based on community feedback and evolving requirements.
• Add support for inter-extension dependencies, allowing extensions to declare, discover, and resolve dependencies on other extensions. This will enable more complex and composable extension ecosystems, where extensions can build on top of each other and leverage shared functionality in a reliable and predictable way.
• Continue to refine and optimize the extension lifecycle, activation, and deactivation processes.
• Develop a fully typed REST API client for Superset, providing a consistent and type-safe interface for interacting with the host application’s APIs. This client will simplify extension development, improve type safety, and enhance the overall developer experience for both core contributors and extension authors.
• Develop a robust testing framework that enables extension authors to write and run unit tests for their extensions independently of the host application, ensuring compatibility and reliability.
• We plan to introduce backend contribution types to the extension manifest at a later stage to allow extension functionalities to be easily discovered.

These efforts will help ensure that Superset remains flexible, secure, and easy to extend as new use cases and technologies emerge.

Special thanks

A heartfelt thank you to the following folks for your invaluable feedback and support:

• Diego Pucci @geido
• Jacob Amrany @jamra
• Jesse Yang @ktmud
• Kasia Zajac @kasiazjc
• Krist Wongsuphasawat @kristw
• Maxime Beauchemin @mistercrunch

✍🏼 Michael, Ville and Evan.