More reorganization of content, deleting extra content.

docs/source/extension/extension_dev.rst

@@ -93,6 +93,19 @@ A typical plugin is specified by the following metadata. The ``id`` and ``activa
 - ``provides`` is the token associated with the service your plugin is providing to the system. A token can only be registered with the system once. If your plugin does not provide a service to the system, omit this field and do not return a value from your ``activate`` function.
 - ``activate`` is the function called when your plugin is activated. The arguments are, in order, the Application object, the services corresponding to the ``requires`` tokens, then the services corresponding to the ``optional`` tokens (or ``null`` if that particular ``optional`` token is not registered in the system). The return value of the ``activate`` function (or resolved return value if a promise is returned) will be stored in the system as the service associated with the ``provides`` token.
 
+Application Object
+""""""""""""""""""
+
+A Jupyter front-end application object is given to each plugin in its
+``activate()`` function. The application object has:
+
+-  ``commands`` - an extensible registry used to add and execute commands in the application.
+-  ``commandLinker`` - used to connect DOM nodes with the command registry so that clicking on them executes a command.
+-  ``docRegistry`` - an extensible registry containing the document types that the application is able to read and render.
+-  ``restored`` - a promise that is resolved when the application has finished loading.
+-  ``serviceManager`` - low-level manager for talking to the Jupyter REST API.
+-  ``shell`` - a generic Jupyter front-end shell instance, which holds the user interface for the application.
+
 
 
 package.json metadata
@@ -102,18 +115,135 @@ package.json metadata
 Custom webpack config
 """""""""""""""""""""
 
+.. warning::
+   This feature is *experimental*, as it makes it possible to override the base config used by the
+   JupyterLab Federated Extension System.
+
+   It also exposes the internals of the federated extension build system (namely ``webpack``) to extension authors, which was until now
+   kept as an implementation detail.
+
+The JupyterLab Federated Extension System uses ``webpack`` to build federated extensions, relying on the
+`Module Federation System <https://webpack.js.org/concepts/module-federation/>`_ added in webpack 5.
+
+To specify a custom webpack config to the federated extension build system, extension authors can add the ``webpackConfig`` subkey to the
+``package.json`` of their extension::
+
+    "jupyterlab": {
+      "webpackConfig": "webpack.config.js"
+    }
+
+The webpack config file can be placed in a different location with a custom name::
+
+    "jupyterlab": {
+      "webpackConfig": "./config/test-config.js"
+    }
+
+Here is an example of a custom config that enables the async WebAssembly and top-level ``await`` experiments:
+
+.. code-block:: javascript
+
+    module.exports = {
+      experiments: {
+          topLevelAwait: true,
+          asyncWebAssembly: true,
+      }
+    };
+
+This custom config will be merged with the `default config <https://github.com/jupyterlab/jupyterlab/blob/master/builder/src/webpack.config.base.ts>`_
+when building the federated extension with ``jlpm run build``.
+
+
 Disabling other extensions
 """"""""""""""""""""""""""
 
-Federation data
+Prebuilt data
 """""""""""""""
 
 Sharing configuration
 """""""""""""""""""""
 
+
+.. _ext-author-companion-packages:
+
 Companion packages
 """"""""""""""""""
 
+If your extensions depends on the presence of one or more packages in the
+kernel, or on a notebook server extension, you can add metadata to indicate
+this to the extension manager by adding metadata to your package.json file.
+The full options available are::
+
+    "jupyterlab": {
+      "discovery": {
+        "kernel": [
+          {
+            "kernel_spec": {
+              "language": "<regexp for matching kernel language>",
+              "display_name": "<regexp for matching kernel display name>"   // optional
+            },
+            "base": {
+              "name": "<the name of the kernel package>"
+            },
+            "overrides": {   // optional
+              "<manager name, e.g. 'pip'>": {
+                "name": "<name of kernel package on pip, if it differs from base name>"
+              }
+            },
+            "managers": [   // list of package managers that have your kernel package
+                "pip",
+                "conda"
+            ]
+          }
+        ],
+        "server": {
+          "base": {
+            "name": "<the name of the server extension package>"
+          },
+          "overrides": {   // optional
+            "<manager name, e.g. 'pip'>": {
+              "name": "<name of server extension package on pip, if it differs from base name>"
+            }
+          },
+          "managers": [   // list of package managers that have your server extension package
+              "pip",
+              "conda"
+          ]
+        }
+      }
+    }
+
+
+A typical setup for e.g. a jupyter-widget based package will then be::
+
+    "keywords": [
+        "jupyterlab-extension",
+        "jupyter",
+        "widgets",
+        "jupyterlab"
+    ],
+    "jupyterlab": {
+      "extension": true,
+      "discovery": {
+        "kernel": [
+          {
+            "kernel_spec": {
+              "language": "^python",
+            },
+            "base": {
+              "name": "myipywidgetspackage"
+            },
+            "managers": [
+                "pip",
+                "conda"
+            ]
+          }
+        ]
+      }
+    }
+
+
+Currently supported package managers are ``pip`` and ``conda``.
+
 
 Packaging extensions
 ^^^^^^^^^^^^^^^^^^^^
@@ -121,7 +251,6 @@ Packaging extensions
 Prebuilt Extensions
 ^^^^^^^^^^^^^^^^^^^
 
-
 ``install.json``
 
 How prebuilt extensions work
@@ -133,22 +262,78 @@ Steps for building
 Directory walkthrough
 """""""""""""""""""""
 
+Plugins
+-------
 
+.. _rendermime:
 
-Migrating extensions
-^^^^^^^^^^^^^^^^^^^^
+Mime Renderer Plugins
+^^^^^^^^^^^^^^^^^^^^^
 
+Mime Renderer plugins are a convenience for creating an plugin
+that can render mime data and potentially render files of a given type.
+We provide an extension cookiecutter for mime renderer plugins in TypeScript
+`here <https://github.com/jupyterlab/mimerender-cookiecutter-ts>`__.
 
+Mime renderer plugins are more declarative than standard plugins.
+The extension is treated the same from the command line perspective
+(``jupyter labextension install`` ), but it does not directly create
+JupyterLab plugins. Instead it exports an interface given in the
+`rendermime-interfaces <https://jupyterlab.github.io/jupyterlab/interfaces/_rendermime_interfaces_src_index_.irendermime.iextension.html>`__
+package.
 
-Plugins
--------
+The JupyterLab repo has an example mime renderer extension for
+`pdf <https://github.com/jupyterlab/jupyterlab/tree/master/packages/pdf-extension>`__
+files. It provides a mime renderer for pdf data and registers itself as
+a document renderer for pdf file types.
+
+The JupyterLab organization also has a mime renderer extension tutorial
+which adds mp4 video rendering to the application
+`here <https://github.com/jupyterlab/jupyterlab-mp4>`__.
+
+The ``rendermime-interfaces`` package is intended to be the only
+JupyterLab package needed to create a mime renderer extension (using the
+interfaces in TypeScript or as a form of documentation if using plain
+JavaScript).
+
+The only other difference from a standard extension is that has a
+``jupyterlab`` key in its ``package.json`` with ``"mimeExtension"``
+metadata. The value can be ``true`` to use the main module of the
+package, or a string path to a specific module (e.g. ``"lib/foo"``).
+
+The mime renderer can update its data by calling ``.setData()`` on the
+model it is given to render. This can be used for example to add a
+``png`` representation of a dynamic figure, which will be picked up by a
+notebook model and added to the notebook document. When using
+``IDocumentWidgetFactoryOptions``, you can update the document model by
+calling ``.setData()`` with updated data for the rendered MIME type. The
+document can then be saved by the user in the usual manner.
 
-Mime renderers
-^^^^^^^^^^^^^^
 
 Theme plugins
 ^^^^^^^^^^^^^
 
+A theme is a JupyterLab plugin that uses a ``ThemeManager`` and can
+be loaded and unloaded dynamically. The package must include all static
+assets that are referenced by ``url()`` in its CSS files. Local URLs can
+be used to reference files relative to the location of the referring sibling CSS files. For example ``url('images/foo.png')`` or
+``url('../foo/bar.css')``\ can be used to refer local files in the
+theme. Absolute URLs (starting with a ``/``) or external URLs (e.g.
+``https:``) can be used to refer to external assets. The path to the
+theme asset entry point is specified ``package.json`` under the ``"jupyterlab"``
+key as ``"themePath"``. See the `JupyterLab Light
+Theme <https://github.com/jupyterlab/jupyterlab/tree/master/packages/theme-light-extension>`__
+for an example. Ensure that the theme files are included in the
+``"files"`` metadata in ``package.json``.  Note that if you want to use SCSS, SASS, or LESS files,
+you must compile them to CSS and point JupyterLab to the CSS files.
+
+The theme extension is installed in the same way as a regular extension (see
+`extension authoring <#extension-authoring>`__).
+
+It is also possible to create a new theme using the
+`TypeScript theme cookiecutter <https://github.com/jupyterlab/theme-cookiecutter>`__.
+
+
 .. _services:
 
 Plugins Interacting with Each Other
@@ -185,9 +370,6 @@ Deduplication in JupyterLab happens in two ways. For source extensions, JupyterL
 To ensure that a consumer gets the same token instance as the provider provided to the sytem, any required tokens that are imported by a consumer extension should list the exporting extension as a singleton package in their ``jupyterlab.sharedPackages`` config. Required token packages should be listed as ``bundled: false`` - this will generate a JavaScript error if the package (and thus the token) is not present in the system at runtime. Optional token packages should be listed as singletons that are bundled (otherwise, if they are not present in the system, it will cause a js error when you try to import them).
 
 
-
-
-
 Advanced Plugins
 ----------------
 
@@ -243,171 +425,6 @@ added in the application settings directory (by default this is the
     }
   }
 
-State Database
-^^^^^^^^^^^^^^
-
-The state database can be accessed by importing ``IStateDB`` from
-``@jupyterlab/statedb`` and adding it to the list of ``requires`` for
-a plugin:
-
-.. code:: typescript
-
-    const id = 'foo-extension:IFoo';
-
-    const IFoo = new Token<IFoo>(id);
-
-    interface IFoo {}
-
-    class Foo implements IFoo {}
-
-    const plugin: JupyterFrontEndPlugin<IFoo> = {
-      id,
-      autoStart: true,
-      requires: [IStateDB],
-      provides: IFoo,
-      activate: (app: JupyterFrontEnd, state: IStateDB): IFoo => {
-        const foo = new Foo();
-        const key = `${id}:some-attribute`;
-
-        // Load the saved plugin state and apply it once the app
-        // has finished restoring its former layout.
-        Promise.all([state.fetch(key), app.restored])
-          .then(([saved]) => { /* Update `foo` with `saved`. */ });
-
-        // Fulfill the plugin contract by returning an `IFoo`.
-        return foo;
-      }
-    };
-
-
-
-Custom webpack config for prebuilt extensions
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-.. warning::
-   This feature is *experimental*, as it makes it possible to override the base config used by the
-   JupyterLab Federated Extension System.
-
-   It also exposes the internals of the federated extension build system (namely ``webpack``) to extension authors, which was until now
-   kept as an implementation detail.
-
-The JupyterLab Federated Extension System uses ``webpack`` to build federated extensions, relying on the
-`Module Federation System <https://webpack.js.org/concepts/module-federation/>`_ added in webpack 5.
-
-To specify a custom webpack config to the federated extension build system, extension authors can add the ``webpackConfig`` subkey to the
-``package.json`` of their extension::
-
-    "jupyterlab": {
-      "webpackConfig": "webpack.config.js"
-    }
-
-The webpack config file can be placed in a different location with a custom name::
-
-    "jupyterlab": {
-      "webpackConfig": "./config/test-config.js"
-    }
-
-Here is an example of a custom config that enables the async WebAssembly and top-level ``await`` experiments:
-
-.. code-block:: javascript
-
-    module.exports = {
-      experiments: {
-          topLevelAwait: true,
-          asyncWebAssembly: true,
-      }
-    };
-
-This custom config will be merged with the `default config <https://github.com/jupyterlab/jupyterlab/blob/master/builder/src/webpack.config.base.ts>`_
-when building the federated extension with ``jlpm run build``.
-
-Companion Packages
-^^^^^^^^^^^^^^^^^^
-
-If your extensions depends on the presence of one or more packages in the
-kernel, or on a notebook server extension, you can add metadata to indicate
-this to the extension manager by adding metadata to your package.json file.
-The full options available are::
-
-    "jupyterlab": {
-      "discovery": {
-        "kernel": [
-          {
-            "kernel_spec": {
-              "language": "<regexp for matching kernel language>",
-              "display_name": "<regexp for matching kernel display name>"   // optional
-            },
-            "base": {
-              "name": "<the name of the kernel package>"
-            },
-            "overrides": {   // optional
-              "<manager name, e.g. 'pip'>": {
-                "name": "<name of kernel package on pip, if it differs from base name>"
-              }
-            },
-            "managers": [   // list of package managers that have your kernel package
-                "pip",
-                "conda"
-            ]
-          }
-        ],
-        "server": {
-          "base": {
-            "name": "<the name of the server extension package>"
-          },
-          "overrides": {   // optional
-            "<manager name, e.g. 'pip'>": {
-              "name": "<name of server extension package on pip, if it differs from base name>"
-            }
-          },
-          "managers": [   // list of package managers that have your server extension package
-              "pip",
-              "conda"
-          ]
-        }
-      }
-    }
-
-
-A typical setup for e.g. a jupyter-widget based package will then be::
-
-    "keywords": [
-        "jupyterlab-extension",
-        "jupyter",
-        "widgets",
-        "jupyterlab"
-    ],
-    "jupyterlab": {
-      "extension": true,
-      "discovery": {
-        "kernel": [
-          {
-            "kernel_spec": {
-              "language": "^python",
-            },
-            "base": {
-              "name": "myipywidgetspackage"
-            },
-            "managers": [
-                "pip",
-                "conda"
-            ]
-          }
-        ]
-      }
-    }
-
-
-Currently supported package managers are:
-
-- ``pip``
-- ``conda``
-
-
-
-
-.. _ext-author-companion-packages:
-
 
 Development workflow
 --------------------
@@ -420,13 +437,6 @@ to any GitHub extension repository.
 Older Docs
 ==========
 
-Writing extensions
-------------------
-
-Starting in JupyterLab 3.0, extensions are distributed as ``pip`` or
-``conda`` packages that contain federated JavaScript bundles.  You can write extensions in JavaScript or any language that compiles to JavaScript. We recommend writing extensions in `TypeScript <https://www.typescriptlang.org/>`_, which is used for the JupyterLab core extensions and many popular community extensions.  You use our build tool to generate the bundles that are shipped with the package, typically through a cookiecutter.
-
-
 Implementation
 --------------
 - We provide a ``jupyter labextension build`` script that is used to build federated bundles
@@ -446,98 +456,6 @@ Implementation
 - We provide a script to update extensions: ``python -m jupyterlab.upgrade_extension``
 - We update the ``extension-manager`` to target metadata on ``pypi``/``conda`` and consume those packages.
 
-Tools
------
-- ``jupyter labexension build`` python command line tool
-- ``jupyter labextension develop`` python command line tool
-- ``python -m jupyterlab.upgrade_extension`` python command line tool
-- ``cookiecutter`` for extension authors
-
-Workflow for extension authors
-------------------------------
-- Use the ``cookiecutter`` to create the extension
-- Run ``jupyter labextension develop`` to build and symlink the files
-- Run ``jlpm run watch`` to start watching
-- Run ``jupyter lab``
-- Make changes to source
-- Refresh the application page
-- When finished, publish the package to ``pypi``/``conda``
-
-
-Plugins
--------
-
-A plugin adds a core functionality to the application:
-
--  A plugin can require other plugins for operation.
--  A plugin is activated when it is needed by other plugins, or when
-   explicitly activated.
--  Plugins require and provide ``Token`` objects, which are used to
-   provide a typed value to the plugin's ``activate()`` method.
--  The module providing plugin(s) must meet the
-   `JupyterLab.IPluginModule <https://jupyterlab.github.io/jupyterlab/interfaces/_application_src_index_.jupyterlab.ipluginmodule.html>`__
-   interface, by exporting a plugin object or array of plugin objects as
-   the default export.
-
-The default plugins in the JupyterLab application include:
-
--  `Terminal <https://github.com/jupyterlab/jupyterlab/blob/master/packages/terminal-extension/src/index.ts>`__
-   - Adds the ability to create command prompt terminals.
--  `Shortcuts <https://github.com/jupyterlab/jupyterlab/blob/master/packages/shortcuts-extension/src/index.ts>`__
-   - Sets the default set of shortcuts for the application.
--  `Images <https://github.com/jupyterlab/jupyterlab/blob/master/packages/imageviewer-extension/src/index.ts>`__
-   - Adds a widget factory for displaying image files.
--  `Help <https://github.com/jupyterlab/jupyterlab/blob/master/packages/help-extension/src/index.tsx>`__
-   - Adds a side bar widget for displaying external documentation.
--  `File
-   Browser <https://github.com/jupyterlab/jupyterlab/blob/master/packages/filebrowser-extension/src/index.ts>`__
-   - Creates the file browser and the document manager and the file
-   browser to the side bar.
--  `Editor <https://github.com/jupyterlab/jupyterlab/blob/master/packages/fileeditor-extension/src/index.ts>`__
-   - Add a widget factory for displaying editable source files.
--  `Console <https://github.com/jupyterlab/jupyterlab/blob/master/packages/console-extension/src/index.ts>`__
-   - Adds the ability to launch Jupyter Console instances for
-   interactive kernel console sessions.
-
-Here is a dependency graph for the core JupyterLab components: |dependencies|
-
-.. danger::
-
-    Installing an extension allows for arbitrary code execution on the
-    server, kernel, and in the client's browser. You should therefore
-    take steps to protect against malicious changes to your extension's
-    code. This includes ensuring strong authentication for your PyPI
-    account.
-
-
-Application Object
-------------------
-
-A Jupyter front-end application object is given to each plugin in its
-``activate()`` function. The application object has:
-
--  ``commands`` - an extensible registry used to add and execute commands in the application.
--  ``commandLinker`` - used to connect DOM nodes with the command registry so that clicking on them executes a command.
--  ``docRegistry`` - an extensible registry containing the document types that the application is able to read and render.
--  ``restored`` - a promise that is resolved when the application has finished loading.
--  ``serviceManager`` - low-level manager for talking to the Jupyter REST API.
--  ``shell`` - a generic Jupyter front-end shell instance, which holds the user interface for the application.
-
-Lumino
-------
-
-The Lumino library is used as the underlying architecture of
-JupyterLab and provides many of the low level primitives and widget
-structure used in the application. Lumino provides a rich set of
-widgets for developing desktop-like applications in the browser, as well
-as patterns and objects for writing clean, well-abstracted code. The
-widgets in the application are primarily **Lumino widgets**, and
-Lumino concepts, like message passing and signals, are used
-throughout. **Lumino messages** are a *many-to-one* interaction that
-enables information like resize events to flow through the widget
-hierarchy in the application. **Lumino signals** are a *one-to-many*
-interaction that enable listeners to react to changes in an observed
-object.
 
 Extension Authoring
 -------------------
@@ -730,72 +648,7 @@ Finally, you will need to configure babel with a ``babel.config.js`` file contai
      ]
    };
 
-.. _rendermime:
-
-Mime Renderer Extensions
-------------------------
-
-Mime Renderer extensions are a convenience for creating an extension
-that can render mime data and potentially render files of a given type.
-We provide a cookiecutter for mime renderer extensions in TypeScript
-`here <https://github.com/jupyterlab/mimerender-cookiecutter-ts>`__.
 
-Mime renderer extensions are more declarative than standard extensions.
-The extension is treated the same from the command line perspective
-(``jupyter labextension install`` ), but it does not directly create
-JupyterLab plugins. Instead it exports an interface given in the
-`rendermime-interfaces <https://jupyterlab.github.io/jupyterlab/interfaces/_rendermime_interfaces_src_index_.irendermime.iextension.html>`__
-package.
-
-The JupyterLab repo has an example mime renderer extension for
-`pdf <https://github.com/jupyterlab/jupyterlab/tree/master/packages/pdf-extension>`__
-files. It provides a mime renderer for pdf data and registers itself as
-a document renderer for pdf file types.
-
-The JupyterLab organization also has a mime renderer extension tutorial
-which adds mp4 video rendering to the application
-`here <https://github.com/jupyterlab/jupyterlab-mp4>`__.
-
-The ``rendermime-interfaces`` package is intended to be the only
-JupyterLab package needed to create a mime renderer extension (using the
-interfaces in TypeScript or as a form of documentation if using plain
-JavaScript).
-
-The only other difference from a standard extension is that has a
-``jupyterlab`` key in its ``package.json`` with ``"mimeExtension"``
-metadata. The value can be ``true`` to use the main module of the
-package, or a string path to a specific module (e.g. ``"lib/foo"``).
-
-The mime renderer can update its data by calling ``.setData()`` on the
-model it is given to render. This can be used for example to add a
-``png`` representation of a dynamic figure, which will be picked up by a
-notebook model and added to the notebook document. When using
-``IDocumentWidgetFactoryOptions``, you can update the document model by
-calling ``.setData()`` with updated data for the rendered MIME type. The
-document can then be saved by the user in the usual manner.
-
-Themes
-------
-
-A theme is a JupyterLab extension that uses a ``ThemeManager`` and can
-be loaded and unloaded dynamically. The package must include all static
-assets that are referenced by ``url()`` in its CSS files. Local URLs can
-be used to reference files relative to the location of the referring sibling CSS files. For example ``url('images/foo.png')`` or
-``url('../foo/bar.css')``\ can be used to refer local files in the
-theme. Absolute URLs (starting with a ``/``) or external URLs (e.g.
-``https:``) can be used to refer to external assets. The path to the
-theme asset entry point is specified ``package.json`` under the ``"jupyterlab"``
-key as ``"themePath"``. See the `JupyterLab Light
-Theme <https://github.com/jupyterlab/jupyterlab/tree/master/packages/theme-light-extension>`__
-for an example. Ensure that the theme files are included in the
-``"files"`` metadata in ``package.json``.  Note that if you want to use SCSS, SASS, or LESS files,
-you must compile them to CSS and point JupyterLab to the CSS files.
-
-The theme extension is installed in the same way as a regular extension (see
-`extension authoring <#extension-authoring>`__).
-
-It is also possible to create a new theme using the
-`TypeScript theme cookiecutter <https://github.com/jupyterlab/theme-cookiecutter>`__.
 
 
 

docs/source/extension/extension_points.rst

@@ -607,3 +607,40 @@ Widget tracker tokens are provided for many activities in JupyterLab, including
 notebooks, consoles, text files, mime documents, and terminals.
 If you are adding your own activities to JupyterLab, you might consider providing
 a ``WidgetTracker`` token of your own, so that other extensions can make use of it.
+
+
+State Database
+--------------
+
+The state database can be accessed by importing ``IStateDB`` from
+``@jupyterlab/statedb`` and adding it to the list of ``requires`` for
+a plugin:
+
+.. code:: typescript
+
+    const id = 'foo-extension:IFoo';
+
+    const IFoo = new Token<IFoo>(id);
+
+    interface IFoo {}
+
+    class Foo implements IFoo {}
+
+    const plugin: JupyterFrontEndPlugin<IFoo> = {
+      id,
+      autoStart: true,
+      requires: [IStateDB],
+      provides: IFoo,
+      activate: (app: JupyterFrontEnd, state: IStateDB): IFoo => {
+        const foo = new Foo();
+        const key = `${id}:some-attribute`;
+
+        // Load the saved plugin state and apply it once the app
+        // has finished restoring its former layout.
+        Promise.all([state.fetch(key), app.restored])
+          .then(([saved]) => { /* Update `foo` with `saved`. */ });
+
+        // Fulfill the plugin contract by returning an `IFoo`.
+        return foo;
+      }
+    };