Move “Advanced Usage” docs to their own file/section.

docs/source/index.rst

@@ -46,6 +46,7 @@ JupyterLab is the next-generation web-based user interface for Project Jupyter.
    user/jupyterhub
    user/export
    user/language
+   user/directories
 
 .. toctree::
    :maxdepth: 1

docs/source/user/directories.rst

@@ -0,0 +1,287 @@
+
+Advanced Usage
+==============
+
+JupyterLab manages several different locations for its data. These locations are shown by running ``jupyter lab path``:
+
+- **Application Directory**: where JupyterLab stores the main build of JupyterLab with associated data, including extensions built into JupyterLab.
+- **User Settings Directory**: where JupyterLab stores user-level settings for JupyterLab extensions
+- **Workspaces Directory**: where JupyterLab stores workspaces
+
+
+Additionally, JupyterLab can load dynamic federated extensions, i.e., extensions that bundle their dependencies, from the ``labextensions`` subdirectories of the Jupyter ``data`` directories in the Jupyter path hierarchy. See the locations of these ``data`` paths by running ``jupyter --path``.
+
+JupyterLab Build Process
+------------------------
+
+To rebuild the app directory, run ``jupyter lab build``. By default, the
+``jupyter labextension install`` command builds the application, so you
+typically do not need to call ``build`` directly.
+
+Building consists of:
+
+-  Populating the ``staging/`` directory using template files
+-  Handling any locally installed packages
+-  Ensuring all installed assets are available
+-  Bundling the assets
+-  Copying the bundled assets to the ``static`` directory
+
+Note that building will always use the latest JavaScript packages that meet
+the dependency requirements of JupyterLab itself and any installed extensions.
+If you wish to run JupyterLab with the set of pinned requirements that was
+shipped with the Python package, you can launch as
+``jupyter lab --core-mode``.
+
+The build process uses a specific ``yarn`` version with a default working
+combination of npm packages stored in a ``yarn.lock`` file shipped with
+JupyterLab. Those package source urls point to the default yarn registry.
+However, if you defined your own yarn registry in the yarn configuration, the
+default yarn registry will be replaced by your custom registry. If you then
+switch back to the default yarn registry, you will need to clean your
+``staging`` folder before building:
+
+.. code:: bash
+
+    jupyter lab clean
+    jupyter lab build
+
+
+Disabling Rebuild Checks
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+JupyterLab automatically checks to see if it needs to rebuild on startup. In
+some cases, such as automated testing, you may wish to disable the startup
+rebuild checks altogether. This can be achieved through setting ``buildCheck``
+and ``buildAvailable`` in ``jupyter_notebook_config.json`` (or ``.py``
+equivalent) in any of the ``config`` locations returned by ``jupyter
+--paths``.
+
+
+.. code:: json
+
+    {
+      "LabApp": {
+        "tornado_settings": {
+          "page_config_data": {
+            "buildCheck": false,
+            "buildAvailable": false,
+          }
+        }
+      }
+    }
+
+
+JupyterLab Application Directory
+--------------------------------
+
+The application directory contains the main JupyterLab application files,
+including built assets, files for rebuilding JupyterLab, installed extensions
+and linked packages, etc.
+
+By default, the application directory is at
+``<sys-prefix>/share/jupyter/lab``, where ``<sys-prefix>`` is the
+site-specific directory prefix of the current Python environment. You can
+query the current application path by running ``jupyter lab path``. The
+application directory can be overridden using the ``--app-dir`` command-line
+option in any of the JupyterLab commands, or by setting the ``JUPYTERLAB_DIR``
+environment variable.
+
+We recommend users not install JupyterLab in a system location on Unix-like
+systems, because then the application directory will be read-only. Instead,
+use a conda environment or ``pip install --user jupyterlab`` so the
+application directory is writeable by the user.
+
+This directory location and semantics do *not* follow the standard Jupyter
+config semantics because we need to build a single unified application, and
+the default config location for Jupyter is at the user level (user's home
+directory). By explicitly using a sys-prefix directory, we can ensure better
+isolation in virtual environments.
+
+The JupyterLab application directory contains the subdirectories
+``extensions``, ``schemas``, ``settings``, ``staging``, ``static``, and
+``themes``. In the rest of this section, we will explain each subdirectory.
+
+.. _extensions-1:
+
+extensions
+^^^^^^^^^^
+
+The ``extensions`` directory has the packed tarballs for each of the
+installed extensions for the app. If the application directory is not
+the same as the ``sys-prefix`` directory, the extensions installed in
+the ``sys-prefix`` directory will be used in the app directory. If an
+extension is installed in the app directory that exists in the
+``sys-prefix`` directory, it will shadow the ``sys-prefix`` version.
+Uninstalling an extension will first uninstall the shadowed extension,
+and then attempt to uninstall the ``sys-prefix`` version if called
+again. If the ``sys-prefix`` version cannot be uninstalled, its plugins
+can still be ignored using ``ignoredPackages`` metadata in ``settings``.
+
+schemas
+^^^^^^^
+
+The ``schemas`` directory contains `JSON
+Schemas <http://json-schema.org/>`__ that describe the settings used by
+individual extensions. Users may edit these settings using the
+JupyterLab Advanced Settings Editor.
+
+settings
+^^^^^^^^
+
+The ``settings`` directory may contain ``page_config.json``, ``overrides.json``, and/or
+``build_config.json`` files, depending on which configurations are
+set on your system.
+
+.. _page_configjson:
+
+page_config.json
+""""""""""""""""
+
+The ``page_config.json`` data is used to provide configuration data to the
+application environment.
+
+The following configurations may be present in this file:
+
+1. ``terminalsAvailable`` identifies whether a terminal (i.e. ``bash/tsch``
+   on Mac/Linux OR ``PowerShell`` on Windows) is available to be launched
+   via the Launcher. (This configuration was predominantly required for
+   Windows prior to PowerShell access being enabled in Jupyter Lab.) The
+   value for this field is a Boolean: ``true`` or ``false``.
+2. ``disabledExtensions`` controls which extensions should not load at all.
+3. ``deferredExtensions`` controls which extensions should not load until
+   they are required by something, irrespective of whether they set
+   ``autoStart`` to ``true``.
+
+The values for the ``disabledExtensions`` and ``deferredExtensions`` fields
+are arrays of strings. The following sequence of checks are performed
+against the patterns in ``disabledExtensions`` and ``deferredExtensions``.
+
+-  If an identical string match occurs between a config value and a
+   package name (e.g., ``"@jupyterlab/apputils-extension"``), then the
+   entire package is disabled (or deferred).
+-  If the string value is compiled as a regular expression and tests
+   positive against a package name (e.g.,
+   ``"disabledExtensions": ["@jupyterlab/apputils*$"]``), then the
+   entire package is disabled (or deferred).
+-  If an identical string match occurs between a config value and an
+   individual plugin ID within a package (e.g.,
+   ``"disabledExtensions": ["@jupyterlab/apputils-extension:settings"]``),
+   then that specific plugin is disabled (or deferred).
+-  If the string value is compiled as a regular expression and tests
+   positive against an individual plugin ID within a package (e.g.,
+   ``"disabledExtensions": ["-@jupyterlab/apputils-extension:set.*$"]``),
+   then that specific plugin is disabled (or deferred).
+
+An example of a ``page_config.json`` file is:
+
+.. code:: json
+
+    {
+        "disabledExtensions": [
+            "@jupyterlab/toc"
+        ],
+        "terminalsAvailable": false
+    }
+
+.. _overridesjson:
+
+overrides.json
+""""""""""""""
+
+You can override default values of the extension settings by defining new
+default values in an ``overrides.json`` file. For example, if you would like
+to override the default theme to be the dark theme, an ``overrides.json`` file
+containing the following lines should be added in the application settings
+directory (by default this is the ``share/jupyter/lab/settings`` folder).
+
+.. code:: json
+
+  {
+    "@jupyterlab/apputils-extension:themes": {
+      "theme": "JupyterLab Dark"
+    }
+  }
+
+.. _build_configjson:
+
+build_config.json
+"""""""""""""""""
+
+
+The ``build_config.json`` file is used to track the local directories
+that have been installed using
+``jupyter labextension install <directory>``, as well as core extensions
+that have been explicitly uninstalled. An example of a
+``build_config.json`` file is:
+
+.. code:: json
+
+    {
+        "uninstalled_core_extensions": [
+            "@jupyterlab/markdownwidget-extension"
+        ],
+        "local_extensions": {
+            "@jupyterlab/python-tests": "/path/to/my/extension"
+        }
+    }
+
+
+staging and static
+^^^^^^^^^^^^^^^^^^
+
+The ``static`` directory contains the assets that will be loaded by the
+JupyterLab application. The ``staging`` directory is used to create the
+build and then populate the ``static`` directory.
+
+Running ``jupyter lab`` will attempt to run the ``static`` assets in the
+application directory if they exist. You can run ``jupyter lab --core-mode``
+to load the core JupyterLab application from the installation directory (i.e.,
+the application without any extensions) instead.
+
+If JupyterLab is launched and the static assets are not present, it will
+display an error in the console and in the browser.
+
+themes
+^^^^^^
+
+The ``themes`` directory contains assets (such as CSS and icons) for
+JupyterLab theme extensions.
+
+
+JupyterLab User Settings Directory
+----------------------------------
+
+The user settings directory contains the user-level settings for Jupyter
+extensions.
+
+By default, the location is ``$HOME/.jupyter/lab/user-settings/``, where
+``$HOME`` is the user's home directory. This folder is not in the JupyterLab
+application directory because these settings are typically shared across
+Python environments. The location can be modified using the
+``JUPYTERLAB_SETTINGS_DIR`` environment variable.
+
+`JSON5 <https://json5.org/>`__ files are automatically created in this folder
+recording the settings changes a user makes in the JupyterLab Advanced
+Settings Editor. The file names follow the pattern of
+``<extension_name>/<plugin_name>.jupyterlab-settings``. These values override
+the default values given by extensions, as well as the default overrides from
+the :ref:`overrides.json <overridesjson>` file in the application's settings
+directory.
+
+JupyterLab Workspaces Directory
+-------------------------------
+
+JupyterLab sessions always reside in a workspace. Workspaces contain the state
+of JupyterLab: the files that are currently open, the layout of the
+application areas and tabs, etc. When the page is refreshed, the workspace is
+restored.
+
+By default, the location is ``$HOME/.jupyter/lab/workspaces/``, where
+``$HOME`` is the user's home directory. This folder is not in the JupyterLab
+application directory, because these files are typically shared across Python
+environments. The location can be modified using the
+``JUPYTERLAB_WORKSPACES_DIR`` environment variable.
+
+These files can be imported and exported to create default "profiles", using
+the :ref:`workspace command line tool <url-workspaces-cli>`.
+

docs/source/user/extensions.rst

@@ -470,269 +470,3 @@ all jupyterlab organization extensions.
    }
 
 
-
-Advanced Usage
---------------
-
-Any information that JupyterLab persists is stored in its application directory,
-including settings and built assets.
-This is separate from where the Python package is installed (like in ``site_packages``)
-so that the install directory is immutable.
-
-The application directory can be overridden using ``--app-dir`` in
-any of the JupyterLab commands, or by setting the ``JUPYTERLAB_DIR``
-environment variable. If not specified, it will default to
-``<sys-prefix>/share/jupyter/lab``, where ``<sys-prefix>`` is the
-site-specific directory prefix of the current Python environment. You
-can query the current application path by running ``jupyter lab path``.
-Note that the application directory is expected to contain the JupyterLab
-static assets (e.g. ``static/index.html``).  If JupyterLab is launched
-and the static assets are not present, it will display an error in the console and in the browser.
-
-JupyterLab Build Process
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-To rebuild the app directory, run ``jupyter lab build``. By default, the
-``jupyter labextension install`` command builds the application, so you
-typically do not need to call ``build`` directly.
-
-Building consists of:
-
--  Populating the ``staging/`` directory using template files
--  Handling any locally installed packages
--  Ensuring all installed assets are available
--  Bundling the assets
--  Copying the bundled assets to the ``static`` directory
-
-Note that building will always use the latest JavaScript packages that meet
-the dependency requirements of JupyterLab itself and any installed extensions.
-If you wish to run JupyterLab with the set of pinned requirements that was
-shipped with the Python package, you can launch as
-``jupyter lab --core-mode``.
-
-The build process uses a specific ``yarn`` version with a default working
-combination of npm packages stored in a ``yarn.lock`` file shipped with
-JupyterLab. Those package source urls point to the default yarn registry.
-However, if you defined your own yarn registry in the yarn configuration, the
-default yarn registry will be replaced by your custom registry. If you then
-switch back to the default yarn registry, you will need to clean your
-``staging`` folder before building:
-
-.. code:: bash
-
-    jupyter lab clean
-    jupyter lab build
-
-
-Disabling Rebuild Checks
-""""""""""""""""""""""""
-
-JupyterLab automatically checks to see if it needs to rebuild on startup. In
-some cases, such as automated testing, you may wish to disable the startup
-rebuild checks altogether. This can be achieved through setting ``buildCheck``
-and ``buildAvailable`` in ``jupyter_notebook_config.json`` (or ``.py``
-equivalent) in any of the ``config`` locations returned by ``jupyter
---paths``.
-
-
-.. code:: json
-
-    {
-      "LabApp": {
-        "tornado_settings": {
-          "page_config_data": {
-            "buildCheck": false,
-            "buildAvailable": false,
-          }
-        }
-      }
-    }
-
-
-JupyterLab Application Directory
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-The JupyterLab application directory contains the subdirectories
-``extensions``, ``schemas``, ``settings``, ``staging``, ``static``, and
-``themes``.  The default application directory mirrors the location where
-JupyterLab was installed.  For example, in a conda environment, it is in
-``<conda_root>/envs/<env_name>/share/jupyter/lab``.  The directory can be
-overridden by setting a ``JUPYTERLAB_DIR`` environment variable.
-
-It is not recommended to install JupyterLab in a root location (on Unix-like
-systems).  Instead, use a conda environment or ``pip install --user jupyterlab``
-so that the application directory ends up in a writable location.
-
-Note: this folder location and semantics do *not* follow the standard Jupyter
-config semantics because we need to build a single unified application, and the
-default config location for Jupyter is at the user level (user's home directory).
-By explicitly using a directory alongside the currently installed JupyterLab,
-we can ensure better isolation between conda or other virtual environments.
-
-.. _extensions-1:
-
-extensions
-""""""""""
-
-The ``extensions`` directory has the packed tarballs for each of the
-installed extensions for the app. If the application directory is not
-the same as the ``sys-prefix`` directory, the extensions installed in
-the ``sys-prefix`` directory will be used in the app directory. If an
-extension is installed in the app directory that exists in the
-``sys-prefix`` directory, it will shadow the ``sys-prefix`` version.
-Uninstalling an extension will first uninstall the shadowed extension,
-and then attempt to uninstall the ``sys-prefix`` version if called
-again. If the ``sys-prefix`` version cannot be uninstalled, its plugins
-can still be ignored using ``ignoredPackages`` metadata in ``settings``.
-
-schemas
-"""""""
-
-The ``schemas`` directory contains `JSON
-Schemas <http://json-schema.org/>`__ that describe the settings used by
-individual extensions. Users may edit these settings using the
-JupyterLab Settings Editor.
-
-settings
-""""""""
-
-The ``settings`` directory may contain ``page_config.json``, ``overrides.json``, and/or
-``build_config.json`` files, depending on which configurations are
-set on your system.
-
-.. _page_configjson:
-
-*page_config.json*
-
-The ``page_config.json`` data is used to provide configuration data to the
-application environment.
-
-The following configurations may be present in this file:
-
-1. ``terminalsAvailable`` identifies whether a terminal (i.e. ``bash/tsch``
-   on Mac/Linux OR ``PowerShell`` on Windows) is available to be launched
-   via the Launcher. (This configuration was predominantly required for
-   Windows prior to PowerShell access being enabled in Jupyter Lab.) The
-   value for this field is a Boolean: ``true`` or ``false``.
-2. ``disabledExtensions`` controls which extensions should not load at all.
-3. ``deferredExtensions`` controls which extensions should not load until
-   they are required by something, irrespective of whether they set
-   ``autoStart`` to ``true``.
-
-The value for the ``disabledExtensions`` and ``deferredExtensions`` fields
-are an array of strings. The following sequence of checks are performed
-against the patterns in ``disabledExtensions`` and ``deferredExtensions``.
-
--  If an identical string match occurs between a config value and a
-   package name (e.g., ``"@jupyterlab/apputils-extension"``), then the
-   entire package is disabled (or deferred).
--  If the string value is compiled as a regular expression and tests
-   positive against a package name (e.g.,
-   ``"disabledExtensions": ["@jupyterlab/apputils*$"]``), then the
-   entire package is disabled (or deferred).
--  If an identical string match occurs between a config value and an
-   individual plugin ID within a package (e.g.,
-   ``"disabledExtensions": ["@jupyterlab/apputils-extension:settings"]``),
-   then that specific plugin is disabled (or deferred).
--  If the string value is compiled as a regular expression and tests
-   positive against an individual plugin ID within a package (e.g.,
-   ``"disabledExtensions": ["^@jupyterlab/apputils-extension:set.*$"]``),
-   then that specific plugin is disabled (or deferred).
-
-An example of a ``page_config.json`` file is:
-
-.. code:: json
-
-    {
-        "disabledExtensions": [
-            "@jupyterlab/toc"
-        ],
-        "terminalsAvailable": false
-    }
-
-.. _overridesjson:
-
-*overrides.json*
-
-You can override default values of the extension settings by
-defining new default values in an ``overrides.json`` file.
-So for example, if you would like
-to set the dark theme by default instead of the light one, an
-``overrides.json`` file containing the following lines needs to be
-added in the application settings directory (by default this is the
-``share/jupyter/lab/settings`` folder).
-
-.. code:: json
-
-  {
-    "@jupyterlab/apputils-extension:themes": {
-      "theme": "JupyterLab Dark"
-    }
-  }
-
-.. _build_configjson:
-
-*build_config.json*
-
-
-The ``build_config.json`` file is used to track the local directories
-that have been installed using
-``jupyter labextension install <directory>``, as well as core extensions
-that have been explicitly uninstalled. An example of a
-``build_config.json`` file is:
-
-.. code:: json
-
-    {
-        "uninstalled_core_extensions": [
-            "@jupyterlab/markdownwidget-extension"
-        ],
-        "local_extensions": {
-            "@jupyterlab/python-tests": "/path/to/my/extension"
-        }
-    }
-
-
-staging and static
-""""""""""""""""""
-
-The ``static`` directory contains the assets that will be loaded by the
-JupyterLab application. The ``staging`` directory is used to create the
-build and then populate the ``static`` directory.
-
-Running ``jupyter lab`` will attempt to run the ``static`` assets in the
-application directory if they exist. You can run
-``jupyter lab --core-mode`` to load the core JupyterLab application
-(i.e., the application without any extensions) instead.
-
-themes
-""""""
-
-The ``themes`` directory contains assets (such as CSS and icons) for
-JupyterLab theme extensions.
-
-
-JupyterLab User Settings Directory
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-The user settings directory contains the user-level settings for Jupyter extensions.
-By default, the location is ``-/.jupyter/lab/user-settings/``, where ``-`` is the user's home directory. This folder is not in the JupyterLab application directory,
-because these settings are typically shared across Python environments.
-The location can be modified using the ``JUPYTERLAB_SETTINGS_DIR`` environment variable. Files are automatically created in this folder as modifications are made
-to settings from the JupyterLab UI. They can also be manually created.  The files
-follow the pattern of ``<package_name>/<extension_name>.jupyterlab-settings``.
-They are JSON files with optional comments. These values take precedence over the
-default values given by the extensions, but can be overridden by an ``overrides.json``
-file in the application's settings directory.
-
-
-JupyterLab Workspaces Directory
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-JupyterLab sessions always reside in a workspace. Workspaces contain the state
-of JupyterLab: the files that are currently open, the layout of the application
-areas and tabs, etc. When the page is refreshed, the workspace is restored.
-By default, the location is ``-/.jupyter/lab/workspaces/``, where ``-`` is the user's home directory. This folder is not in the JupyterLab application directory,
-because these files are typically shared across Python environments.
-The location can be modified using the ``JUPYTERLAB_WORKSPACES_DIR`` environment variable. These files can be imported and exported to create default "profiles",
-using the :ref:`workspace command line tool <url-workspaces-cli>`.
-