absorb extension-builder source

packages/extension-builder/README.md

@@ -0,0 +1,198 @@
+# JupyterLab Extension Builder
+
+**Tools for building JupyterLab extensions**
+
+A JupyterLab extension provides additional, optional functionality to 
+JupyterLab's built-in capabilities. An extension is a module that provides 
+one or more plugins to the JupyterLab application. To streamline third-party 
+development of extensions, this library provides a build script for generating 
+third party extension JavaScript bundles.  
+
+Simple extensions can be created by using the `buildExtension` function
+with the default options.  More advanced extensions may require additional
+configuration such as custom loaders or WebPack plugins.
+
+A video tutorial walkthrough for building JupyterLab extensions can be found on [YouTube](https://youtu.be/WVDCMIQ3KOk).
+
+
+## Package Install
+
+**Prerequisites**
+- [node](http://nodejs.org/)
+
+```bash
+npm install --save @jupyterlab/extension-builder
+```
+
+
+## Source Build
+
+**Prerequisites**
+- [git](http://git-scm.com/)
+- [node 4+](http://nodejs.org/)
+
+```bash
+git clone https://github.com/jupyterlab/jupyterlab.git
+cd jupyterlab
+npm install
+npm run build
+cd packages/extension-builder
+```
+
+
+### Rebuild Source
+
+```bash
+npm run clean
+npm run build
+```
+
+### Debugging
+
+```
+npm install -g devtool
+```
+
+Insert a `debugger;` statement in the source where you want the execution to
+stop in the debugger. Then execute an extension build with
+
+```
+devtool <my_build_script.js>
+```
+
+or if running WebPack directly,
+
+```
+devtool <path_to_webpack_in_node_modules/bin>
+```
+
+## Usage
+
+Three major usage steps include:
+- creating the [extension entry point](#extension-entry-point)
+- build using the [buildExtension](#buildExtension) script
+- register the extension using [`jupyter labextension`](#jupyter-labextension)
+
+The full API docs can be found [here](http://jupyterlab.github.io/jupyterlab/).
+
+### Extension entry point
+A simple extension entry point that exposes a single application [plugin](http://jupyterlab-tutorial.readthedocs.io/en/latest/plugins.html) could 
+look like: 
+
+```javascript
+module.exports = [{
+    id: 'my-cool-extension',
+    activate: function(app) {
+       console.log(app.commands);
+    }
+}];
+```
+
+The extension entry point *must* be a CommonJS module where the default
+export is an array of plugin objects.  If writing in ES6 format use the default
+export syntax `export default myPlugin` for a single plugin, and the following 
+pattern for multiple exports  (remove the type declaration if not using 
+TypeScript):
+
+```typescript
+import { JupyterLabPlugin } from 'jupyterlab/lib/application';
+// Plugins defined here
+const plugins: JupyterlabPlugin<any>[] = [ ... ];
+export default plugins;
+```
+
+
+### buildExtension
+Build the above example using the following script:
+
+```javascript
+var buildExtension = require('@jupyterlab/extension-builder').buildExtension;
+
+buildExtension({
+    name: 'my-cool-extension',
+    entry: './index.js',
+    outputDir: './build'
+});
+```
+The `name` is a string that will be used for the output filename. The `entry` is the module that exports a plugin definition or array of plugin definitions. The `outputDir` is the directory in which the generated plugin bundle, manifest, and related files will be stored.
+
+Several optional arguments are also available; see the options at the bottom of the [builder.ts](https://github.com/jupyter/jupyterlab-extension-builder/blob/master/src/builder.ts) file.
+
+In this case the builder script will create the following files in the build
+directory:
+
+```
+my-cool-extension.bundle.js
+my-cool-extension.js.manifest
+```
+
+### jupyter labextension
+Other extensions may produce additional files in the build directory depending
+on the complexity of extension.  The two files above, `my-cool-extension.js` and
+`my-cool-extension.js.manifest`, are used by the JupyterLab server to determine
+the entry point file(s) and entry point module(s) for the extension.  The
+extension must also be registered, using the command `jupyter labextension`, in
+order to be added to the JupyterLab application.  See the documentation for
+[labextension](http://jupyterlab-tutorial.readthedocs.io/en/latest/labextensions.html)
+
+
+## Technical overview
+
+The extension bundles are created using WebPack, and the modules produced by
+WebPack are modified to use JupyterLab's custom module registration and loading
+mechanism.
+
+JupyterLab's custom module registration and loading mechanism uses a `define`
+function that registers modules by name, where the name contains the package
+name, version number, and the full path to the module.  For example,
+`'phosphor@0.6.1/lib/ui/widget.js'`.  Within a `define` function, a required
+module is referenced by package name, semver range, and the full path to the
+module.  For example, `require('phosphor@^0.6.0/lib/ui/tabpanel.js')`. The
+semver range is determined by the following criteria (see the
+`getModuleSemverPath` function in `plugin.ts`:
+
+1. If the dependency is in the same package, the exact version of the dependency
+   is used.
+2. If the dependency is a local package (i.e., module given by `file://...`),
+   the semver is the patch-level range (`~`) starting from the installed
+   version.
+3. If the dependency is in the dependency list of the module's `package.json`,
+   then the semver range requested there is used.
+4. Otherwise the installed version of the dependency is used exactly. Note that
+   not listing an external dependency in the package metadata is a bad practice
+   that leads to almost no deduping.
+
+By using a semver range, JupyterLab can perform client-side deduplication of
+modules, where the registered module that maximally satisfies a semver range is
+the one returned by the `require` function call.  This also enables us to
+perform server-side deduplication of modules prior to serving the bundles, and
+the client-side lookup will still load the correct modules.
+
+Reasons to deduplicate code include:
+
+- being able to use `instanceof()` on an object to determine if it is the same class (a technique used by phosphor's drag-drop mechanism)
+- sharing of module-private state between different consumers, such as a list of client-side running kernels in `@jupyterlab/services`.
+
+All client-side `require()` calls are synchronous, which means that the bundles
+containing the `define()` modules must be loaded prior to using any of the
+bundles' functions.  The [loader](http://jupyterlab.github.io/jupyterlab/classes/_application_loader_.moduleloader.html) in JupyterLab provides an `ensureBundle()` 
+function to load a particular bundle or bundles prior to calling `require()` 
+on a module.
+
+### Custom WebPack Configuration and JupyterLabPlugin
+A completely custom WebPack configuration may be needed if there is a case where
+the `buildExtension` function is not sufficient to build the extension. If a
+custom WebPack configuration is needed, the `JupyterLabPlugin` must be used as
+part of the WebPack config to ensure proper handling of module definition and
+requires.
+
+
+## Publishing your extension
+Before you publish your extension to `npm`, add the following `keywords` attribute to your extension's `package.json`:
+```
+{
+    "keywords": ["jupyterlab", "jupyterlab extension"],
+    ...
+}
+```
+Adding these keywords will allow other users to discover your extension with `npm search`.

packages/extension-builder/package.json

@@ -0,0 +1,48 @@
+{
+  "name": "@jupyterlab/extension-builder",
+  "version": "0.11.0",
+  "description": "Tools for building JupyterLab extensions",
+  "main": "lib/index.js",
+  "types": "lib/index.d.ts",
+  "files": [
+    "lib/*.d.ts",
+    "lib/*.js",
+    "style/*.css"
+  ],
+  "directories": {
+    "lib": "lib/"
+  },
+  "dependencies": {
+    "@types/webpack": "^1.12.35",
+    "css-loader": "^0.25.0",
+    "extract-text-webpack-plugin": "^1.0.1",
+    "file-loader": "^0.9.0",
+    "json-loader": "^0.5.4",
+    "semver": "^5.3.0",
+    "style-loader": "^0.13.1",
+    "supports-color": "^3.1.2",
+    "url-loader": "^0.5.7",
+    "webpack": "^2.3.1",
+    "webpack-config": "^6.2.0"
+  },
+  "devDependencies": {
+    "@types/semver": "^5.3.30",
+    "rimraf": "^2.5.2",
+    "typescript": "^2.2.1"
+  },
+  "scripts": {
+    "build": "tsc",
+    "clean": "rimraf lib",
+    "watch": "tsc -w"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/jupyterlab/jupyterlab.git"
+  },
+  "author": "Project Jupyter",
+  "license": "BSD-3-Clause",
+  "bugs": {
+    "url": "https://github.com/jupyterlab/jupyterlab/issues"
+  },
+  "homepage": "https://github.com/jupyterlab/jupyterlab"
+}

packages/extension-builder/src/builder.ts

@@ -0,0 +1,182 @@
+// Copyright (c) Jupyter Development Team.
+// Distributed under the terms of the Modified BSD License.
+
+import * as ExtractTextPlugin
+  from 'extract-text-webpack-plugin';
+
+import * as path
+  from 'path';
+
+import * as webpack
+  from 'webpack';
+
+import {
+  Config
+} from 'webpack-config';
+
+import {
+  JupyterLabPlugin
+} from './plugin';
+
+
+/**
+ * The default file loaders.
+ */
+const
+DEFAULT_LOADERS = [
+  { test: /\.json$/, use: 'json-loader' },
+  { test: /\.html$/, use: 'file-loader' },
+  { test: /\.(jpg|png|gif)$/, use: 'file-loader' },
+  { test: /\.js.map$/, use: 'file-loader' },
+  { test: /\.woff2(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=application/font-woff' },
+  { test: /\.woff(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=application/font-woff' },
+  { test: /\.ttf(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=application/octet-stream' },
+  { test: /\.eot(\?v=\d+\.\d+\.\d+)?$/, use: 'file-loader' },
+  { test: /\.svg(\?v=\d+\.\d+\.\d+)?$/, use: 'url-loader?limit=10000&mimetype=image/svg+xml' }
+];
+
+
+/**
+ * Build a JupyterLab extension.
+ *
+ * @param options - The options used to build the extension.
+ */
+export
+function buildExtension(options: IBuildOptions): Promise<void> {
+  let name = options.name;
+
+  if (!name) {
+    throw Error('Must specify a name for the extension');
+  }
+  if (!options.entry) {
+    throw Error('Must specify an entry module');
+  }
+  if (!options.outputDir) {
+    throw Error('Must specify an output directory');
+  }
+
+  // Create the named entry point to the entryPath.
+  let entry: { [key: string]: string } = {};
+  entry[name] = options.entry;
+
+  let config = new Config().merge({
+    // The default options.
+    entry: entry,
+    output: {
+      path: path.resolve(options.outputDir),
+      filename: '[name].bundle.js',
+      publicPath: `labextension/${name}`
+    },
+    node: {
+      fs: 'empty'
+    },
+    bail: true,
+    plugins: [new JupyterLabPlugin()]
+  // Add the override options.
+  }).merge(options.config || {});
+
+  // Add the CSS extractors unless explicitly told otherwise.
+  if (options.extractCSS !== false) {
+    // Note that we have to use an explicit local public path
+    // otherwise the urls in the extracted CSS will point to the wrong
+    // location.
+    // See https://github.com/webpack-contrib/extract-text-webpack-plugin/tree/75cb09eed13d15cec8f974b1210920a7f249f8e2
+    let cssLoader = ExtractTextPlugin.extract({
+      use: 'css-loader',
+      fallback: 'style-loader',
+      publicPath: './'
+    });
+    config.merge({
+      module: {
+        rules: [
+          {
+            test: /\.css$/,
+            use: cssLoader
+          }
+        ]
+      },
+      plugins: [new ExtractTextPlugin('[name].css')]
+    });
+  }
+
+  // Add the rest of the default loaders unless explicitly told otherwise.
+  if (options.useDefaultLoaders !== false) {
+    config.merge({
+      module: {
+        rules: DEFAULT_LOADERS
+      }
+    });
+  }
+
+  // Set up and run the WebPack compilation.
+  let compiler = webpack(config);
+  compiler.name = name;
+
+  return new Promise<void>((resolve, reject) => {
+    compiler.run((err, stats) => {
+      if (err) {
+        console.error(err.stack || err);
+        if ((err as any).details) {
+          console.error((err as any).details);
+        }
+        reject(err);
+      } else {
+        console.log(`\n\nSuccessfully built "${name}":\n`);
+        process.stdout.write(stats.toString({
+          chunks: true,
+          modules: false,
+          chunkModules: false,
+          colors: require('supports-color')
+        }) + '\n');
+        resolve();
+      }
+    });
+  });
+}
+
+
+/**
+ * The options used to build a JupyterLab extension.
+ */
+export
+interface IBuildOptions {
+  /**
+   * The name of the extension.
+   */
+  name: string;
+
+  /**
+   * The module to load as the entry point.
+   *
+   * The module should export a plugin configuration or array of
+   * plugin configurations.
+   */
+  entry: string;
+
+  /**
+   * The directory in which to put the generated bundle files.
+   *
+   * Relative directories are resolved relative to the current
+   * working directory of the process.
+   */
+  outputDir: string;
+
+  /**
+   * Whether to extract CSS from the bundles (default is True).
+   *
+   * Note: no other CSS loaders should be used if not set to False.
+   */
+  extractCSS?: boolean;
+
+  /**
+   * Whether to use the default loaders for some common file types.
+   *
+   * See [[DEFAULT_LOADERS]].  The default is True.
+   */
+  useDefaultLoaders?: boolean;
+
+  /**
+   * Extra webpack configuration.
+   */
+  config?: webpack.Configuration;
+}

packages/extension-builder/src/index.ts

@@ -0,0 +1,5 @@
+// Copyright (c) Jupyter Development Team.
+// Distributed under the terms of the Modified BSD License.
+
+export * from './builder';
+export * from './plugin';

packages/extension-builder/src/plugin.ts

@@ -0,0 +1,484 @@
+// Copyright (c) Jupyter Development Team.
+// Distributed under the terms of the Modified BSD License.
+
+import * as path
+  from 'path';
+
+import * as webpack
+  from 'webpack';
+
+
+/**
+ * A WebPack plugin that generates custom bundles that use version and
+ * semver-mangled require semantics.
+ */
+export
+class JupyterLabPlugin {
+  /**
+   * Construct a new JupyterLabPlugin.
+   */
+  constructor(options?: JupyterLabPlugin.IOptions) {
+    options = options || {};
+    this._name = options.name || 'jupyter';
+  }
+
+  /**
+   * Plugin installation, called by WebPack.
+   *
+   * @param compiler - The WebPack compiler object.
+   */
+  apply(compiler: webpack.compiler.Compiler) {
+    let publicPath = compiler.options.output.publicPath;
+    if (!publicPath) {
+      throw new Error('Must define a public path');
+    }
+    if (publicPath[publicPath.length - 1] !== '/') {
+      publicPath += '/';
+    }
+    this._publicPath = publicPath;
+
+    // Notes
+    // We use the emit phase because it allows other plugins to act on the
+    // output first.
+    // We can't replace the module ids during compilation, because there are
+    // places in the compilation that assume a numeric id.
+    compiler.plugin('emit', this._onEmit.bind(this));
+  }
+
+  private _onEmit(compilation: any, callback: () => void): void {
+
+    // Explore each chunk (build output):
+    compilation.chunks.forEach((chunk: any) => {
+
+      let sources: string[] = [];
+
+      // A mapping for each module name and its dependencies.
+      let modules: any = {};
+
+      // Explore each module within the chunk (built inputs):
+      chunk.modules.forEach((mod: any) => {
+
+        // We don't allow externals.
+        if (mod.external) {
+          throw Error(`Cannot use externals: ${mod.userRequest}`);
+        }
+
+        // Parse each module.
+        let source = this._parseModule(compilation, mod);
+        sources.push(source);
+
+        // Add dependencies to the manifest.
+        let deps: string[] = [];
+        for (let i = 0; i < mod.dependencies.length; i++) {
+          let dep = mod.dependencies[i];
+          if (dep.module && dep.module.id && dep.module.id !== mod.id) {
+            deps.push(Private.getRequirePath(mod, dep.module));
+          }
+        }
+        modules[Private.getDefinePath(mod)] = deps;
+      });
+
+      let code = sources.join('\n\n');
+
+      // Replace the original chunk file.
+      // Use the first file name, because the mangling of the chunk
+      // file names are private to WebPack.
+      let fileName = chunk.files[0];
+      compilation.assets[fileName] = {
+        source: function() {
+          return code;
+        },
+        size: function() {
+          return code.length;
+        }
+      };
+
+      // Create a manifest for the chunk.
+      let manifest: any = {};
+      if (chunk.entryModule) {
+        manifest['entry'] = Private.getDefinePath(chunk.entryModule);
+      }
+      manifest['hash'] = chunk.hash;
+      manifest['id'] = chunk.id;
+      manifest['name'] = chunk.name || chunk.id;
+      manifest['files'] = chunk.files;
+      manifest['modules'] = modules;
+
+      let manifestSource = JSON.stringify(manifest, null, '\t');
+
+      compilation.assets[`${fileName}.manifest`] = {
+        source: () => {
+          return manifestSource;
+        },
+        size: () => {
+          return manifestSource.length;
+        }
+      };
+
+    });
+
+    callback();
+
+  }
+
+  /**
+   * Parse a WebPack module to generate a custom version.
+   *
+   * @param compilation - The Webpack compilation object.
+   *
+   * @param module - A parsed WebPack module object.
+   *
+   * @returns The new module contents.
+   */
+  private _parseModule(compilation: any, mod: any): string {
+    let pluginName = this._name;
+    let publicPath = this._publicPath;
+    let requireName = `__${pluginName}_require__`;
+    // There is no public API in WebPack to get the raw module source
+    // The method used below is known to work in almost all cases
+    // The base prototype of the module source() method takes no arguments,
+    // but the normal module source() takes three arguments and is intended
+    // to be called by its module factory.
+    // We can call the normal module source() because it has already been
+    // run in the compilation process and will return the cached value,
+    // without relying on the provided arguments.
+    // https://github.com/webpack/webpack/blob/a53799c0ac58983860a27648cdc8519b6a562b89/lib/NormalModule.js#L224-L229
+    let source = mod.source().source();
+
+    // Regular modules.
+    if (mod.userRequest) {
+      // Handle ensure blocks with and without inline comments.
+      // From WebPack dependencies/DepBlockHelpers
+      source = this._handleEnsure(
+        compilation, source, /__webpack_require__.e\/\*.*?\*\/\((\d+)/
+      );
+      source = this._handleEnsure(
+        compilation, source, /__webpack_require__.e\((\d+)/
+      );
+
+      // Replace the require statements with the semver-mangled name.
+      let deps = Private.getAllModuleDependencies(mod);
+      for (let i = 0; i < deps.length; i++) {
+        let dep = deps[i];
+        let target = `__webpack_require__(${dep.id})`;
+        let modPath = Private.getRequirePath(mod, dep);
+        let replacer = `__webpack_require__('${modPath}')`;
+        source = source.split(target).join(replacer);
+      }
+    // Context modules.
+    } else if (mod.context) {
+      // Context modules have to be assembled ourselves
+      // because they are not clearly delimited in the text.
+      source = Private.createContextModule(mod);
+      source = source.split('webpackContext').join(`${pluginName}Context`);
+    }
+
+    // Handle public requires.
+    let requireP = '__webpack_require__.p +';
+    let newRequireP = `'${publicPath}' +`;
+    source = source.split(requireP).join(newRequireP);
+
+    // Replace the require name with the custom one.
+    source = source.split('__webpack_require__').join(requireName);
+
+    // Handle ES6 exports
+    source = source.split('__webpack_exports__').join('exports');
+
+    // Create our header and footer with a version-mangled defined name.
+    let definePath = Private.getDefinePath(mod);
+    let header = `/** START DEFINE BLOCK for ${definePath} **/
+${pluginName}.define('${definePath}', function (module, exports, ${requireName}) {
+\t`;
+    let footer = `
+})
+/** END DEFINE BLOCK for ${definePath} **/
+`;
+
+    // Combine code and indent.
+    return header + source.split('\n').join('\n\t') + footer;
+  }
+
+  /**
+   * Handle an ensure block.
+   *
+   * @param compilation - The Webpack compilation object.
+   *
+   * @param source - The raw module source.
+   *
+   * @param publicPath - The public path of the plugin.
+   *
+   * @param regex - The ensure block regex.
+   *
+   * @returns The new ensure block contents.
+   */
+  private _handleEnsure(compilation: any, source: string, regex: RegExp) {
+    let publicPath = this._publicPath;
+    while (regex.test(source)) {
+      let match = source.match(regex);
+      let chunkId = match[1];
+      let fileName = '';
+      // Use the first file name, because the mangling of the chunk
+      // file name is private to WebPack.
+      compilation.chunks.forEach((chunk: any) => {
+        if (String(chunk.id) === chunkId) {
+          fileName = chunk.files[0];
+        }
+      });
+      let replacement = `__webpack_require__.e('${publicPath}${fileName}'`;
+      source = source.replace(regex, replacement);
+    }
+    return source;
+  }
+
+  private _name = '';
+  private _publicPath = '';
+}
+
+
+/**
+ * A namespace for `JupyterLabPlugin` statics.
+ */
+export
+namespace JupyterLabPlugin {
+  export
+  interface IOptions {
+    /**
+     * The name of the plugin.
+     */
+    name?: string;
+  }
+}
+
+
+/**
+ * A namespace for module private data.
+ */
+namespace Private {
+
+  /**
+   * Get the define path for a WebPack module.
+   *
+   * @param module - A parsed WebPack module object.
+   *
+   * @returns A version-mangled define path for the module.
+   *    For example, 'foo@1.0.1/lib/bar/baz.js'.
+   */
+  export
+  function getDefinePath(mod: any): string {
+    if (!mod.context) {
+      return '__ignored__';
+    }
+    let request = mod.userRequest || mod.context;
+    let parts = request.split('!');
+    let names: string[] = [];
+    for (let i = 0; i < parts.length; i++) {
+      names.push(getModuleVersionPath(parts[i]));
+    }
+    return names.join('!');
+  }
+
+  /**
+   * Get the require path for a WebPack module.
+   *
+   * @param mod - A parsed WebPack module that is requiring a dependency.
+   * @param dep - A parsed WebPack module object representing the dependency.
+   *
+   * @returns A semver-mangled define path for the dependency.
+   *    For example, 'foo@^1.0.0/lib/bar/baz.js'.
+   */
+  export
+  function getRequirePath(mod: any, dep: any): string {
+    if (!dep.context) {
+      return '__ignored__';
+    }
+    let issuer = mod.userRequest || mod.context;
+    let request = dep.userRequest || dep.context;
+    let requestParts = request.split('!');
+    let parts: string[] = [];
+
+    // Handle the loaders.
+    for (let i = 0; i < requestParts.length - 1; i++) {
+      parts.push(getModuleSemverPath(requestParts[i], requestParts[i]));
+    }
+    // Handle the last part.
+    let base = requestParts[requestParts.length - 1];
+    parts.push(getModuleSemverPath(base, issuer));
+    return parts.join('!');
+  }
+
+  /**
+   * Create custom context module source.
+   *
+   * @param module - A parsed WebPack module object.
+   *
+   * @returns The new contents of the context module output.
+   */
+  export
+  function createContextModule(mod: any): string {
+    // Modeled after Webpack's ContextModule.js.
+    let map: { [key: string]: string } = {};
+    let dependencies = mod.dependencies || [];
+    dependencies.slice().sort((a: any, b: any) => {
+      if (a.userRequest === b.userRequest) {
+        return 0;
+      }
+      return a.userRequest < b.userRequest ? -1 : 1;
+    }).forEach((dep: any) => {
+      if (dep.module) {
+        map[dep.userRequest] = getRequirePath(mod, dep.module);
+      }
+    });
+    let mapString = JSON.stringify(map, null, '\t');
+    return generateContextModule(mapString, getDefinePath(mod));
+  }
+
+  /**
+   * Get all of the module dependencies for a module.
+   */
+  export
+  function getAllModuleDependencies(mod: any): any[] {
+    // Extracted from https://github.com/webpack/webpack/blob/ee1b8c43b474b22a20bfc25daf0ee153dfb2ef9f/lib/NormalModule.js#L227
+    let list: any[] = [];
+
+    function doDep(dep: any) {
+      if (dep.module && list.indexOf(dep.module) < 0) {
+        list.push(dep.module);
+      }
+    }
+
+    function doVariable(variable: any) {
+      variable.dependencies.forEach(doDep);
+    }
+
+    function doBlock(block: any) {
+      block.variables.forEach(doVariable);
+      block.dependencies.forEach(doDep);
+      block.blocks.forEach(doBlock);
+    }
+
+    doBlock(mod);
+    return list;
+  }
+
+  /**
+   * Find a package root path from a request.
+   *
+   * @param request - The request path.
+   *
+   * @returns The path to the package root.
+   */
+  function findRoot(request: string): string {
+    let orig = request;
+    if (path.extname(request)) {
+      request = path.dirname(request);
+    }
+    while (true) {
+      try {
+        let pkgPath = require.resolve(path.join(request, 'package.json'));
+        let pkg = require(pkgPath);
+        // Use public packages except for the local package.
+        if (!pkg.private || request === (process as any).cwd()) {
+          return request;
+        }
+      } catch (err) {
+        // no-op
+      }
+      let prev = request;
+      request = path.dirname(request);
+      if (request === prev) {
+        throw Error(`Could not find package for ${orig}`);
+      }
+    }
+  }
+
+  /**
+   * Get the package.json associated with a file.
+   *
+   * @param request - The request path.
+   *
+   * @returns The package.json object for the package.
+   */
+  function getPackage(request: string): any {
+    let rootPath = findRoot(request);
+    return require(path.join(rootPath, 'package.json'));
+  }
+
+  /**
+   * Get a mangled path for a path using the exact version.
+   *
+   * @param modPath - The absolute path of the module.
+   *
+   * @returns A version-mangled path (e.g. 'foo@1.0.0/lib/bar/baz.js')
+   */
+  function getModuleVersionPath(modPath: string): string {
+    let rootPath = findRoot(modPath);
+    let pkg = getPackage(rootPath);
+    modPath = modPath.slice(rootPath.length + 1);
+    let name = `${pkg.name}@${pkg.version}`;
+    if (modPath) {
+      modPath = modPath.split(path.sep).join('/');
+      name += `/${modPath}`;
+    }
+    return name;
+  }
+
+  /**
+   * Get the semver-mangled path for a request.
+   *
+   * @param request - The requested module path.
+   *
+   * @param issuer - The path of the issuer of the module request.
+   *
+   * @returns A semver-mangled path (e.g. 'foo@^1.0.0/lib/bar/baz.js')
+   *
+   * #### Notes
+   * Files in the same package are locked to the exact version number
+   * of the package. Files in local packages (i.e., `file://` packages) are
+   * allowed to vary by patch number (the `~` semver range specifier is added).
+   */
+  function getModuleSemverPath(request: string, issuer: string): string {
+    let rootPath = findRoot(request);
+    let rootPackage = getPackage(rootPath);
+    let issuerPackage = getPackage(issuer);
+    let modPath = request.slice(rootPath.length + 1);
+    let name = rootPackage.name;
+    let semver = ((issuerPackage.dependencies &&
+                   issuerPackage.dependencies[name]) || rootPackage.version);
+    if (issuerPackage.name === rootPackage.name) {
+      semver = `${rootPackage.version}`;
+    } else if (semver.indexOf('file:') === 0) {
+      let sourcePath = path.resolve(rootPath, semver.slice('file:'.length));
+      let sourcePackage = getPackage(sourcePath);
+      // Allow patch version increments of local packages.
+      semver = `~${sourcePackage.version}`;
+    }
+
+    let id = `${name}@${semver}`;
+    if (modPath) {
+      modPath = modPath.split(path.sep).join('/');
+      id += `/${modPath}`;
+    }
+    return id;
+  }
+
+  /**
+   * Generate a context module given a mapping and an id.
+   */
+  function generateContextModule(mapString: string, id: string) {
+    return `
+      var map = ${mapString};
+      function webpackContext(req) {
+        return __webpack_require__(webpackContextResolve(req));
+      };
+      function webpackContextResolve(req) {
+        return map[req] || (function() { throw new Error("Cannot find module '" + req + "'.") }());
+      };
+      webpackContext.keys = function webpackContextKeys() {
+        return Object.keys(map);
+      };
+      webpackContext.resolve = webpackContextResolve;
+      module.exports = webpackContext;
+      webpackContext.id = "${id}";
+    `;
+  }
+}

packages/extension-builder/src/tsconfig.json

@@ -0,0 +1,13 @@
+{
+  "compilerOptions": {
+    "declaration": true,
+    "noImplicitAny": true,
+    "noEmitOnError": true,
+    "lib": ["dom", "es5", "es2015.promise"],
+    "module": "commonjs",
+    "moduleResolution": "node",
+    "target": "ES5",
+    "outDir": "../lib"
+  },
+  "exclude": ["typedoc.d.ts"]
+}

packages/extension-builder/src/typedoc.d.ts

@@ -0,0 +1,11 @@
+// Copyright (c) Jupyter Development Team.
+// Distributed under the terms of the Modified BSD License.
+
+/*
+ * TODO: remove this file and the excludes entry in tsconfig.json
+ * after typedoc understands the lib compiler option and @types packages.
+ */
+/// <reference types="node"/>
+/// <reference path="../node_modules/typescript/lib/lib.dom.d.ts"/>
+/// <reference path="../node_modules/typescript/lib/lib.es5.d.ts"/>
+/// <reference path="../node_modules/typescript/lib/lib.es2015.promise.d.ts"/>

packages/extension-builder/src/typings.d.ts

@@ -0,0 +1 @@
+/// <reference path="../typings/webpack-config/webpack-config.d.ts"/>

packages/extension-builder/tsconfig.json

@@ -0,0 +1,15 @@
+{
+  "compilerOptions": {
+    "declaration": true,
+    "noImplicitAny": true,
+    "noEmitOnError": true,
+    "noUnusedLocals": true,
+    "module": "commonjs",
+    "moduleResolution": "node",
+    "target": "ES5",
+    "outDir": "./lib",
+    "lib": ["ES5", "ES2015.Promise", "DOM"],
+    "types": []
+  },
+  "include": ["src/*"]
+}

packages/extension-builder/typings/webpack-config/webpack-config.d.ts

@@ -0,0 +1,9 @@
+// Type definitions for webpack-config
+// Project: https://github.com/mdreizin/webpack-config
+// Definitions by: Steven Silvester <https://github.com/blink1073>
+
+declare module 'webpack-config' {
+  export class Config {
+    merge(value: any): any;
+  }
+}