aihub-dag

JupyterLab DAG 插件

该组件为 JupyterLab 提供了一个 DAG 的可视化编辑组件, 能够将 Notebook 或 Python 脚本连接形成工作流, 并基于 Airflow 做定时调度执行. 本项目 fork 自 https://github.com/elyra-ai/elyra, 在官方代码的基础上进行二次开发.

构建及安装方法

make install

配置文件

配置文件路径:

<sys-prefix>/share/jupyter/metadata/runtimes/<文件名>.json

对应的 JupyterHub Docker 镜像的挂载路径:

/usr/local/share/jupyter/metadata/runtimes/yili-test.json

配置文件样例 (yili-test.json):

{
  "display_name": "yili-test",
  "metadata": {
    "tags": [],
    "display_name": "yili-test",
    "user_namespace": "airflow01",
    "git_type": "GITLAB",
    "github_api_endpoint": "http://XXXX/",
    "api_endpoint": "http://XXXX/",
    "github_repo": "repo/dags",
    "github_branch": "master",
    "github_repo_token": "XXXXXXXXXXXX",
    "cos_auth_type": "USER_CREDENTIALS",
    "cos_endpoint": "http://XXXX/",
    "cos_bucket": "mybucket",
    "cos_username": "local",
    "cos_password": "local1234",
    "runtime_type": "APACHE_AIRFLOW"
  },
  "schema_name": "airflow"
}