Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: add documentation for ADMIN ALTER DDL JOBS command #19085

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
+114 −23
Open

docs: add documentation for ADMIN ALTER DDL JOBS command #19085

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8b32418
docs: add documentation for ADMIN ALTER DDL JOBS command
tangenta Nov 15, 2024
db58424
Update sql-statement-admin-alter-ddl.md
hfxsd Nov 15, 2024
8ba0b7e
Update sql-statements/sql-statement-admin-alter-ddl.md
tangenta Nov 15, 2024
ee66331
Update sql-statements/sql-statement-admin-alter-ddl.md
tangenta Nov 15, 2024
3187695
docs: clarify parameter value range consistency with system variables…
tangenta Nov 15, 2024
c91a9da
Update sql-statements/sql-statement-admin-alter-ddl.md
hfxsd Nov 18, 2024
540be02
Update sql-statements/sql-statement-admin-alter-ddl.md
tangenta Nov 20, 2024
32ad9c1
update admin show ddl jobs doc
tangenta Nov 22, 2024
6d99637
Update TOC.md
hfxsd Nov 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions TOC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -736,6 +736,7 @@
- SQL 语句
- [概览](/sql-statements/sql-statement-overview.md)
- [`ADMIN`](/sql-statements/sql-statement-admin.md)
- [`ADMIN ALTER DDL JOBS`](sql-statements/sql-statement-admin-alter-ddl.md)
- [`ADMIN CANCEL DDL`](/sql-statements/sql-statement-admin-cancel-ddl.md)
- [`ADMIN CHECKSUM TABLE`](/sql-statements/sql-statement-admin-checksum-table.md)
- [`ADMIN CHECK [TABLE|INDEX]`](/sql-statements/sql-statement-admin-check-table-index.md)
Expand Down
81 changes: 81 additions & 0 deletions sql-statements/sql-statement-admin-alter-ddl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
title: ADMIN ALTER DDL JOBS
summary: TiDB 数据库中 `ADMIN ALTER DDL JOBS` 的使用概况。
---

# ADMIN ALTER DDL JOBS

`ADMIN ALTER DDL JOBS` 语句可用于修改单个正在运行的 DDL 作业相关的参数。例如：

```sql
ADMIN ALTER DDL JOBS 101 THREAD = 8;
```

其中 `101` 表示 DDL 作业的 ID，该 ID 可以通过查询 [`ADMIN SHOW DDL JOBS`](/sql-statements/sql-statement-admin-show-ddl.md) 获得。`THREAD` 表示当前 DDL 作业的并发度，其初始值由系统变量 [`tidb_ddl_reorg_worker_cnt`](/system-variables.md#tidb_ddl_reorg_worker_cnt) 设置。

目前支持 `ADMIN ALTER DDL JOBS` 的 DDL 作业类型包括：`ADD INDEX`、`MODIFY COLUMN` 和 `REORGANIZE PARTITION`。对于其他 DDL 作业类型，执行 `ADMIN ALTER DDL JOBS` 会报 `unsupported DDL operation` 的错误。

tangenta marked this conversation as resolved.
Show resolved Hide resolved
目前在一条 `ADMIN ALTER DDL JOBS` 中仅支持对单个 DDL 作业的参数进行调整，不支持同时调整多个 ID 对应的参数。
以下是不同 DDL 作业类型支持的各项参数，以及它们对应的系统变量：
tangenta marked this conversation as resolved.
Show resolved Hide resolved

- `ADD INDEX`:
- `THREAD`: 并发度，初始值由系统变量 `tidb_ddl_reorg_worker_cnt` 设置。
- `BATCH_SIZE`: 批大小，初始值由系统变量 [`tidb_ddl_reorg_batch_size`](/system-variables.md#tidb_ddl_reorg_batch_size) 设置。
- `MAX_WRITE_SPEED`: 向每个 TiKV 导入索引记录时的最大带宽限制，初始值由系统变量 [`tidb_ddl_reorg_max_write_speed`](/system-variables.md#tidb_ddl_reorg_max_write_speed-从-v850-版本开始引入) 设置。
Copy link
Collaborator

@hfxsd hfxsd Nov 18, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reviewer: 变量 tidb_ddl_reorg_max_write_speed 在这个 PR 中：pingcap/docs#19445


- `MODIFY COLUMN`:
- `THREAD`: 并发度，初始值由系统变量 `tidb_ddl_reorg_worker_cnt` 设置。
- `BATCH_SIZE`: 批大小，初始值由系统变量 `tidb_ddl_reorg_batch_size` 设置。

- `REORGANIZE PARTITION`:
- `THREAD`: 并发度，初始值由系统变量 `tidb_ddl_reorg_worker_cnt` 设置。
- `BATCH_SIZE`: 批大小，初始值由系统变量 `tidb_ddl_reorg_batch_size` 设置。

参数的取值范围和系统变量的保持一致。

`ADMIN ALTER DDL JOBS` 仅对正在运行的 DDL 作业生效。如果 DDL 作业不存在或者已经结束，执行该语句会报 `ddl job ? is not running` 的错误。

以下是部分语句示例：

```sql
ADMIN ALTER DDL JOBS 101 THREAD = 8;
ADMIN ALTER DDL JOBS 101 BATCH_SIZE = 256;
ADMIN ALTER DDL JOBS 101 MAX_WRITE_SPEED = '200MiB';
ADMIN ALTER DDL JOBS 101 THREAD = 8, BATCH_SIZE = 256;
```

要查看某个 DDL 作业当前的参数值，可以执行 `ADMIN SHOW DDL JOBS`，结果显示在 `COMMENTS` 列：

```sql
mysql> admin show ddl jobs 1;
+--------+---------+------------+-----------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+-----------------------+
| JOB_ID | DB_NAME | TABLE_NAME | JOB_TYPE | SCHEMA_STATE | SCHEMA_ID | TABLE_ID | ROW_COUNT | CREATE_TIME | START_TIME | END_TIME | STATE | COMMENTS |
+--------+---------+------------+-----------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+-----------------------+
| 124 | test | t | add index | public | 2 | 122 | 3 | 2024-11-15 11:17:06.213000 | 2024-11-15 11:17:06.213000 | 2024-11-15 11:17:08.363000 | synced | ingest, DXF, thread=8 |
+--------+---------+------------+-----------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+-----------------------+
1 row in set (0.01 sec)
```

## 语法图

```ebnf+diagram
AdminPauseDDLStmt ::=
'ADMIN' 'ALTER' 'DDL' 'JOBS' Int64Num AlterJobOptionList

AlterJobOptionList ::=
AlterJobOption ( ',' AlterJobOption )*

AlterJobOption ::=
identifier "=" SignedLiteral
```

## MySQL 兼容性

`ADMIN ALTER DDL JOBS` 语句是 TiDB 对 MySQL 语法的扩展。

## 另请参阅

* [`ADMIN SHOW DDL [JOBS|QUERIES]`](/sql-statements/sql-statement-admin-show-ddl.md)
* [`ADMIN CANCEL DDL`](/sql-statements/sql-statement-admin-cancel-ddl.md)
* [`ADMIN PAUSE DDL`](/sql-statements/sql-statement-admin-pause-ddl.md)
* [`ADMIN RESUME DDL`](/sql-statements/sql-statement-admin-resume-ddl.md)
55 changes: 32 additions & 23 deletions sql-statements/sql-statement-admin-show-ddl.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ OWNER_ADDRESS: 0.0.0.0:4000

### `ADMIN SHOW DDL JOBS`

`ADMIN SHOW DDL JOBS` 语句用于查看当前 DDL 作业队列中的所有结果（包括正在运行以及等待运行的任务）以及已执行完成的 DDL 作业队列中的最近十条结果。该语句的返回结果字段描述如下：
`ADMIN SHOW DDL JOBS` 语句用于查看当前 DDL 作业队列中的所有结果（包括正在运行以及等待运行的任务）以及已执行完成的 DDL 作业队列中的最近的结果。该语句的返回结果字段描述如下：

- `JOB_ID`：每个 DDL 操作对应一个 DDL 任务，`JOB_ID` 全局唯一。
- `DB_NAME`：执行 DDL 操作的数据库的名称。
Expand All @@ -64,9 +64,7 @@ OWNER_ADDRESS: 0.0.0.0:4000
- `create schema`：[`CREATE SCHEMA`](/sql-statements/sql-statement-create-database.md) 操作。
- `create table`：[`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) 操作。
- `create view`：[`CREATE VIEW`](/sql-statements/sql-statement-create-view.md) 操作。
- `ingest`：通过 [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-从-v630-版本开始引入) 配置的加速索引回填的 ingest 任务。
- `txn`：基本的事务性回填。
- `add index /* txn-merge */`：在回填完成时，将临时索引与原始索引合并的事务性回填。
- `add index`: [`ADD INDEX`](/sql-statements/sql-statement-add-index.md) 操作。
- `SCHEMA_STATE`：DDL 所操作的 schema 对象的当前状态。如果 `JOB_TYPE` 是 `ADD INDEX`，则为索引的状态；如果是 `ADD COLUMN`，则为列的状态；如果是 `CREATE TABLE`，则为表的状态。常见的状态有以下几种：
- `none`：表示不存在。一般 `DROP` 操作或者 `CREATE` 操作失败回滚后，会变为 `none` 状态。
- `delete only`、`write only`、`delete reorganization`、`write reorganization`：这四种状态是中间状态，具体含义请参考 [TiDB 中在线 DDL 异步变更的原理](/ddl-introduction.md#tidb-在线-ddl-异步变更的原理)。由于中间状态转换很快，一般操作中看不到这几种状态，只有执行 `ADD INDEX` 操作时能看到处于 `write reorganization` 状态，表示正在添加索引数据。
Expand All @@ -89,6 +87,15 @@ OWNER_ADDRESS: 0.0.0.0:4000
- `pausing`：表示正在暂停该操作。
- `paused`：表示 DDL 已被暂停运行。这个状态只有在用 [`ADMIN PAUSED DDL JOBS`](/sql-statements/sql-statement-admin-pause-ddl.md) 命令暂停 DDL 任务时才会出现。可以通过 [`ADMIN RESUME DDL JOBS`](/sql-statements/sql-statement-admin-resume-ddl.md) 命令进行恢复运行。
- `done`：表示该操作在 TiDB owner 节点已经执行成功，但其他 TiDB 节点还没有同步该 DDL 任务所执行的变更。
- `COMMENTS`：包含其他辅助诊断用的信息。
- `ingest`：通过 [`tidb_ddl_enable_fast_reorg`](/system-variables.md#tidb_ddl_enable_fast_reorg-从-v630-版本开始引入) 配置的加速索引回填的 ingest 任务。
- `txn`：基本的事务性回填。
- `txn-merge`：在回填完成时，将临时索引与原始索引合并的事务性回填。
- `DXF`：通过 [`tidb_enable_dist_task`](/system-variables.md#tidb_enable_dist_task-从-v710-版本开始引入) 配置的用分布式框架执行的任务。
- `service_scope`：通过 [`tidb_service_scope`](/system-variables.md#tidb_service_scope-从-v740-版本开始引入) 配置的 TiDB 节点的服务范围。
- `thread`：回填型任务的并发度，可通过 `tidb_ddl_reorg_worker_cnt` 设置初始值。支持 `ADMIN ALTER DDL JOBS` 动态修改。
- `batch_size`：回填性任务的批大小，可通过 `tidb_ddl_reorg_batch_size` 设置初始值。支持 `ADMIN ALTER DDL JOBS` 动态修改。
Comment on lines +96 to +97
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

统一下 回填型任务 和 回填性任务

- `max_write_speed`：ingest 任务导入过程中的流量控制，可通过 `tidb_ddl_reorg_max_write_speed` 设置初始值。支持 `ADMIN ALTER DDL JOBS` 动态修改。
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `max_write_speed`：ingest 任务导入过程中的流量控制，可通过 `tidb_ddl_reorg_max_write_speed` 设置初始值。支持 `ADMIN ALTER DDL JOBS` 动态修改。
- `max_write_speed`：ingest 任务导入过程中的流量控制，可通过 `tidb_ddl_reorg_max_write_speed` 设置初始值。支持 [`ADMIN ALTER DDL JOBS`](sql-statements/sql-statement-admin-alter-ddl.md) 动态修改。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

上面这些都要吗

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

有个链接，方便用户参考，还可以提升文档 SEO。


示例如下：

Expand All @@ -97,30 +104,32 @@ ADMIN SHOW DDL JOBS;
```

```sql
+--------+---------+--------------------+--------------+----------------------+-----------+----------+-----------+-----------------------------------------------------------------+---------+
| JOB_ID | DB_NAME | TABLE_NAME | JOB_TYPE | SCHEMA_STATE | SCHEMA_ID | TABLE_ID | ROW_COUNT | CREATE_TIME | START_TIME | END_TIME | STATE |
+--------+---------+--------------------+--------------+----------------------+-----------+----------+-----------+---------------------+-------------------------------------------+---------+
| 59 | test | t1 | add index | write reorganization | 1 | 55 | 88576 | 2020-08-17 07:51:58 | 2020-08-17 07:51:58 | NULL | running |
| 60 | test | t2 | add index | none | 1 | 57 | 0 | 2020-08-17 07:51:59 | 2020-08-17 07:51:59 | NULL | none |
| 58 | test | t2 | create table | public | 1 | 57 | 0 | 2020-08-17 07:41:28 | 2020-08-17 07:41:28 | 2020-08-17 07:41:28 | synced |
| 56 | test | t1 | create table | public | 1 | 55 | 0 | 2020-08-17 07:41:02 | 2020-08-17 07:41:02 | 2020-08-17 07:41:02 | synced |
| 54 | test | t1 | drop table | none | 1 | 50 | 0 | 2020-08-17 07:41:02 | 2020-08-17 07:41:02 | 2020-08-17 07:41:02 | synced |
| 53 | test | t1 | drop index | none | 1 | 50 | 0 | 2020-08-17 07:35:44 | 2020-08-17 07:35:44 | 2020-08-17 07:35:44 | synced |
| 52 | test | t1 | add index | public | 1 | 50 | 451010 | 2020-08-17 07:34:43 | 2020-08-17 07:34:43 | 2020-08-17 07:35:16 | synced |
| 51 | test | t1 | create table | public | 1 | 50 | 0 | 2020-08-17 07:34:02 | 2020-08-17 07:34:02 | 2020-08-17 07:34:02 | synced |
| 49 | test | t1 | drop table | none | 1 | 47 | 0 | 2020-08-17 07:34:02 | 2020-08-17 07:34:02 | 2020-08-17 07:34:02 | synced |
| 48 | test | t1 | create table | public | 1 | 47 | 0 | 2020-08-17 07:33:37 | 2020-08-17 07:33:37 | 2020-08-17 07:33:37 | synced |
| 46 | mysql | stats_extended | create table | public | 3 | 45 | 0 | 2020-08-17 06:42:38 | 2020-08-17 06:42:38 | 2020-08-17 06:42:38 | synced |
| 44 | mysql | opt_rule_blacklist | create table | public | 3 | 43 | 0 | 2020-08-17 06:42:38 | 2020-08-17 06:42:38 | 2020-08-17 06:42:38 | synced |
+--------+---------+--------------------+--------------+----------------------+-----------+----------+-----------+---------------------+---------------------+-------------------------------+
12 rows in set (0.00 sec)
+--------+---------+------------+---------------------------------+----------------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+----------+-------------+
| JOB_ID | DB_NAME | TABLE_NAME | JOB_TYPE | SCHEMA_STATE | SCHEMA_ID | TABLE_ID | ROW_COUNT | CREATE_TIME | START_TIME | END_TIME | STATE | COMMENTS |
+--------+---------+------------+---------------------------------+----------------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+----------+-------------+
| 565 | test | sbtest1 | add index | write reorganization | 554 | 556 | 0 | 2024-11-22 12:39:25.475000 | 2024-11-22 12:39:25.524000 | NULL | running | ingest, DXF |
| 566 | test | sbtest1 | add index | none | 554 | 556 | 0 | 2024-11-22 12:39:26.425000 | NULL | NULL | queueing | |
| 564 | test | sbtest1 | alter table multi-schema change | none | 554 | 556 | 0 | 2024-11-22 12:39:02.925000 | 2024-11-22 12:39:02.925000 | 2024-11-22 12:39:03.275000 | synced | |
| 564 | test | sbtest1 | drop index /* subjob */ | none | 554 | 556 | 0 | 2024-11-22 12:39:02.925000 | 2024-11-22 12:39:02.925000 | 2024-11-22 12:39:03.275000 | done | |
| 564 | test | sbtest1 | drop index /* subjob */ | none | 554 | 556 | 0 | 2024-11-22 12:39:02.925000 | 2024-11-22 12:39:02.975000 | 2024-11-22 12:39:03.275000 | done | |
| 563 | test | sbtest1 | modify column | public | 554 | 556 | 0 | 2024-11-22 12:38:35.624000 | 2024-11-22 12:38:35.624000 | 2024-11-22 12:38:35.674000 | synced | |
| 562 | test | sbtest1 | add index | public | 554 | 556 | 1580334 | 2024-11-22 12:36:58.471000 | 2024-11-22 12:37:05.271000 | 2024-11-22 12:37:13.374000 | synced | ingest, DXF |
| 561 | test | sbtest1 | add index | public | 554 | 556 | 1580334 | 2024-11-22 12:36:57.771000 | 2024-11-22 12:36:57.771000 | 2024-11-22 12:37:04.671000 | synced | ingest, DXF |
| 560 | test | sbtest1 | add index | public | 554 | 556 | 1580334 | 2024-11-22 12:34:53.314000 | 2024-11-22 12:34:53.314000 | 2024-11-22 12:34:57.114000 | synced | ingest |
| 559 | test | sbtest1 | drop index | none | 554 | 556 | 0 | 2024-11-22 12:34:43.565000 | 2024-11-22 12:34:43.565000 | 2024-11-22 12:34:43.764000 | synced | |
| 558 | test | sbtest1 | add index | public | 554 | 556 | 1580334 | 2024-11-22 12:34:06.215000 | 2024-11-22 12:34:06.215000 | 2024-11-22 12:34:14.314000 | synced | ingest, DXF |
| 557 | test | sbtest1 | create table | public | 554 | 556 | 0 | 2024-11-22 12:32:09.515000 | 2024-11-22 12:32:09.915000 | 2024-11-22 12:32:10.015000 | synced | |
| 555 | test | | create schema | public | 554 | 0 | 0 | 2024-11-22 12:31:51.215000 | 2024-11-22 12:31:51.264000 | 2024-11-22 12:31:51.264000 | synced | |
| 553 | test | | drop schema | none | 2 | 0 | 0 | 2024-11-22 12:31:48.615000 | 2024-11-22 12:31:48.615000 | 2024-11-22 12:31:48.865000 | synced | |
+--------+---------+------------+---------------------------------+----------------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+----------+-------------+
14 rows in set (0.00 sec)
```

由上述 `ADMIN` 查询结果可知：

- `job_id` 为 59 的 DDL 作业当前正在进行中（`STATE` 列显示为 `running`）。`SCHEMA_STATE` 列显示了表当前处于 `write reorganization` 状态，一旦任务完成，将更改为 `public`，以便用户会话可以公开观察到状态变更。`end_time` 列显示为 `NULL`，表明当前作业的完成时间未知。
- `job_id` 为 565 的 DDL 作业当前正在进行中（`STATE` 列显示为 `running`）。`SCHEMA_STATE` 列显示了表当前处于 `write reorganization` 状态，一旦任务完成，将更改为 `public`，以便用户会话可以公开观察到状态变更。`end_time` 列显示为 `NULL`，表明当前作业的完成时间未知。

- `job_id` 为 60 的 `JOB_TYPE` 显示为 `add index`，表明正在排队等待 `job_id` 为 59 的作业完成。当作业 59 完成时，作业 60 的 `STATE` 将更改为 `running`。
- `job_id` 为 566 的 `STATE` 显示为 `queueing`，表明它正在排队等待。当作业 565 完成后，作业 566 开始执行时，作业 566 的 `STATE` 将更改为 `running`。

- 对于破坏性的更改（例如删除索引或删除表），当作业完成时，`SCHEMA_STATE` 将变为 `none`。对于附加更改，`SCHEMA_STATE` 将变为 `public`。

Expand Down
Loading