Skip to content
ClickHouse Docs
ClickHouse DocsClickHouse Docs

deduplicate_insert_* session settings

These settings are available in system.settings and are autogenerated from source.

deduplicate_insert

Type
DeduplicateInsertMode
Default
enable
Version history
VersionDefault valueComment
26.2enableEnable deduplication for all sync and async inserts by default.

Enables or disables block deduplication of INSERT INTO (for Replicated* tables). The setting applies to both synchronous and asynchronous inserts, and it supersedes the insert_deduplicate and async_insert_deduplicate settings. That setting has three possible values:

  • disable — Deduplication is disabled for INSERT INTO query.
  • enable — Deduplication is enabled for INSERT INTO query.
  • backward_compatible_choice — Deduplication is enabled if insert_deduplicate or async_insert_deduplicate are enabled for specific insert type.

For INSERT SELECT queries, deduplicate_insert_select is consulted first. Deduplication also requires the destination table to keep a deduplication log, see replicated_deduplication_window and non_replicated_deduplication_window.

deduplicate_insert_select

Type
DeduplicateInsertSelectMode
Default
enable_when_possible
Version history
VersionDefault valueComment
26.1enable_when_possiblechange the default behavior of deduplicate_insert_select to ENABLE_WHEN_POSSIBLE
25.12enable_even_for_bad_queriesNew setting, replace insert_select_deduplicate

Enables or disables block deduplication of INSERT SELECT (for Replicated* tables). For INSERT SELECT queries this setting is consulted before deduplicate_insert. A SELECT is stable when it carries ORDER BY ALL and its pipeline ends in a single stream. A non-empty insert_deduplication_token is an equivalent substitute for stability. That setting has four possible values:

  • disable — Deduplication is disabled for INSERT SELECT query.
  • force_enable — Deduplication is enabled for INSERT SELECT query, regardless of deduplicate_insert. If select result is not stable and no token is provided, exception is thrown.
  • enable_when_possible — Deduplication is enabled if it is enabled by deduplicate_insert and the select result is stable, otherwise disabled.
  • enable_even_for_bad_queries - Deduplication is enabled if it is enabled by deduplicate_insert, regardless of whether the select result is stable. If select result is not stable, a message is logged, but query is executed with deduplication. This option is for backward compatibility. Consider to use other options instead as it may lead to unexpected results.
Navigation