s3_* session settings

These settings are available in system.settings and are autogenerated from source.

s3_base

The base URL used to resolve relative URLs in the s3 table function and the S3 table engine, as well as in the table functions that share their configuration (s3Cluster, gcs, oss).

When set, a URL without a scheme is resolved against s3_base per RFC 3986, using the same rules as url_base. URLs that already contain a scheme are used as-is.

For example, if s3_base is s3://clickhouse-public-datasets/, then s3('hits_compatible/hits.csv') reads s3://clickhouse-public-datasets/hits_compatible/hits.csv.

The base URL can use any form accepted by the s3 table function, e.g. s3://bucket/, https://bucket.s3.amazonaws.com/ or https://endpoint/bucket/.

s3_check_objects_after_upload

Check each uploaded object to s3 with head request to be sure that upload was successful

s3_connect_timeout_ms

Connection timeout for host from s3 disks.

s3_create_new_file_on_insert

Enables or disables creating a new file on each insert in s3 engine tables. If enabled, on each insert a new S3 object will be created with the key, similar to this pattern:

initial: data.Parquet.gz -> data.1.Parquet.gz -> data.2.Parquet.gz, etc.

Possible values:

• 0 — INSERT query creates a new file or fail if file exists and s3_truncate_on_insert is not set.
• 1 — INSERT query creates a new file on each insert using suffix (from the second one) if s3_truncate_on_insert is not set.

See more details here.

s3_disable_checksum

Do not calculate a checksum when sending a file to S3. This speeds up writes by avoiding excessive processing passes on a file. It is mostly safe as the data of MergeTree tables is checksummed by ClickHouse anyway, and when S3 is accessed with HTTPS, the TLS layer already provides integrity while transferring through the network. While additional checksums on S3 give defense in depth.

s3_ignore_file_doesnt_exist

Ignore absence of file if it does not exist when reading certain keys.

Possible values:

• 1 — SELECT returns empty result.
• 0 — SELECT throws an exception.

s3_list_object_keys_size

Maximum number of files that could be returned in batch by ListObject request

s3_min_upload_part_size

The minimum size of part to upload during multipart upload to S3.

s3_path_filter_limit

Maximum number of _path values that can be extracted from query filters to use for file iteration instead of glob listing. 0 means disabled.

s3_request_timeout_ms

Idleness timeout for sending and receiving data to/from S3. Fail if a single TCP read or write call blocks for this long.

s3_skip_empty_files

Enables or disables skipping empty files in S3 engine tables.

Possible values:

• 0 — SELECT throws an exception if empty file is not compatible with requested format.
• 1 — SELECT returns empty result for empty file.

s3_slow_all_threads_after_network_error

When set to true, all threads executing S3 requests to the same backup endpoint are slowed down after any single s3 request encounters a retryable network error, such as socket timeout. When set to false, each thread handles S3 request backoff independently of the others.

s3_strict_upload_part_size

The exact size of part to upload during multipart upload to S3 (some implementations does not supports variable size parts).

s3_throw_on_zero_files_match

Throw an error, when ListObjects request cannot match any files

s3_truncate_on_insert

Enables or disables truncate before inserts in s3 engine tables. If disabled, an exception will be thrown on insert attempts if an S3 object already exists.

Possible values:

• 0 — INSERT query creates a new file or fail if file exists and s3_create_new_file_on_insert is not set.
• 1 — INSERT query replaces existing content of the file with the new data.

See more details here.

s3_uri_style

Force the s3 endpoint style. Possible values: auto, virtual_hosted, path.

s3_use_adaptive_timeouts

When set to true than for all s3 requests first two attempts are made with low send and receive timeouts. When set to false than all attempts are made with identical timeouts.