Key Default Type Description
legacy-timestamp-mapping.enabled
false Boolean If true, map Paimon TIMESTAMP to Spark TIMESTAMP instead of TIMESTAMP_NTZ.
read.allow.fullScan
true Boolean Whether to allow full scan when reading a partitioned table.
read.changelog
false Boolean Whether to read row in the form of changelog (add rowkind column in row to represent its change type).
read.stream.maxBytesPerTrigger
(none) Long The maximum number of bytes returned in a single batch.
read.stream.maxFilesPerTrigger
(none) Integer The maximum number of files returned in a single batch.
read.stream.maxRowsPerTrigger
(none) Long The maximum number of rows returned in a single batch.
read.stream.maxTriggerDelayMs
(none) Long The maximum delay between two adjacent batches, which used to create MinRowsReadLimit with read.stream.minRowsPerTrigger together.
read.stream.minRowsPerTrigger
(none) Long The minimum number of rows returned in a single batch, which used to create MinRowsReadLimit with read.stream.maxTriggerDelayMs together.
requiredSparkConfsCheck.enabled
true Boolean Whether to verify SparkSession is initialized with required configurations.
source.split.target-size-with-column-pruning
false Boolean Whether to adjust the target split size based on pruned (projected) columns. If enabled, split size estimation uses only the columns actually being read.
vector-search.lateral-join.parallelism
16 Integer Parallelism used to repartition a single-partition LIMIT input before executing a lateral vector search.
write.data-evolution.update-conflict-retry.max-attempts
20 Integer Maximum attempts for Spark V1 UPDATE on data-evolution tables when concurrent partial-column updates conflict on the same row-id range and update columns. Values less than 2 disable retry.
write.data-evolution.update-conflict-retry.wait-ms
10 Long Wait time in milliseconds between retry attempts for Spark V1 UPDATE on data-evolution tables after row-id range update conflicts.
write.merge-schema
false Boolean If true, evolve the table schema to accept new columns from the incoming data. Existing column types are preserved and incoming values are cast to them; to also widen existing types, enable 'write.merge-schema.type-widening'.
write.merge-schema.explicit-cast
false Boolean Only effective when 'write.merge-schema.type-widening' is true. If true, also allow lossy type changes between compatible types (e.g. BIGINT -> INT, STRING -> DATE).
write.merge-schema.type-widening
false Boolean Only effective when 'write.merge-schema' is true. If true, widen an existing column type when the incoming data has a wider compatible type (e.g. INT -> BIGINT, DECIMAL precision increase). Lossy changes are still rejected unless 'write.merge-schema.explicit-cast' is also true.
write.use-v2-write
false Boolean If true, v2 write will be used. Currently, only HASH_FIXED and BUCKET_UNAWARE bucket modes are supported. Will fall back to v1 write for other bucket modes. Currently, Spark V2 write does not support TableCapability.STREAMING_WRITE.