pyspark.sql.functions.exists#

pyspark.sql.functions.exists(col, f)[source]#

Returns whether a predicate holds for one or more elements in the array.

New in version 3.1.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str: name of column or expression
ffunction: (x: Column) -> Column: ... returning the Boolean expression. Can use methods of Column, functions defined in pyspark.sql.functions and Scala UserDefinedFunctions. Python UserDefinedFunctions are not supported (SPARK-27052).

Returns

Column: True if “any” element of an array evaluates to True when passed as an argument to given function and False otherwise.

Examples

>>> df = spark.createDataFrame([(1, [1, 2, 3, 4]), (2, [3, -1, 0])],("key", "values"))
>>> df.select(exists("values", lambda x: x < 0).alias("any_negative")).show()
+------------+
|any_negative|
+------------+
|       false|
|        true|
+------------+