Saturday, March 21, 2020

Window function in spark

Case - If you have multiple category in each state and want to find out the max age of people in each category within state. Then use below window function

 df.withColumn("mx",max('age').over(Window.partitionBy('cat','state'))).show()

No comments:

Post a Comment