Thursday, September 17, 2020

Add new column in exiting Dataframe using pyspark

 from pyspark.sql.functions import *

(1) Add Integer type column

       df3=df2.withColumn('column_name', lit(1))


(2) Add Double Type column

       df3=df2.withColumn('column_name', lit(1.0))


(3) Add StringType column

       df3=df2.withColumn('column_name', lit(""))


(3) Add BooleanType column

       df3=df2.withColumn('column_name', lit(True))

No comments:

Post a Comment