1

With the below code I am getting an error message, name 'when' is not defined.

voter_df = voter_df.withColumn('random_val',
when(voter_df.TITLE == 'Councilmember', F.rand())
.when(voter_df.TITLE == 'Mayor', 2)
.otherwise(0))

Add a column to voter_df named random_val with the results of the F.rand() method for any voter with the title Councilmember. Set random_val to 2 for the Mayor. Set any other title to the value 0

4
  • Please provide the entire error message, as well as a minimal reproducible example.
    – AMC
    Commented Mar 9, 2020 at 2:27
  • Error: Add a column to voter_df named random_val with the results of the F.rand() method for any voter with the title Councilmember. Set random_val to 2 for the Mayor. Set any other title to the value 0 Commented Mar 9, 2020 at 3:45
  • What is that? We’re still missing the minimal reproducible example, by the way.
    – AMC
    Commented Mar 9, 2020 at 12:21
  • Might be unrelated but could help someone, from pyspark.sql.functions import * Commented Dec 25, 2021 at 3:24

2 Answers 2

3

The second when statement is a method of dataframe, but the first when statement is not.

Solution: use ....'random_val',F.when(....

0
1

use from pyspark.sql.functions import when

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.