I have a Dataframe with one column. Each row of that column has an Array of String values:
Values in my Spark 2.2 Dataframe
["123", "abc", "2017", "ABC"]
["456", "def", "2001", "ABC"]
["789", "ghi", "2017", "DEF"]
org.apache.spark.sql.DataFrame = [col: array]
root
|-- col: array (nullable = true)
| |-- element: string (containsNull = true)
What is the best way to access elements in the array? For example, I would like extract distinct values in the fourth element for the year 2017 (answer "ABC", "DEF").