I am working over poc in which I need to create dataframe and then save it as ctrl A delimited file. My query to create intermediate result is below
val grouped = results.groupBy("club_data","student_id_add","student_id").agg(sum(results("amount").cast(IntegerType)).as("amount"),count("amount").as("cnt")).filter((length(trim($"student_id")) > 1) && ($"student_id").isNotNull)
Saving result in text file
grouped.select($"club_data", $"student_id_add", $"amount",$"cnt").rdd.saveAsTextFile("/amit/spark/output4/")
Output :
[amit,DI^A356035,581,1]
It saves data as comma separated but I need to save it as ctrl-A separate I tried option("delimiter", "\u0001") but seems it's not supported by dataframe/rdd.
Is there any function which helps?
option("delimiter", "\\01")
– philantrovert Jan 3 '18 at 13:02