I am enabling Athena to query on Cloudtrail s3 logs using Terraform. To do this, I need to create database and tables in Glue Catalog. I am following this link.
In Terraform I am using aws_glue_catalog_table resource
.
How can I define columns with type struct and Array in terraform file?
I tried defining below ways but did not work.
resource "aws_glue_catalog_database" "cloud_logs" {
name = "trail_logs_db"
}
resource "aws_glue_catalog_table" "cloud_table" {
name = "trail_logs"
database_name = "${aws_glue_catalog_database.cloud_logs.name}"
table_type = "EXTERNAL_TABLE"
parameters = {
EXTERNAL = "TRUE"
}
storage_descriptor {
location = "s3://<BUCKET NAME>/AWSLogs/<AWS ACCOUNT ID>/"
input_format = "com.amazon.emr.cloudtrail.CloudTrailInputFormat"
output_format = "org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat"
ser_de_info {
name = "trail-logs"
serialization_library = "com.amazon.emr.hive.serde.CloudTrailSerde"
parameters {
serialization.format = 1
}
}
columns = [
{
name = "useridentity"
type = "struct<type:string,
principalid:string,
arn:string,
accountid:string,
invokedby:string,
accesskeyid:string,
userName:string,>"
comment = ""
},
{
name = "resources"
type = "array<STRUCT<ARN:string,
accountId:string,
type:string>>"
comment = ""
},
]
}
}
WHen I run terraform init
It throws below error:
Error: Error parsing test.tf: At 33:27: illegal char