3

Is there a way to output the AWS cli with filters to csv format?

For example if I wanted to take this command and output to csv:

aws ec2 describe-images --owner self   --query 'Images[*].{ID:ImageId,"Virtualization Type":VirtualizationType}'

How would I do that? Normally I would use jq to output the aws cli to csv. But in this case I was able to get to the info I wanted easier with the filter option instead of jq.

This is the full command I want to output to CSV:

aws ec2 describe-images --owner self   --query 'Images[*].{ID:ImageId,"Virtualization Type":VirtualizationType,Architechture:Architecture,Hypervisor:Hypervisor,State:State,ImageID:ImageId,"Device Names":BlockDeviceMappings[].DeviceName,"Snapshot IDs":BlockDeviceMappings[].Ebs.SnapshotId,"Delete On Termination":BlockDeviceMappings[].Ebs.DeleteOnTermination,"Voluem Type":BlockDeviceMappings[].Ebs.VolumeType,"Volume Size":BlockDeviceMappings[].Ebs.VolumeSize,Encrypted:BlockDeviceMappings[].Ebs.Encrypted,"Image Location":ImageLocation,"Root Device Type":RootDeviceType,"Owner ID":OwnerId,"Creation Date":CreationDate,Public:Public,"Image Type":ImageType,Name:Name}'

2 Answers 2

6

One solution I can think of is to output in text --output text and then replace the spaces with a comma:

aws ec2 describe-images --owner self   --query 'Images[*].{ID:ImageId,"Virtualization Type":VirtualizationType}' --output text

Output

ami-1234567890    hvm
ami-1a2b3c4d5e    hvm
ami-9876543210    hvm

Replace the blanks with a comma. There are many ways to do this using sed or tr or awk or paste.

aws ec2 describe-images --owner self   --query 'Images[*].{ID:ImageId,"Virtualization Type":VirtualizationType}' --output text | sed -E 's/\s+/,/g'

Output

ami-1234567890,hvm
ami-1a2b3c4d5e,hvm
ami-9876543210,hvm
2
  • This will output incorrect csv if a column value has a space in it. For example aws --region us-east-1 ec2 describe-instance-types --query 'InstanceTypes[?GpuInfo.Gpus!=null].[InstanceType, GpuInfo.Gpus[0].Count, GpuInfo.Gpus[0].Manufacturer, GpuInfo.Gpus[0].MemoryInfo.SizeInMiB, GpuInfo.Gpus[0].Name]' --output text outputs bash g5.4xlarge 1 NVIDIA 24576 A10G ... g4ad.8xlarge 2 AMD 8192 Radeon Pro V520 g4ad.4xlarge 1 AMD 8192 Radeon Pro V520 Your script would output g4ad.4xlarge,1,AMD,8192,Radeon,Pro,V520 instead of g4ad.4xlarge,1,AMD,8192,Radeon Pro V520
    – Anshul
    Jan 4, 2022 at 23:47
  • @Anshul I agree but the question was for AMI and Virtualization Type which do not have space in them. You can easily to refine the command to achieve what you need.
    – helloV
    Jan 4, 2022 at 23:50
4

We had to do something very similar today and we used jq to get the csv output natively.

aws ec2 describe-instances --output json --query 'foo' | \
  jq -r '.[][] | @csv'
  • Note: the .[][] notation was required for us because our query was reformatting the output in a specific way. I would NOT expect most cases will require referencing the output in the same way.

  • Note 2: I don't recall the original question saying "without jq", so while my cli works, I now realize it does not meet the OP goal.

2
  • Thanks, I updated the command to reflect that.
    – Todd Lyons
    Dec 27, 2021 at 16:07
  • Perfect, I'll remove the comment as it might be confusing otherwise.
    – Weeble
    Dec 27, 2021 at 16:14

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.