Please help get past this. I have to enter 3 logical ids to rollback my update, but the regex in cloudformation doesn't allow for that because the regex doesn't appear to allow commas. But the continue ROLLBACK instructions from AWS clearly state to use commas:
"To skip resources, type a list of comma-separated logical resource IDs. Include only the resources that are blocking the rollback."
But the commas are not allowed by the cloudformation regex???? What am I missing here? I can't go forward or back. My cloudformation stack is stuck.
My error on trying to roll back:
Failed to rollback: 1 validation error detected: Value '[rRoute10,rRoute192,rRoute172]' at 'resourcesToSkip' failed to satisfy constraint: Member must satisfy constraint: [Member must satisfy regular expression pattern: [a-zA-Z0-9]+|[a-zA-Z][-a-zA-Z0-9]*.[a-zA-Z0-9]+]
My errors in cloudformation:
14:06:04 UTC-0400 UPDATE_FAILED AWS::EC2::Route rRoute192 The gateway ID 'vgw-0e7d969e316a7b5d5' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidGatewayID.NotFound; Request ID: 6d4cdb5f-31c5-4cf3-8777-3e3eb361d594)
14:06:04 UTC-0400 UPDATE_FAILED AWS::EC2::Route rRoute10 The gateway ID 'vgw-0e7d969e316a7b5d5' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidGatewayID.NotFound; Request ID: 315ecc3a-d70c-46b7-b1cd-05f4c6765edd)
14:06:04 UTC-0400 UPDATE_FAILED AWS::EC2::Route rRoute172 The gateway ID 'vgw-0e7d969e316a7b5d5' does not exist (Service: AmazonEC2; Status Code: 400; Error Code: InvalidGatewayID.NotFound; Request ID: 2a6e20f0-4d41-4647-85f0-ffbfc0326680)
For background, someone deleted our VPG and created a different gateway. They also manually updated the routes. Now, I am trying to get the stack in sync on the route tables. However, the stack failed to update and now it fails to roll back.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
RDS
SAM Template for creating an RDS in a secure Fault-Tolerant fashion...
Parameters:
pDBName:
Default: MyDatabase
Description: The database name
Type: String
MinLength: '1'
MaxLength: '64'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: must begin with a letter and contain only alphanumeric characters.
pDBUser:
NoEcho: 'true'
Description: The database admin account username
Type: String
MinLength: '1'
MaxLength: '16'
AllowedPattern: '[a-zA-Z][a-zA-Z0-9]*'
ConstraintDescription: must begin with a letter and contain only alphanumeric
characters.
pDBPassword:
NoEcho: 'true'
Description: The database admin account password
Type: String
MinLength: '1'
MaxLength: '41'
AllowedPattern: '[a-zA-Z0-9]+'
ConstraintDescription: must contain only alphanumeric characters.
pDBAllocatedStorage:
Default: '20'
Description: The size of the database (Gb)
Type: Number
MinValue: '20'
MaxValue: '16384'
ConstraintDescription: must be between 5 and 1024Gb.
pDBInstanceClass:
Description: The database instance type
Type: String
Default: db.t2.micro
AllowedValues: [db.t1.micro, db.m1.small, db.m1.medium, db.m1.large, db.m1.xlarge,
db.m2.xlarge, db.m2.2xlarge, db.m2.4xlarge, db.m3.medium, db.m3.large, db.m3.xlarge,
db.m3.2xlarge, db.m4.large, db.m4.xlarge, db.m4.2xlarge, db.m4.4xlarge, db.m4.10xlarge,
db.r3.large, db.r3.xlarge, db.r3.2xlarge, db.r3.4xlarge, db.r3.8xlarge, db.m2.xlarge,
db.m2.2xlarge, db.m2.4xlarge, db.cr1.8xlarge, db.t2.micro, db.t2.small, db.t2.medium,
db.t2.large]
ConstraintDescription: must select a valid database instance type.
# TODO - Future DBEngine allowed values: [aurora-mysql, aurora-postgresql, mariadb, oracle-ee, oracle-se2, sqlserver-ee, sqlserver-se, sqlserver-ex, sqlserver-web]
pDBEngine:
Description: The database type to create
Type: String
Default: oracle-se2
AllowedValues: [aurora-mysql, oracle-ee, oracle-se2]
ConstraintDescription: must select a valid database engine
pDBEngineVersion:
Description: The version of database to create
Type: String
pMultiAZ:
Description: Multi-AZ master database
Type: String
Default: 'false'
AllowedValues: ['true', 'false']
ConstraintDescription: must be true or false.
pDatabaseDeletionProtection:
Description: Do we even allow a database to be deleted?
Type: String
Default: 'false'
AllowedValues: ['true', 'false']
ConstraintDescription: must be true or false.
pDatabaseLicenseModel:
Description: What licensing do we use?
Type: String
Default: "license-included"
AllowedValues: ["license-included", "bring-your-own-license", "general-public-license"]
ConstraintDescription: must be license-included or bring-your-own-license or general-public-license
pDBParameterGroupName:
Description: Use an existing parameter group in Amazon RDS - must match the database.
Type: String
Default: "default.aurora-mysql5.7"
pStorageType: # https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBInstance.html
Description: The type of storage to be used by the database.
Type: String
Default: 'standard'
AllowedValues: ['standard', 'gp2', 'io1']
pRDSVPC:
Description: The vpc to use?
Type: AWS::EC2::VPC::Id
pCreateNewSubnets:
Description: Set to true if we want new subnets for our databases
Type: String
Default: "true"
AllowedValues: ['true', 'false']
ConstraintDescription: must be true or false.
pExistingSubnetId1:
Description: Use an id of an existing private subnet
Type: String
Default: "subnet-0455e1cfa66facd17"
pExistingSubnetId2:
Description: Use an id of a second existing private subnet
Type: String
Default: "subnet-0536176832ba42dfd"
pSubnetCIDR1:
Description: The cidrblock to use?
Type: String
pSubnetCIDR2:
Description: The cidrblock to use?
Type: String
pGatewayID:
Description: The virtual private gateway id for the vpc, to manage our databases.
Type: String
Default: "vgw-0e7d969e316a7b5d5"
pOperatorEMail:
Description: EMail address to notify if there are any operational issues
Type: String
AllowedPattern: >-
([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\.)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)
ConstraintDescription: must be a valid email address.
# This next parameter is a hold-over from older pipelines. It must exist, or there will be a cloudformation error.
BuildBucket:
Description: Unused in this template
Type: String
Default: ""
# Warning, I haven't actually tested all these databases or these mappings - be ready to debug :-)
Mappings:
DBPortMap:
aurora-mysql:
port: 3306
aurora-postgresql:
port: 5432
mariadb:
port: 3306
mysql:
port: 3306
oracle-ee:
port: 1521
oracle-se2:
port: 1521
postgres:
port: 5432
sqlserver-ee:
port: 1433
sqlserver-se:
port: 1433
sqlserver-ex:
port: 1433
sqlserver-web:
port: 1433
Conditions:
CreateMultiAZ: !Equals [ !Ref pMultiAZ, "true" ]
CreateNewSubnets: !Equals [ !Ref pCreateNewSubnets, "true" ]
CreateAurora: !Or
- !Equals [ !Ref pDBEngine, "aurora-mysql" ]
- !Equals [ !Ref pDBEngine, "aurora-postgresql" ]
CreateNonAurora: !Not [Condition: CreateAurora ]
CreateAuroraMultiAZ: !And
- Condition: CreateAurora
- Condition: CreateMultiAZ
Resources:
# TODO - SECURE THE PASSWARD PARAMETERS IN AWS PARAMETER STORE
# See: https://github.com/aws-samples/aws-aurora-cloudformation-samples/blob/master/cftemplates/Aurora-Postgres-DB-Cluster.yml
# TODO - DBClusterParameterGroups and DBParameterGroups are not auto-created.
# Subnets
#
# Create one subnet if pCreateNewSubnets is true
# Create two new subnets if both pCreateNewSubnets and pMultiAZ are true...
# Otherwise, the user better have specified existing subnet(s) in pExistingSubnetId1 and pExistingSubnetId2
#
# Note that this template assumes we use Availability Zones 0 and 1. Perhaps I should make those parameters?
#
rDBSubnet1:
Type: AWS::EC2::Subnet
Condition: CreateNewSubnets
Properties:
VpcId: !Ref 'pRDSVPC'
CidrBlock: !Ref 'pSubnetCIDR1'
AvailabilityZone: !Select
- 0
- !GetAZs
Ref: 'AWS::Region'
Tags:
- Key: "Name"
Value: !Join [ "-", [!Ref "AWS::StackName", !Ref "AWS::Region", "RDS-AZA-SUBNET"] ]
rDBSubnet2:
Type: AWS::EC2::Subnet
Condition: CreateNewSubnets
Properties:
VpcId: !Ref 'pRDSVPC'
CidrBlock: !Ref 'pSubnetCIDR2'
AvailabilityZone: !Select
- 1
- !GetAZs
Ref: 'AWS::Region'
Tags:
- Key: "Name"
Value: !Join [ "-", [!Ref "AWS::StackName", !Ref "AWS::Region", "RDS-AZB-SUBNET"] ]
# Route Table
#
# If we create new subnets, we need to use a route table. I thought about reusing the existing route table for the
# primary private subnet, but this is a different class of traffic. I feel it is best practice to create new route tables.
#
rCustomRouteTable:
Type: AWS::EC2::RouteTable
Condition: CreateNewSubnets
Properties:
VpcId: !Ref 'pRDSVPC'
Tags:
- Key: "Name"
Value: !Join [ "-", [!Ref "AWS::StackName", !Ref "AWS::Region", "RTB-RDS"] ]
rRoute10:
Type: AWS::EC2::Route
Condition: CreateNewSubnets
Properties:
RouteTableId: !Ref rCustomRouteTable
DestinationCidrBlock: 10.0.0.0/8
GatewayId: !Ref pGatewayID
rRoute172:
Type: AWS::EC2::Route
Condition: CreateNewSubnets
Properties:
RouteTableId: !Ref rCustomRouteTable
DestinationCidrBlock: 172.16.0.0/12
GatewayId: !Ref pGatewayID
rRoute192:
Type: AWS::EC2::Route
Condition: CreateNewSubnets
Properties:
RouteTableId: !Ref rCustomRouteTable
DestinationCidrBlock: 192.168.0.0/16
GatewayId: !Ref pGatewayID
# attach route tables to our previously created subnets.
rSubnetRouteTableAssociation1:
Type: AWS::EC2::SubnetRouteTableAssociation
Condition: CreateNewSubnets
Properties:
SubnetId: !Ref rDBSubnet1
RouteTableId: !Ref rCustomRouteTable
rSubnetRouteTableAssociation2:
Type: AWS::EC2::SubnetRouteTableAssociation
Condition: CreateNewSubnets
Properties:
SubnetId: !Ref rDBSubnet2
RouteTableId: !Ref rCustomRouteTable
# Security group.
#
# This is to make sure our databases only allow traffic on the correct port.
#
# Technically, we could be more specific than 0.0.0.0/0
rDBEC2SecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Open database for access
VpcId: !Ref pRDSVPC
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: !FindInMap
- DBPortMap
- !Ref 'pDBEngine'
- port
ToPort: !FindInMap
- DBPortMap
- !Ref 'pDBEngine'
- port
CidrIp: 0.0.0.0/0
Description: !Ref 'pDBEngine'
Tags:
- Key: "Name"
Value: !Join [ "-", [!Ref "AWS::StackName", !Ref "AWS::Region", "RDS-SEC"] ]
# Subnet Groups
#
# Here is the deal. We don't attach subnets directly to databases. Rather, we group subnets and attach the group to the
# databases. So, here is the complex "if" logic to determine which subnets to put into our database subnet group.
rDBSubnetGroup:
Type: "AWS::RDS::DBSubnetGroup"
Properties:
DBSubnetGroupDescription: "description"
SubnetIds:
- !If [CreateNewSubnets, !Ref rDBSubnet1, !Ref pExistingSubnetId1 ]
- !If [CreateNewSubnets, !Ref rDBSubnet2, !Ref pExistingSubnetId2 ]
# Database Builds...
#
# Aurora builds... first
rDatabaseCluster:
Type: AWS::RDS::DBCluster
Condition: CreateAurora
Properties:
DatabaseName: !Ref 'pDBName'
Engine: !Ref 'pDBEngine'
EngineVersion: !Ref pDBEngineVersion
MasterUsername: !Ref 'pDBUser'
MasterUserPassword: !Ref 'pDBPassword'
DBSubnetGroupName: !Ref rDBSubnetGroup
DBClusterParameterGroupName: !Ref pDBParameterGroupName
DeletionProtection: !Ref 'pDatabaseDeletionProtection'
VpcSecurityGroupIds: [!GetAtt [rDBEC2SecurityGroup, GroupId]]
rAuroraDB1:
Type: AWS::RDS::DBInstance
Condition: CreateAurora
Properties:
CopyTagsToSnapshot: true
DBInstanceClass: !Ref 'pDBInstanceClass'
DBClusterIdentifier: !Ref rDatabaseCluster
DBSubnetGroupName: !Ref rDBSubnetGroup
Engine: !Ref 'pDBEngine'
EngineVersion: !Ref pDBEngineVersion
DBParameterGroupName: !Ref pDBParameterGroupName
PubliclyAccessible: false
LicenseModel: !Ref 'pDatabaseLicenseModel'
rAuroraDB2:
Type: AWS::RDS::DBInstance
Condition: CreateAuroraMultiAZ
Properties:
CopyTagsToSnapshot: true
DBInstanceClass: !Ref 'pDBInstanceClass'
DBClusterIdentifier: !Ref rDatabaseCluster
DBSubnetGroupName: !Ref rDBSubnetGroup
Engine: !Ref 'pDBEngine'
EngineVersion: !Ref pDBEngineVersion
DBParameterGroupName: !Ref pDBParameterGroupName
PubliclyAccessible: false
LicenseModel: !Ref 'pDatabaseLicenseModel'
# Now for the Non-Aurora builds...
rMasterDB:
Type: AWS::RDS::DBInstance
Condition: CreateNonAurora
Properties:
AllocatedStorage: !Ref 'pDBAllocatedStorage'
CopyTagsToSnapshot: true
DBInstanceClass: !Ref 'pDBInstanceClass'
DBName: !Ref 'pDBName'
DBSubnetGroupName: !Ref rDBSubnetGroup
Engine: !Ref 'pDBEngine'
EngineVersion: !Ref pDBEngineVersion
DBParameterGroupName: !Ref pDBParameterGroupName
MasterUsername: !Ref 'pDBUser'
MasterUserPassword: !Ref 'pDBPassword'
MultiAZ: !Ref 'pMultiAZ'
PubliclyAccessible: false
StorageType: !Ref 'pStorageType'
VPCSecurityGroups: [!GetAtt [rDBEC2SecurityGroup, GroupId]]
LicenseModel: !Ref 'pDatabaseLicenseModel'
DeletionProtection: !Ref 'pDatabaseDeletionProtection'
# Cloudwatch alarms
#
# Nothing too special here. I just make sure the DB is operational.
rAlarmTopic:
Type: 'AWS::SNS::Topic'
Properties:
Subscription:
- Endpoint: !Ref pOperatorEMail
Protocol: email
rCPUAlarmHighMasterDB:
Type: 'AWS::CloudWatch::Alarm'
Condition: CreateNonAurora
Properties:
EvaluationPeriods: 10
Statistic: Average
Threshold: 50
AlarmDescription: >-
Alarm if CPU too high or metric disappears indicating the RDS database
instance is having issues
Period: 60
Namespace: AWS/RDS
MetricName: CPUUtilization
Dimensions:
- Name: DBInstanceIdentifier
Value: !Ref rMasterDB
ComparisonOperator: GreaterThanThreshold
AlarmActions:
- !Ref rAlarmTopic
InsufficientDataActions:
- !Ref rAlarmTopic
rCPUAlarmHighAuroraDB1:
Type: 'AWS::CloudWatch::Alarm'
Condition: CreateAurora
Properties:
EvaluationPeriods: 10
Statistic: Average
Threshold: 50
AlarmDescription: >-
Alarm if CPU too high or metric disappears indicating the RDS database
instance is having issues
Period: 60
Namespace: AWS/RDS
MetricName: CPUUtilization
Dimensions:
- Name: DBInstanceIdentifier
Value: !Ref rAuroraDB1
ComparisonOperator: GreaterThanThreshold
AlarmActions:
- !Ref rAlarmTopic
InsufficientDataActions:
- !Ref rAlarmTopic
rCPUAlarmHighAuroraDB2:
Type: 'AWS::CloudWatch::Alarm'
Condition: CreateAuroraMultiAZ
Properties:
EvaluationPeriods: 10
Statistic: Average
Threshold: 50
AlarmDescription: >-
Alarm if CPU too high or metric disappears indicating the RDS database
instance is having issues
Period: 60
Namespace: AWS/RDS
MetricName: CPUUtilization
Dimensions:
- Name: DBInstanceIdentifier
Value: !Ref rAuroraDB2
ComparisonOperator: GreaterThanThreshold
AlarmActions:
- !Ref rAlarmTopic
InsufficientDataActions:
- !Ref rAlarmTopic
#
# Consider adding JDBC string...
#
Outputs:
Name:
Description: Aurora Stack Name
Value: !Ref AWS::StackName
Export:
Name: !Sub ${AWS::StackName}-Name
RDSEndPointAddress:
Description: Database Endpoint Address
Value: !GetAtt [rMasterDB, Endpoint.Address]
Export:
Name: !Sub ${AWS::StackName}-RDSEndPointAddress
Condition: CreateNonAurora
RDSEndPointPort:
Description: Database Endpoint port
Value: !GetAtt [rMasterDB, Endpoint.Port]
Export:
Name: !Sub ${AWS::StackName}-RDSEndPointPort
Condition: CreateNonAurora
AuroraClusterId:
Description: Aurora Cluster ID
Value: !Ref rDatabaseCluster
Export:
Name: !Sub ${AWS::StackName}-AuroraClusterID
Condition: CreateAurora
AuroraEndPointAddress:
Description: Database Endpoint Address
Value: !GetAtt [rDatabaseCluster, Endpoint.Address]
Export:
Name: !Sub ${AWS::StackName}-AuroraDatabaseURL
Condition: CreateAurora
AuroraEndPointPort:
Description: Database Endpoint port
Value: !GetAtt [rDatabaseCluster, Endpoint.Port]
Export:
Name: !Sub ${AWS::StackName}-AuroraDatabasePort
Condition: CreateAurora
EndPointDBName:
Description: Database Name
Value: !Ref 'pDBName'
Export:
Name: !Sub ${AWS::StackName}-DBName
JDBCConnectionString:
Description: JDBC connection string for a mysql database
Value: !Join ['', ['jdbc:mysql://', !GetAtt [rDatabaseCluster, Endpoint.Address], ':', !GetAtt [
rDatabaseCluster, Endpoint.Port], /, !Ref 'pDBName']]
Export:
Name: !Sub ${AWS::StackName}-MySQLJDBCString
Condition: CreateAurora