Attention
For production workloads, we recommend sing Lookup you can use existing OpenSearch clusters with your new services. This will avoid accidental deletions or rollback situations where both your DB and services have to rollback.
x-rds ¶
x-rds:
dbA:
Properties: {}
MacroParameters: {}
Services: {}
Lookup: {}
Define RDS DB Instances/Clusters and/or use existing ones and link access to the services.
When creating new RDS resources, compose-x will use AWS Secrets Manager to automatically register the new database username / password and passes it on to the services. See SecretsMapping and Database Credentials for more details.
See also
See Change AWS RDS / DocumentDB secret environment variables for a step by step example to change the SecretsMapping
Services ¶
Access ¶
Access:
DBCluster: RO
GenerateConnectionStringSecret: <string>
The only valid key for Access is DBCluster. The only valid value is
RO
for read-only, which allows IAM calls to RDS
to describe the cluster.
ReturnValues ¶
Use the AWS RDS DBCluster Return Values to expose the value for these properties to your service as an environment variable. If you are creating a RDS DB Instance, see AWS RDS DB Instance Return Values .
The value
DBCluster
can be used to expose the value for
Ref
GrantTaskAccess ¶
Allow to grant secret access to the Task Role. Can be either a boolean or mapping to specify an environment variable name with the ARN of the secret
Services:
app01:
Access: RW
app03:
Access: RW
GrantTaskAccess:
SecretEnvName: DB_B_SECRET # Grants access to the secret, sets env var DB_B_SECRET to the ARN
youtoo:
Access: RW
GrantTaskAccess: True # Grants access to the secret, not setting an env var
GenerateConnectionStringSecret ¶
This option enables to create a new secret that will be generated from the RDS Secret. The feature works for both newly created DBs and existing DBs (using Lookup).
Warning
Once the secret has been created, if the root secret of the DB has changed, the value in the generated secret will not be updated! Use at your own risks
Hint
Avoid to require the DB Connection string and have a separate environment variable for each part of the connection.
Example:
postgres://username:password@cluster-hostname:port/dbname
SecretsMapping ¶
This is an optional feature that allows you to map the secret key stored into Secrets Manager (see Database Credentials ) to a different environment variable.
x-rds:
wordpress-db:
Properties:
Engine: "aurora-mysql"
EngineVersion: "5.7"
BackupRetentionPeriod: 1
DatabaseName: wordpress
StorageEncrypted: True
Tags:
- Key: Name
Value: "dummy-db"
Services:
wordpress:
Access:
DBCluster: RO
SecretsMappings:
Mappings:
host: MARIADB_HOST
port: MARIADB_PORT_NUMBER
username: WORDPRESS_DATABASE_USER
password: WORDPRESS_DATABASE_PASSWORD
dbname: WORDPRESS_DATABASE_NAME
Hint
Using a mapping/dict format avoids duplicates. If you need the same value more than once, use the list syntax, as shown in the previous example.
If you combine the mappings for all services to specific mappings for one service, the specific ones take precedence.
Properties ¶
RDS clusters or instances need a lot of properties. In order to keep compatibility you can still provide all the properties that the RDS Cluster or RDS Instance would need with the same definition as in AWS CloudFormation.
However, some settings will be replaced automatically (at least for the foreseeable future), such as the master username and password. The reason for it is to allow to keep integration to your ECS Services as seamless as possible.
Note
When using Properties, you can use either the RDS Aurora Cluster properties or RDS Instances properties. ECS ComposeX will attempt to automatically identify whether this is a DB Cluster or DB Instance properties set. If successful, it will ingest all your properties, and interpolate a few with new ones created for you.
Properties overridden ¶
MacroParameters ¶
MacroParameters for RDS allow you to set only very little settings / properties and let ECS ComposeX do the rest for you.
Engine: str
EngineVersion: str
UseServerless: bool
UseMultiAz: bool
ParametersGroup: {} # Properties for parameters group as per AWS CFN definition
Instances: [] # Only valid when creating a DBCluster, allows to define multiple DB Instances
RdsFeatures: {} # Custom settings to define AWS RDS AssociatedRoles
PermissionsBoundary: str # Allow you to define an IAM boundary policy that will be used for the RDS IAM role(s)
Engine: aurora-postgresql # Same as AWS CFN Engine property
EngineVersion: 11.7 # Same as AWS CFN EngineVersion property
UseServerless: False
UseMultiAz: True
ParametersGroups:
Description: Some description
Family: aurora-postgresql-11.7
Parameters: {}
Instances: []
RdsFeatures:
- Name: s3Import
Resources:
- x-s3::bucket-01
- arn:aws:s3:::bucket/path/allowed/*
- bucket-name
PermissionsBoundary ¶
Allows to define whether an IAM Policy boundary is required for the IAM roles that will be created around the RDS Cluster/Instance.
Hint
This value can be either a policy name or policy ARN. When a policy Name, the ARN is built based on your Account ID.
RdsFeatures ¶
RdsFeatures:
- Name: <DB Engine feature name>
- Resources: [<str>]
The RDS Features is a wrapper to automatically define which RDS Features, supported by the Engine family, you might want to enable. For these features, which require an IAM role, it will create a new IAM role specifically linked to RDS and grant permissions based on the what the feature requires.
If you had set AssociatedRoles already in the permissions, then each FeatureName you have already defined that you might re-define in RdsFeatures will be skipped. If you wish to use RdsFeatures then remove that feature from the AssociateRoles definition.
Attention
This was primarily developed to allow feature request #375 so at the moment it only supports s3Import and s3Export.
x-rds:
dbB:
Properties: {}
MacroParameters:
PermissionsBoundary: policy-name
RdsFeatures:
- Name: s3Import
Resources:
- x-s3::bucket-01
- arn:aws:s3:::sacrificial-lamb/folder/*
- bucket-name
- Name: s3Export
Resources:
- x-s3::bucket-01
- arn:aws:s3:::sacrificial-lamb/folder/*
- bucket-name
Hint
You can reference a S3 bucket defined in x-s3 . This supports S3 buckets created and referenced via Lookup
Settings ¶
Refer to Settings
Lookup ¶
For x-rds, Lookup will 2 sub-arguments (cluster and instance are mutually exclusive). cluser or instance allow you to identify the RDS DB Cluster or DB Instance you wish your services to connect to.
Optionally (highly recommended), you can also set secret which indicates which secret in AWS Secrets Manager the service should be allowed to retrieve to connect to said Cluster / Instance.
It will also automatically identify the DB security group and add an ingress rule.
x-rds:
dba:
Lookup:
cluster:
Name: cluster-identifier
Tags:
- sometag: value
instance:
Name: DB Instance Id
Tags:
- sometag: value
secret:
Tags:
- sometag: value
Name: secret/in/secretsmanager
Database Credentials ¶
Aurora and traditional RDS Databases support both Username/Password generic authentication. Due to the wide adoption of that authentication mechanism, all RDS Dbs will come with a username/password, auto generated and stored in AWS Secrets Manager.
AWS Secrets Manager integrates very nicely to AWS RDS. This has no intention to implement the rotation system at this point in time, however, it will generate the password for the database and expose it securely to the microservices which can via environment variables fetch
After attachment between the RDS and the secret, the secret will not only contain the username and password, but additional information that is required by your application to connect to the database.
{
"password": "string<>"
"dbname": "string<>",
"engine": "string<>",
"port": int<port>,
"host": "string<>"
"username": "string<>"
}
Hint
We do plan to allow a tick button to enable Aurora authentication with IAM, however have not received a Feature Request for it.
Examples ¶
x-rds:
dbname:
Properties:
Engine: aurora-mysql
EngineVersion: 5.7.12
Services:
app01:
Access:
DBCluster: RO
x-rds:
existing-cluster-dbA:
Lookup:
cluster:
Tags:
- key: value
secret:
Tags:
- key: value
Hint
The DB Family group will be found automatically and the setting will allow creation of a new RDS Parameter group for the Cluster / DB Instance.
Tip
You can find the test files here to use as reference for your use-case.
JSON Schema ¶
Model ¶
x-rds ¶
|
x-rds.spec.json |
||||||
|
x-rds specification |
||||||
|
type |
object |
|||||
|
properties |
||||||
|
#/definitions/Lookup |
|||||
|
type |
object |
||||
|
#/definitions/SettingsDef |
|||||
|
#/definitions/ServicesDef |
|||||
|
#/definitions/RdsMacros |
|||||
|
additionalProperties |
False |
|||||
|
definitions |
||||||
|
x-rds-common.spec.json#/definitions/ServicesObjectDef |
|||||
|
type |
object |
||||
|
patternProperties |
||||||
|
Object representation of the service to use. |
|||||
|
properties |
||||||
|
oneOf |
type |
string |
|||
|
type |
object |
|||||
|
#/definitions/SecretsMappingsDef |
|||||
|
If set, creates an additional secret that will represent the connection string to the DB, and sets the environment variable |
|||||
|
type |
string |
|||||
|
Set the CFN Return Value and the environment variable name you want to expose to the service |
|||||
|
type |
object |
|||||
|
patternProperties |
||||||
|
oneOf |
x-resources.common.spec.json#/definitions/varNameDef |
||||
|
type |
object |
|||||
|
properties |
||||||
|
x-resources.common.spec.json#/definitions/varNameDef |
|||||
|
additionalProperties |
False |
|||||
|
additionalProperties |
False |
|||||
|
type |
array |
||||
|
items |
type |
object |
||||
|
properties |
||||||
|
type |
string |
||||
|
oneOf |
type |
string |
|||
|
type |
object |
|||||
|
properties |
||||||
|
type |
string |
||||
|
additionalProperties |
True |
|||||
|
#/definitions/SecretsMappingsDef |
|||||
|
type |
object |
||||
|
properties |
||||||
|
#/definitions/SecretsMappingsDef |
|||||
|
type |
string |
||||
|
type |
object |
||||
|
properties |
||||||
|
type |
string / boolean |
||||
|
oneOf |
#/definitions/SecretsMappingsList |
||||
|
#/definitions/SecretsMappingsObject |
||||||
|
additionalProperties |
False |
|||||
|
type |
array |
||||
|
items |
type |
object |
||||
|
properties |
||||||
|
type |
string |
||||
|
type |
string |
||||
|
pattern |
^[a-zA-Z0-9_]+$ |
|||||
|
additionalProperties |
False |
|||||
|
type |
object |
||||
|
patternProperties |
||||||
|
type |
string |
||||
|
type |
object |
||||
|
properties |
||||||
|
x-resources.common.spec.json#/definitions/Lookup/properties/RoleArn |
|||||
|
type |
object |
||||
|
properties |
||||||
|
type |
string |
||||
|
x-resources.common.spec.json#/definitions/Lookup/properties/Tags |
|||||
|
type |
object |
||||
|
properties |
||||||
|
type |
string |
||||
|
x-resources.common.spec.json#/definitions/Lookup/properties/Tags |
|||||
|
type |
object |
||||
|
properties |
||||||
|
type |
string |
||||
|
x-resources.common.spec.json#/definitions/Lookup/properties/Tags |
|||||
|
type |
object |
||||
|
properties |
||||||
|
The RDS Engine name |
|||||
|
type |
string |
|||||
|
type |
string |
||||
Definition ¶
{
"$schema": "http://json-schema.org/draft-07/schema#",
"id": "x-rds.spec.json",
"$id": "x-rds.spec.json",
"title": "x-rds",
"description": "x-rds specification",
"type": "object",
"additionalProperties": false,
"properties": {
"Lookup": {
"$ref": "#/definitions/Lookup"
},
"Properties": {
"type": "object"
},
"Settings": {
"$ref": "#/definitions/SettingsDef"
},
"Services": {
"$ref": "#/definitions/ServicesDef"
},
"MacroParameters": {
"$ref": "#/definitions/RdsMacros"
}
},
"definitions": {
"ServicesDef": {
"$ref": "x-rds-common.spec.json#/definitions/ServicesObjectDef"
},
"ServicesObjectDef": {
"type": "object",
"patternProperties": {
"[\\x20-\\x7E]+$": {
"description": "Object representation of the service to use.",
"properties": {
"Access": {
"oneOf": [
{
"type": "string"
},
{
"type": "object"
}
]
},
"SecretsMappings": {
"$ref": "#/definitions/SecretsMappingsDef"
},
"GenerateConnectionStringSecret": {
"type": "string",
"description": "If set, creates an additional secret that will represent the connection string to the DB, and sets the environment variable"
},
"ReturnValues": {
"type": "object",
"description": "Set the CFN Return Value and the environment variable name you want to expose to the service",
"additionalProperties": false,
"patternProperties": {
"[\\x20-\\x7E]+$": {
"oneOf": [
{
"$ref": "x-resources.common.spec.json#/definitions/varNameDef"
},
{
"type": "object",
"additionalProperties": false,
"properties": {
"EnvVarName": {
"$ref": "x-resources.common.spec.json#/definitions/varNameDef"
}
}
}
]
}
}
}
}
}
}
},
"ServicesListDef": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"access": {
"oneOf": [
{
"type": "string"
},
{
"type": "object",
"additionalProperties": true,
"required": [
"DBCluster"
],
"properties": {
"DBCluster": {
"type": "string"
}
}
}
]
},
"SecretsMappings": {
"$ref": "#/definitions/SecretsMappingsDef"
}
}
}
},
"SettingsDef": {
"type": "object",
"properties": {
"SecretsMappings": {
"$ref": "#/definitions/SecretsMappingsDef"
},
"Subnets": {
"type": "string"
}
}
},
"SecretsMappingsDef": {
"type": "object",
"additionalProperties": false,
"properties": {
"PrefixWithDbName": {
"type": [
"string",
"boolean"
]
},
"Mappings": {
"oneOf": [
{
"$ref": "#/definitions/SecretsMappingsList"
},
{
"$ref": "#/definitions/SecretsMappingsObject"
}
]
}
}
},
"SecretsMappingsList": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"SecretKey"
],
"properties": {
"SecretKey": {
"type": "string"
},
"VarName": {
"type": "string",
"pattern": "^[a-zA-Z0-9_]+$"
}
}
}
},
"SecretsMappingsObject": {
"type": "object",
"patternProperties": {
"^[a-zA-Z0-9_]+$": {
"type": "string"
}
}
},
"Lookup": {
"type": "object",
"oneOf": [
{
"required": [
"cluster"
]
},
{
"required": [
"db"
]
}
],
"properties": {
"RoleArn": {
"$ref": "x-resources.common.spec.json#/definitions/Lookup/properties/RoleArn"
},
"cluster": {
"type": "object",
"properties": {
"Name": {
"type": "string"
},
"Tags": {
"$ref": "x-resources.common.spec.json#/definitions/Lookup/properties/Tags"
}
}
},
"db": {
"type": "object",
"properties": {
"Name": {
"type": "string"
},
"Tags": {
"$ref": "x-resources.common.spec.json#/definitions/Lookup/properties/Tags"
}
}
},
"secret": {
"type": "object",
"properties": {
"Name": {
"type": "string"
},
"Tags": {
"$ref": "x-resources.common.spec.json#/definitions/Lookup/properties/Tags"
}
}
}
}
},
"RdsMacros": {
"type": "object",
"properties": {
"Engine": {
"type": "string",
"description": "The RDS Engine name"
},
"EngineVersion": {
"type": "string"
}
}
}
}
}
Test files ¶
You can find the test files here to use as reference for your use-case.