Converting Encrypted Data in RDS from a KMS CMK to a KMS MRK
With a security-first mindset, it is essential during application development that certain user information is encrypted in databases. If the AWS KMS service was utilized for this task on or before June of 2021, then a Symmetric CMK was most likely used. CMKs are regional. In the event of a DR incident, the encrypted values cannot be decrypted by a CMK in another region. This directly impacts Recovery Time Objectives (RTO). This blog will explain how to re-encrypt fields using a new MRK if needed.
Terminology:
- CMK – Customer Managed Key – A customer-managed Symmetric KMS key that AWS does not manage. This increases security for the user by allowing the organization to manage symmetric keys as opposed to AWS. These keys are regional. Meaning they can only be used by the region they were created in and the underlying zones.
- MRK – AWS KMS keys in different AWS regions that can be used interchangeably – as though you had the same key in multiple regions. Each set of related multi-region keys has the same key material and key ID, so you can encrypt data in one AWS Region and decrypt it in a different AWS Region without re-encrypting or making a cross-Region call to AWS KMS.
- AWS Encryption SDK – The AWS Encryption SDK is a client-side encryption library designed to make it easy for everyone to encrypt and decrypt data using industry standards and best practices. It enables you to focus on the core functionality of your application, rather than on how to best encrypt and decrypt your data. The AWS Encryption SDK is provided free of charge under the Apache 2.0 license.
- AWS Boto3 Library for Python – You use the AWS SDK for Python (Boto3) to create, configure, and manage AWS services, such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3). The SDK provides an object-oriented API as well as low-level access to AWS services.
- Encryption Context – A key-value pair of additional data that you want associated with AWS KMS-protected information. This is then incorporated into the additional authenticated data (AAD) of the authenticated encryption in AWS KMS-encrypted ciphertexts. If you submit the encryption context value in the encryption operation, you are required to pass it in the corresponding decryption operation.
Technical Tools utilized:
- Programming Language: Python3
- RDS Type: Inconsequential, this example uses Aurora Serverless, and PostgreSQL Python3
- AWS SDK (boto3) for Python Python3
- Optional: Encryption SDK for Python – won’t be covered but worth mentioning. Python3
- Psycopg2 library for postgres
- Secrets Manager for Auth
In some cases when designing applications the need for security and encryption if not an initial requirement is accepted to be an inevitability. Unintentional sharing of customer information is every CTO’s worst nightmare. Security best practices should be at the forefront of any company that values its customers’ privacy and wants to remain in good standing with their user-base. Few things can kill an application quicker than a security breach. Have I hammered home the point? I’ll assume you are nodding your head in agreement. Good. In short breach-bad, encryption-good.
We all know of many ways to encrypt our data. Mozilla has SOPs, there’s PGP, SSH, Asymmetric encryption, and many other solutions. That leads us to the AWS solution for Encryption. Enter AWS Key Management Service, or KMS for short. KMS allows for you (the brilliant and security-minded engineer) to create Asymmetric or Symmetric keys, or if you want AWS to manage the keys they have a solution as well. Here are the features of KMS:
| Type of KMS key | Can view KMS key metadata | Can manage KMS key | Used only for my AWS account | Automatic rotation |
|---|---|---|---|---|
| Customer managed key | Yes | Yes | Yes | Optional. Every 365 days (1 year). |
| AWS managed key | Yes | Yes | Yes | Required. Every 1095 days (3 years). |
| AWS owned key | No | No | No | Varies |
Pretty cool, right? Here are a few reasons why I like KMS and why it’s pretty rad as a solution:
- The IAM user, or my role must have permission to perform the decrypt and encrypt operations, adding a sweet layer of increased security, while following the AAA mindset of security.
- Key rotation – yes you can now manage rotation properly.
- Increased layers of security through Encryption Context.
We utilize KMS to encrypt strings and objects we want secure. Let’s move on to why this blog post exists and what I am explaining.
Prior to June 16th of 2021, Customer Managed Keys were single-region. Meaning that the keys could only be used for encrypt and decrypt operations in the region in which they were created. This caused some issues. The first being a DR scenario. What if your main region went down? Well, you’d be failing over to the DR environment I assume. If you need that decryption and encryption to still happen in your RDS, you might have an issue. Since the keys are regional, they won’t work in the DR Region. This obviously hurts RTO and can cause issues with your application performing operations properly.
So what’s the solution? Well on June 16th of 2021 AWS released Multi-Region Keys for the purpose of allowing the keys to be used in; you guessed it: multiple regions. This is great, and allows for a lot more room to implement sound DR strategies, Load Balancing, Geo-Location Routing, etc. Awesome. But if you are like many others you’ve probably already used your regional CMK in some capacity in your application backend. That’s where Foghorn can help. If you’ve encrypted certain columns of data in your databases, let me explain how you can get them re-encrypted with your new MRK.
I wrote a tool in Python that connects to my RDS instance and performs a query. Depending on your schema and how you structure your data, you may need to perform some join operations or query multiple DBs and Tables. How you do that is in your court, it would be impossible for me to cover all the cases of how data is organized. Nevertheless, once you have that part figured out the rest can be quite simple. Let’s tackle this:
Step 1: Creating the MRK you will be using
The first step is creating the new MRK that you need to re-encrypt tuple values to. So let’s start that process. You’ll need the kms createKey action permitted to your IAM user. So in the Management Console you can create the Key in the Key Management Service:
Once your key is created you can swipe the ARN for it and move to the next step.
Step 2: Creating the Service role or IAM Account to perform these operations
To preface: Be sure to read the best practices for IAM policies with KMS. There’s a ton of great stuff in there about how to best build this policy. I’m going to selectively post some of my personal favorites. First being that you should limit the Resources to the ARNs of the CMK you used previously, and the new Key:
"Resource": [ "arn:aws:kms:us-west-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab",
"arn:aws:kms:us-west-2:111122223333:key/0987dcba-09fe-87dc-65ba-ab0987654321"]
Secondly, you’ll need the appropriate DB Operations for your RDS instances. Read, Write, Insert, Join, and so on. Be sure to include those as well. Since you’ll be performing some DB operations as part of this.
Step 3: The Methods
First, we can do our variable assignments:
secret_name = SECRET NAME
secret_arn = SECRETS MANAGER ARN CONTAINING THE PASSWORD
db_name = DBNAME
region = REGION
session = boto3.client('kms')
db_cluster_arn = DB CLUSTER ARN
key_arn = OLD CMK ARN
key_id = OLD CMK KEY ID
mrk_key_arn = NEW MRK ARN
mrk_key_id = NEW MRK IDtarget_secret_name = NAME OF THE SECRET
session = boto3.session.Session()
client = session.client(
service_name='secretsmanager',
region_name=region
)
secret = client.get_secret_value(
SecretId=target_secret_name
)
try:
secret_dict = json.loads(secret['SecretString'])
except Exception as e:
print(e)
target_username = secret_dict['username']
target_passw = secret_dict['password']
#grab the decrypted passwords and pass into connect string for target DB here:
target_db_connection_string = "host= PUT YOUR CONNECTION STRING HERE dbname= YOUR DB user={} password={}".format(target_username,target_passw)
#connect
target = psycopg2.connect(target_db_connection_string)
target_engine = target.cursor()def decrypt_and_reencrypt():
results = engine.execute(name_of_Your_Query)
results = results.fetchall()
#calls the KMS Client in boto3:
client = boto3.client('kms')
for result in results:
Column_Header_1, column_header_2, column_header_n = result
encryption_context = {"KEY": value,
"KEY": value
}
#In this example the encrypted bytes were converted to base64, if you do the same this will convert it to a byte stream.
encrypted_data = bytes(base64.b64decode(encrypted_column_tuple))
#Decrypt the tuple value
try:
decrypted_data = client.decrypt(
CiphertextBlob=encrypted_data,
EncryptionAlgorithm="SYMMETRIC_DEFAULT",
EncryptionContext=encryption_context,
KeyId=key_id
).get('Plaintext')
#encrypt the tuple value
reenc_data = client.encrypt(
Plaintext=decrypted_data,
EncryptionAlgorithm="SYMMETRIC_DEFAULT",
EncryptionContext=encryption_context,
KeyId=mrk_key_id
).get('CiphertextBlob')
reenc_cc_type = base64.b64encode(reenc_data)
target_engine.execute(update_query, (reenc_data,primary_key))
target_engine
except Exception as e:
print(e)
And that’s basically it. You might ask why I elected to do the Decrypt and ReEncrypt operations separately instead of just calling the kms ReEncrypt method. The answer to that is because I like to test that the values were updating and outputting properly prior to performing the updates. You can certainly call the ReEncrypt operation, and it’s just as viable.
After completion of this (which could take a long time), your values should now be able to be stored using the MRK, awatnd Decrypting in a multi-region scenario.
If you are interested in learning more about how Foghorn can help customize cloud solutions for your business, please reach out. We would love to chat with you.
