'How to fix Amazon RDS database that is in an incompatible-network state
I am having some issue with our AWS Aurora MySQL RDS DB. DB instance status is in incompatible-network
state. This DB is not being used actively and hence might have been stopped and later restarted by some automation etc. and I think the incompatible-network state of the RDS DB instance could be the result of that (mentioned as one of the reasons in the following article).
I followed this AWS article: How do I resolve issues with an Amazon RDS database that is in an incompatible-network state? in order to fix it, which says
To fix an RDS DB instance that is already in an incompatible-network state, and if you previously enabled automated backups for your RDS DB instance, temporarily stop any writes to the database, and then perform a point-in-time recovery (PITR).
However, Even when I restored the DB from backup the newly created DB is still in the same Incompatible-network
state. How can I get it to an Active state.
The DB created by point-in-time restore from the backup
Moreover, is there another way to get the DB in Available/Active state, if I do not care about the data in the DB?
This DB was created though CloudFormation stack and an update to the stack failed due to this issue and caused the CF stack to be in UPDATE_ROLLBACK_FAILED
state. Essentially, I need the DB to be in sync with the CloudFormation stack through which it was created.
Solution 1:[1]
Here are the details about the Root cause and the Resolution of the issue:
Root Cause:
I got help from Marcin's comments here to go in the right direction in terms of the confirming the root cause of the issue.
As I mentioned this DB is not being used actively and hence might have been stopped and later restarted by some automation etc. and when it was getting restarted, due to lack of available IP Addresses in the subnet, it got into incompatible-network
state.
Resolution:
As there were no IP addresses available, the point-in-time restore (PITR) from the backup (which is suggested in this aws article) also resulted in new DB being in incompatible-network
state.
Hence, I went onto to get some IPs freed up in the mentioned subnet. For me there were some other unused RDS DBs also using the same subnet(and the AZ), so deleted those. After freeing up IPs in the mentioned subnet.
- Created new DB by PIT restore of the old one (which was in incompatible-network state). This time it was in Available state.
- Deleted the old db & Renamed the new DB cluster/instance with original names
- Did continue-update-rollback of the CloudFormation stack which was in
UPDATE_ROLLBACK_FAILED
. With that the stack got it into the stable state ‘UPDATE_ROLLBACK_COMPLETE’ - Ran CF stack update with the same original changes and it went trough fine.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 |