Emergency Changes
An emergency change is a change that must be implemented as quickly as possible to address a critical incident or to prevent a critical incident. Emergency changes should only be used in genuine emergency situations where the normal change process would be too slow to prevent major impacts. Emergency changes should not be used for changes that are urgent but not critical as this can introduce unnecessary risk to the service.
Emergency changes, by necessity, will bypass or significantly shorten some of the usual change management process due to the urgency involved. This document will describe how an emergency change should be handled and which parts of the change management process will be bypassed or shortened.
Identification
A critical issue is identified and the need for an emergency change is recognised by you, the team or an external individual or organisation.
Emergency changes can take many forms. Here are some examples:
- Security patches to address critical vulnerabilities which are able to be exploited by the user community.
- Cybersecurity incident responses where a configuration change is required to mitigate or protect.
- Modifications to infrastructure to resolve a major outage.
- Activation of a cold stand-by server to restore service.
- Isolating network components to mitigate a DDoS attack.
Notification
Notify the Change Management Board Chair and/or Deputy if you can as quickly as possible. They will arrange an Emergency Change Management Board meeting. This meeting will be arranged for as quickly as possible to discuss the issue.
If you cannot get hold of anyone from the Change Management Board you should notify the Leadership Team. If this is still not possible you should gather together as many of the remaining team members as you feel is appropriate to manage the situation.
You should attend the meeting if possible, but if not you must attempt to get another member of the team to represent the emergency change. Look at the next section to ensure you are prepared to answer the questions which will be asking during the meeting.
Emergency Change Management Board
To determine if the change can go ahead the following will be discussed verbally:
- Agree who will be taking notes for the meeting.
- What is the critical issue that needs to be addressed?
- What are the consequences if the emergency change is not implemented immediately?
- What exactly is being changed? Are there any other ways to mitigate the issue? Do we actually need to do anything?
- What are the risks associated with the emergency change? Could the change cause further issues or disruptions?
- Could this emergency change affect other systems, applications or services?
- How will the change be implemented? What are the specific steps and who is responsible for each task?
- What happens if the emergency change fails or causes unexpected issues?
- Who do we need to make aware and how do we do that?
- How will the change be monitored after implementation?
- What is the expected time to execute the emergency change?
- Do we need to meet again after the emergency change is complete?
- What other changes are planned for the same time and what happens to them?
These topics of discussion cover the same requirements as a normal change. It is only the speed and method of communication which has changed.
If there are no other questions from the group the decision to approve or not will be discussed. The following acceptance criteria may be used as a guide. Adjustments are acceptable depending on the circumstances as long as the Emergency Change Management Board is willing to accept those adjustments.
- All questions above have been answered to the satisfaction of the Emergency Change Management Board.
- The implementation plan is clear and acceptable.
- If they are possible the test and backout plans are clear and acceptable.
- A clear plan for communication has been developed.
- The timing of the change is acceptable.
- Other changes which conflict are able to be cancelled or re-scheduled.
- The change has the resources it needs to be completed as described.
- Any actions which are to be performed have been assigned to individuals and they are aware or will be made aware and how.
- The Emergency Change Management Board is satisfied the change can be completed safely.
If approved the emergency change can go ahead as discussed and the decision formally recorded.
If the emergency change is not approved the rationale should be formally noted and what the next steps are for the emergency change if any.
From this point follow the Change Management Process procedure starting at the Implementation stage adjusting as required.
Documentation
When you are able the emergency change must be recorded. Follow the Change Management Process procedure to create the change in Zammad with the required details. Please ensure the title of the ticket starts Emergency Change:
to make it easier to find in the future.
Refer to the notes taken at the Emergency Change Management Board and/or ask the designated note taker to help you finish writing the ticket.
If the change is complete the ticket can be immediately closed with the appropriate tags added.
Inform the Change Management Board Chair and/or Deputy to inform them the change has been raised.