IV. The Processes for Patch Management

/* The following article is extracted from the "Information Security Newsletter" published by the JUCC IS Task Force. */ 
To build an effective patch management process that manages the risks from both external (i.e. vulnerabilities of the information systems) and internal factors (i.e. vulnerabilities related to patch itself), universities are recommended to consider the following practices:

1. Security and Patch Information Sources  
A key component of patch management process is the intake and vetting of information regarding both security issues and patch release. The IT security staff must know which security issues and software updates are relevant to universities' environment. Designated staff for each information system should be appointed to keep up to date on newly released patches and vulnerabilities through trusted sources, such as Microsoft Security Bulletin and Symantec. By leveraging the IT asset register, universities can determine whether all existing or critical information systems are covered by the patch management process.
Universities should also maintain close contact with the vendors of their key information systems, including operating systems, applications and network devices, to facilitate timely response to emerging vulnerabilities.
2. Patch Prioritisation and Scheduling
Universities should first establish a patch cycle that guides the normal application of patches to information systems. The cycle can be triggered regularly or event-driven. For example, weekly security patch update can be activated on universities PC desktops and laptops that use Windows platform as their operating systems. The IT security staff could also manually initiate the cycle when there is release of service packs or important security patches. In either instance, deployment of patches should be made based on system criticality, availability requirements and available resources.
Once the patch cycle is established, universities should integrate the prioritisation and scheduling process.
In general, patches are prioritised by first categorised into "security-related" and "everything else" types. Higher priority should be given to "security-related" patches by default. A more detailed prioritisation can be performed by referring to the following criteria: 
  • Vendor Reported Criticality - vendor reported criticality is a key input for calculating a patch's significance. Higher priority should be considered for "High" vendor reported criticality as reverse engineering of the patches may result in severe security breaches.
  • System / Service Criticality - The relative importance of the information systems or data is another valuable input for assessing the priority. The servers of a university's financial system or student information registry are more critical than desktops and should be patched first.
  • System Exposure - information systems accessible by external users or the general public are exposed to higher probability of malicious attacks and therefore require close attention by the universities.

For example, vulnerabilities or program bugs for DMZ systems could lead to more dangerous security incidents than those related to internal file servers. Therefore, the relevant patches of DMZ systems should enter the patch cycle earlier.

Based on the nature of the IT environments and service characteristics, each university should establish a prioritisation matrix, which provides clear guidance and criteria on patch prioritisation for the responsible IT security staff.

3. Patch Validation and Testing

The patch testing process begins with the acquisition of the patches and continues through acceptance testing after production deployment. 

The first step is the verification of the patches' source and integrity. This step helps ensure that the update is valid and has not been maliciously or accidentally altered. Responsible IT security staff could rely on digital signatures (e.g. MD5), checksum (e.g. Cyclic Redundancy Check) or any other integrity verification means to perform patch validation. 
Once a patch has been determined valid, it is should be tested in a test environment which mimics at least the majority of the production infrastructure to ensure a smooth and predictable rollout. Based on the system criticality, availability requirements, available resources and patch priority, the testing could be simply making sure system reboots or a series of detailed test scenarios. Access to the testing environment must be restricted to authorised testers to prevent tested patches from being replaced by malicious files.
For patches on important systems, the testing should include the fallback procedures to ensure that the critical services or operations supported by these systems can be restored correctly and timely.
4. Change Management
All patch management activities should follow the change management procedures established by the universities. Authorisations on source of system patches, acceptance testing and installation should be obtained from respective system owners and universities' IT management.
Like any critical changes to universities' IT environment, patch deployment plans submitted through change management must have associated fallback plans that defines the handling procedures if something goes wrong during or as a result of the application of a patch.
5. Patch Installation and Deployment
Deployment of patches should be conducted in a controlled manner.
Universities should not grant users and even administrators of critical information systems with the access rights to apply patches arbitrarily. The IT security staff must ensure adequate level of physical and logical access controls being implemented to restrict any unauthorised patch deployment. Only authorised IT operations staff are permitted to install patches in the production environment or initiate fallback plan if required.
6. Other Work-arounds
Occasionally patches released by software vendor may not be 100% compatible with the existing software of the universities, resulting in system crashes, instability or various kinds of disruption to the production environment. While deployment of these patches may not be feasible in the above cases, the risks and vulnerabilities associated to the patch should not be overseen. The following actions should be considered as an alternative: 
  • Report to software vendor and obtain an understanding of the underlying risks for not deploying the patch, and request an updated version of the patch
  • Implement additional security controls to mitigate associated risks
  • Shut down the software function where the vulnerability resides so that it cannot be exploited

V. Summary

While patches are necessary components for most of today's information systems to continuously refine and enhancement their functionalities and security measures, it may also lead to risks and vulnerabilities if they are improperly utilised by users.
A successful patch management process encompasses the identification, prioritization, scheduling, testing, change management and deployment of patches in a structured manner. It ensures that vulnerabilities or errors in the information systems, hardware and firmware are timely remediated without causing any adverse effect.
To achieve an effective patch management practice, universities management should establish relevant policy and procedures and appoint appropriate IT staff resources to maintain and monitor the execution of the above process.