MSI Fundamentals

Rule 1: Learn the Windows Installer Technology

The importance of this rule cannot be overstated. If you only follow one rule, this is the one to choose.

Microsoft Product Support often speak to Installer users who are installing packages which they created with a sophisticated high-level re-packaging tool, without any understanding of how the Installer actually works. While these tools are excellent at what they do and they abstract the user from the details on a day-to-day basis, the lack of Installer knowledge becomes a real problem when the package does not work as expected. Setup authoring is not simply about copying files. The Installer offers extensive functionality and complexity; understand it before you start authoring packages.

Start with this guide, then read the white papers, books, etc mentioned in the Resources section, then follow Rule 2.

Rule 2: Know Your Way Around the Installer SDK

The best resource to help with Rule 1 is the Windows Installer SDK (usually just called “the SDK”). It is the definitive guide to the Installer technology and covers all aspects of creating packages, including detailed descriptions of database tables, properties, custom actions, etc.

The SDK is designed as a reference and so you shouldn’t try to read it from start to finish. However, it does contain many tutorials and specific examples you may want to follow and try out for yourself. In any case, you should make yourself familiar with its contents and use it as your first port of call when you need to know something about the Installer.

Rule 3: Use the “Windows Logo” Program as a Basis For Good Practices

The "Designed for Windows XP" Logo Program is designed to help give users confidence in the software and hardware they buy to work with Windows. The logo ensures that any software conforms to a Microsoft-defined set of criteria that are designed to provide a consistent, high quality software experience. So, even if you do not intend to submit your application to the logo program, you can still use the guidelines to help make your Installer packages better.

Many of the logo requirements for software are covered if you use the Installer as your installation mechanism. Some of these rules are shown below, with more details available on the logo program website.
• Do not attempt to replace files that are protected by Windows File Protection
• Migrate from earlier versions of Windows
• Do not overwrite non-proprietary files with older versions
• Do not require a reboot inappropriately
• Install to Program Files by default
• Install any shared files that are not side-by-side to the correct locations
• Support Add or Remove Programs properly
• Support “All Users” installs
• Support Autorun for CDs and DVDs

Rule 4: Always Use the Latest Version of the Installer

Using the latest version of the Installer gives three important benefits:
• Extra features. Version 3.1 is the Installer version at the time of writing. This contains many new and enhanced features over earlier editions. In particular, from version 3.0 of the Installer, patching and patch management was greatly improved.
• Bug fixes. Using the latest code, and keeping this patched with current updates from Microsoft Update, ensures that you are more secure and will avoid known problems.
• Support. Microsoft does not support or issue bugfixes for older Installer versions, such as v1.1. Always using the latest version means that Microsoft Product Support will be able to assist with problems and supply updates and patches.

Rule 5: Build Setup Into Your Application Development from the Start

Installation absolutely should not be an afterthought in the development process. Many developers don’t like the idea that “setup” should influence the design of a product, but when using the Installer it is crucial to develop the two in parallel.

Thinking up-front about how an application will be installed makes it easier to take advantage of the idea of “features”, “self-repair”, etc. So, rather than forcing the software into some artificial split or limiting very large applications to a single feature, you will end up with a more user-friendly and resilient install.

Remember that:
• The developer writing the code knows better on what, how and where the configuration data for thier application/module should be
• It is better to grow the stability and confidence of the product setup along with the confidence in the product itself: a poor product setup will leave lasting bitterness with the user


Rule 6: Get to Know ORCA

ORCA is an MSI package editing tool that ships with the Installer SDK. It has a basic GUI interface but supports advanced editing of Installer Databases. It is possible to create a new package completely with ORCA, but this would be very tedious and error prone; equivalent to writing the Microsoft.com website in Notepad - possible, but not something you’d really want to do.

Where ORCA excels is in letting you quickly view and edit package contents for troubleshooting and testing. This reason alone should be enough reason to become familiar with using ORCA, but it also provides a convenient way to validate your packages - a vital step during package creation and testing.

Packaging

Rule 7: Work On a Copy

"My file server won't boot and I really need it up and running now!"
"Ok, let's re-install Windows and restore the data from backup."
"What backup ...?"

Been there? Done that? While losing a single MSI package might not be as disastrous as losing your file server, you really don't want to start from scratch because of some accidental corruption.

So, always make a copy of your package before you start work, then work on the copy. That way, it is easy to revert to a known good version if the working one becomes corrupt or has other problems you cannot easily correct.

Rule 8: Never Cancel a Package Build Before it Finishes

To avoid corrupting a package, allow the packaging tool to finish building any new version once initiated. Even if you realise that you need to make more changes or did something wrong, allowing the package to build rather than stopping part way through helps avoid possible corruption. Obviously, if the package does become corrupt, all is not lost if you followed Rule 7.

Rule 9: Use a Clean System for Repackaging

If you are repackaging an existing application rather than creating your own original package, then it is important to do this on a “clean” system. Repackaging tools commonly use “snapshots” of the system state before and after the legacy application is installed, then create an MSI package based on the differences. To avoid any extraneous changes making their way into the packages the systems should:
• Match the target system as closely as possible.
• Be clear of any unnecessary applications
• Have all non-essential system services stopped
• Have all unnecessary processes stopped
Some limitations of repackaging are discussed in the following article:

INFO: Disadvantages of Repackaging Applications

In addition to this, you should check with 3-rd party vendors what their support policy is for repackaging their applications. It is possible that you will void any support agreement if you do not use the vendor-supplied installation mechanism.

Rule 10: Do Not Repackage Microsoft Updates

Microsoft uses a different packaging technology to create Service Packs and other updates for many products, including Windows, Internet Explorer, Exchange Server, SQL Server and ISA Server. This technology is not related to the Windows installer and it is completely unsupported to convert the updates to Installer packages. Refer to the following article for support details:

Repackaging software updates to use Windows Installer is not supported

You may be aware that Windows Service Packs ship with an “Update.msi” package that can be used to deploy the Service Pack via Group Policy. This package simply calls the normal .exe-based install as a custom action. This is the only supported use of this package and it is not supported to create your own similar MSI wrapper for Service Packs or other updates. Apart from the one situation mentioned, deploying updates in this way is not tested by Microsoft and the results cannot be guaranteed and hence are unsupported.

Rule 11: Do Not Repackage MSI-Based Applications

The Installer does not just copy files and registry keys to the target system. It also creates a lot of “configuration data” to help it keep track of the components, features, etc. If you use a “snapshotting” technique to repackage an MSI-based application your package will include all of this extra data, which it will treat as normal files and registry information. However, this will almost certainly lead to serious problems with the Installer, since this data is actually Installer data from another install.

Rule 12: Modify Vendor Packages Using Transforms

As previously mentioned vendor may not support re-packaging their legacy packages. It is also possible that they will not support direct modification of their MSI packages either. For example, Microsoft does not support customisation of Office installations by editing the Office MSI package.

The recommended (and more likely to be supported) method of modifying vendor packages is via transforms. You can create these in all major packaging tools or by using the MsiTrans.exe tool that ships with the SDK. You should also consider the use of transforms for customising your own packages. Using this technique you can have a single base package that can be deployed to a variety of clients, with customisation handled via one or more transforms.

Rule 13: Be careful with Installer GUIDS

The Installer uses GUIDs to uniquely identify applications, packages, features, etc.

For example, the ProductID is the GUID used by the Installer to distinguish one application from another. It doesn’t matter if the package or application names for two different applications are different, if the IDs are the same, the Installer is likely to become confused and you will encounter problems with installation, repair and uninstall.

You can use the same ProductID when a package is a newer build of an existing one, but you need to then have different PackageIDs to distinguish the application versions.

If you want the packages to be upgrades of each other, then you need them to have the same UpgradeID.

GUIDs should be all uppercase

Rule 14: Use Consistent Package Naming Conventions

The name of the MSI package itself doesn’t really matter as far as the Installer is concerned. As Rule 13 mentions, the Installer uses GUIDs to identify packages, etc. However, while the Installer doesn’t care about the original MSI name, once it registers that name for the product, it does care. Any small and/or minor update that is issued via an MSI must use the same base package name (the source location can be different depending upon source policy, but the base package name must be the same).

Also, package naming is important from a user point of view. Consistent, user-friendly naming allows the user to identify a package's contents from the name alone. For example, the following convention can be useful in a corporate environement where there may be many in-house packages available to users:

___.msi

Following this, a repackaged version of Adobe Acrobat Reader v7.1 available to all users would be named:

Adobe_AcrobatReader_71_ALL.msi

Rule 15: Do Not Try to Replace Protected System Files

Since the release of Windows 2000, Windows has shipped with Windows File Protection (WFP), a mechanism for monitoring and restoring system files that have been renamed, deleted, etc.

The Installer has no mechanism for by-passing WFP and if your package attempts to replace a protected file, the installation may not give the expected result. For the sake of application compatibility, the install does not fail if it is unable to update a WFP file. Instead, an entry is written to the application event log. However, it is important to note that the application may not be using the intended version of the file.

Do not be tempted to work around this by using a custom action or other means. WFP will roll-back any changes you make. In any case, even if you managed to force your changes onto the system replacing protected files is simply unsupported by any means except Microsoft-approved methods. For example, by installing a Service Pack.

Rule 16: Follow Component Rules

Components are a very important part of the Installer technology. They are the means whereby the Installer manages the resources that make up your application. The SDK provides the following guidelines for creating components in your package:
• Never create two components that install a resource under the same name and target location. If a resource must be duplicated in multiple components, change its name or target location in each component. This rule should be applied across applications, products, product versions, and companies.
• Two components must not have the same key path file. This is a consequence of the previous rule. The key path value points to a particular file or folder belonging to the component that the installer uses to detect the component. If two components had the same key path file, the installer would be unable to distinguish which component is installed. Two components however may share a key path folder.
• Do not create a version of a component that is incompatible with all previous versions of the component. This rule should be applied across applications, products, product versions, and companies.
• Do not create components containing resources that will need to be installed into more than one directory on the user’s system. The installer installs all of the resources in a component into the same directory. It is not possible to install some resources into subdirectories.
• Do not include more than one COM server per component. If a component contains a COM server, this must be the key path for the component.
• Do not specify more than one file per component as a target for the Start menu or a Desktop shortcut.

Rule 17: Understand File Versioning Rules

The Installer has strict rules that it follows when determining whether to replace an existing file or not. File version, timestamp and language are all used to determine what to do in most cases. It is particularly important to understand what happens with unversioned files. Generally, these are treated as user data, and if any modification has taken place since the original install the Installer will not over-write.

Refer to the SDK for full details of the versioning rules.

Rule 18: Improve Performance by Limiting System Restore During Setup

In a corporate environment where re-imaging or some other technique may be the preferred method for workstation recovery, and System Restore is never used, the additional time and disk space required to create the System Restore Points is essentially wasted. Turning it off for Installer activity will improve performance.

You can turn off System Restore snapshots for installation using this per-machine policy setting:

HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Windows\Installer
LimitSystemRestoreCheckpointing = 1

This setting is affects only Installer-initiated restore activity and is available in Group Policy to aid its deployment to workstations.

Note that System Restore is a very important feature of Windows and in most circumstances it is recommended that you do not switch it off, so this rule is only applicable for corporate scenarios where this feature is not used.

Rule 19: Avoid Using the SelfReg Table

Using the self-registering capabilities of certain DLLs is highly discouraged. Any activity performed by the self-registration (e.g. addition of registry entries) is out of the control of the Installer, so cannot be part of advertisement, repair and is not removed on uninstall. Instead you should have the Installer manage the data for you by using the appropriate tables in the MSI database.

Some problems and limitations associated with self registration are:
< li>
• The ability to use advertisement is reduced if class or extension server registration is performed within self-registration routines.
• The installer automatically handles HKCR keys in the registry tables for both per-user and per-machine installations. DllRegisterServer routines currently do not support the notion of a per-user HKCR key.
• If multiple users are using a self-registered application on the same computer, each user must install the application the first time they run it. Otherwise the installer cannot easily determine that the proper HKCU registry keys exist.
• The DllRegisterServer can be denied access to network resources such as type libraries if a component is both specified as run-from-source and is listed in the SelfReg table. This can cause the installation of the component to fail to during an administrative installation.
• Self-registering DLLs are more susceptible to coding errors because the new code required for DllRegisterServer is commonly different for each DLL. Instead use the registry tables in the database to take advantage of existing code provided by the installer.
• Self-registering DLLs can sometimes link to auxiliary DLLs that are not present or are the wrong version. In contrast, the installer can register the DLLs using the registry tables with no dependency on the current state of the system.

Rule 20: Avoid Nested Installs

A nested installation action installs another Windows Installer package during a currently running installation. This is typically used to install some pre-requisite component(s) for the main application.

Nested installs are deprecated and highly discouraged. Below are some of the limitations and problems associated with nested installs:
• Concurrent installations cannot share components.
• An administrative installation cannot also contain a concurrent installation.
• Patching and upgrading may not work with concurrent installations. The installer may not properly cost a concurrent installation.
• Integrated ProgressBars cannot be used with concurrent installations.
• Resources that are to be advertised cannot be installed by the concurrent installation.
• Nested installations live under the context of the parent product and do not appear in the Add or Remove Programs Control Panel applet. Therefore uninstall of the parent product is the only way to uninstall a nested install.
• A package that performs a concurrent installation of an application should also uninstall the concurrent application when the parent product is uninstalled.
The recommendation would be to use the bootstrap executable wrapper to install the MSI packages one after another. Each individual MSI would therefore be serviceable and can be treated independently.

The caveat to the bootstrap is that it can make deployment difficult. Group Policy deployment doesn’t have a concept of ordered dependencies in deployment of installation packages so if order is important you may not be as successful using this technique. This is especially problematic with low rights users since ZAP files do not allow for elevated installations.

Rule 21: Avoid Using Configuration Data You Don’t Own

As part of the initial design for the Installer, the consistency of installations was achieved via Microsoft taking ownership of the code for the install engine. This includes the location and format of all configuration data. This data is managed by the Installer and direct access by users or applications is discouraged; in fact some of the data is encoded to make it very difficult to manipulate manually.

You should not try to peek directly into Windows Installer configuration information. Instead, use the Windows Installer API to get the information you need. Accessing the data this way ensures that your package or application will continue to work even if the underlying configuration data changes location or format.

Rule 22: Differentiate Between User and Application Data

All user registration data should be written to the HKEY_CURRENT_USER hive and application data to the HKEY_LOCAL_MACHINE hive.

Also, your package should avoid over-writing user data during install, repair, etc. The user doesn’t want to have to re-configure their user-specific application settings after they repair the application.

Rule 23: Don’t Use Resources You are Installing

It is generally not a good practice to rely on the installation of certain non-critical resources. For example, a custom action that relies on the installation of a file that is part of an advertisable feature might fail when a user chooses to advertise the feature.

Another typical scenario is a custom action sequenced ahead of InstallFinalize that uses methods exposed by an assembly that your setup is installing. Since assemblies are not committed to the GAC until towards the end of InstallFinalize, you can not use them until after that point.

Rule 24: Use Cabinet Files to Reduce Package Sizes

The Installer allows for packages to contain compressed and uncompressed source files. Compressing the files and storing them in a cabinet (.cab) file reduces the size of the package. The Installer allows the .cab file to be stored as a separate external file, or as a data stream in the MSI package itself.

Note that for extremely large packages (i.e. with more than 32767 files), you must alter the schema to increase the number of files allowed. The steps to do this are documented in the SDK.

Rule 25: Follow Custom Action Rules

Custom actions allow for great flexibility in package installation, but are often the source of problems. The following are some simple rules to help get the best from your custom actions:
• Do not initiate a System Restore snapshot from a custom action. Even if you want System Restore to operate during installs, you should not attempt to create a restore point yourself from a custom action. Such a snapshot will contain whatever partial changes that have taken place up to that point and restoring the system to this time would leave the system in an unknown state. At the least your application is likely to be non-functional, at worst the system could be unstable.
• Log the Activity of Your Custom Actions. As is mentioned in several Rules, the Installer supports very detailed logging of its activities. However, the code inside your custom actions is hidden from the Installer, so it has no way to log what you're doing. Using either the Win32 API or the COM interface it is straightforward for your custom actions to log information about their activities, making it much easier to troubleshoot in the event of a problem.
• Write a rollback custom action for every deferred custom action. Although custom actions that schedule system operations by inserting rows into database table are reversed by a rollback of the installation, custom actions that change the system directly, or that issue commands to other system services, cannot always be reversed by a rollback. You should write your own rollback custom action to take care of removing these changes.
• Ensure that your impersonated custom action runs fine as a non-administrator account. This applies to all aspects of your package really, but it is easy to overlook in your own custom code. If you intend low-rights users to install your package then ensure that you test for this scenario.
• Avoid using script custom actions for doing non-trivial setup actions. Scripted actions are easy to write, but are usually hard to maintain.
• Think of ways to make your custom actions declarative. For example, use a custom tables to hold details of registry entries and files your custom action manipulates. Have your custom action read from these tables as it updates the system. This will give system administrators installing your package a better idea what your custom action is doing.
• Do not show dialogue boxes from custom actions. Use MsiProcessMessage to send messages to the installer service that will decide on how to handle the message based on the UI level and other constraints. This is a much more scalable approach.
• Be careful when calling Installer functions from a custom action. Calling certain functions - such as MsiUseFeature, MsiConfigureProduct, MsiApplyPatch, etc - will almost certainly lead to problems. Refer to the SDK for a full list of these functions.
• Do not change the system state from an immediate custom action. You should not attempt to use an immediate execution custom action to change the system state, because every custom action that changes the state needs to have a corresponding rollback custom action to undo the system state change on an installation rollback. All rollback custom actions are also deferred custom actions and must precede the action they undo

Rule 26: Consider Storing User-specific Data in a File

Many applications store customisation data in the user’s registry on first use, and when the user is not logged on this data is stored in the user's .dat registry file on disk.

This is all fairly standard, but a problem can arise if the application is removed when the user is not logged on - the data is inaccessible to the Installer and the application and so is effectively orphaned. This breaks the idea of having a clean uninstall.

To work around this, you can store these setting in a file on disk, which will always be available during uninstall, or at least is very easy for the user to delete later themselves.

This can be a simple text file, or, if you want to keep up with current trends, an XML file.

Rule 27: Consider Maintaining Setup in Text File Format

Many authoring tools such as the WiX toolkit now have underlying formats that permit building from text files. This makes viewing setup changes a lot easier and may be somewhat more intuitive for developers already used to dealing with version-controlled text files when writing application code.

WiX is free and gives you the ability to stay very close to the Windows Installer technology without having to worry about every tiny detail (such as keeping foreign keys cased correctly).

Rule 28: Think about Localisation

What if your package is English and a user in China installs it onto a Chinese system? In order to anticipate and cope with such situations learn about Installer properties like ProductLAnguage, Template Summary information property, code pages of the database and summary information for the package.

You should prepare for localization when authoring the original installation package. Design the layout of localised files such that different language versions can safely coexist when installed on the user’s computer. Organise files requiring localization into separate components and install these files into separate directories.

Rule 29: Follow the Assembly Rules

The Installer can install, remove and update Win32 and .NET assemblies, including side-by-side and private assemblies in Windows XP. To avoid common problems, follow these rules when using assemblies:

General:
• A component should contain no more than one assembly
• All of the files in an assembly should be in a single component
• Each component that contains an assembly should have an entry in the MsiAssembly table
• The strong assembly cache name of each assembly should be authored into the MsiAssemblyName table
• Use the Registry table instead of the Class table when you register COM Interop for an assembly
• Assemblies that have the same strong name are the same assembly. When the same assembly is installed by different applications, the components that contain the assembly should use the same value for the ComponentId in their Component tables
Win32 Assemblies:
• Do not use the manifest file or the catalogue file as the KeyPath in the Component table for the component containing the Win32 assembly
• The KeyPath value in the Component table for a component that contains a Win32 policy assembly should be Null.
• Add a row to the MsiAssemblyName table for each name and value pair that are listed in the section of the Win32 assembly's manifest
.NET Assemblies:
• The KeyPath value in the Component table for a component that contains the assembly should not be Null
• When you install an assembly used by the common language runtime to the global assembly cache, the value in the File_Application column of the MsiAssembly table must be Null
• Add a row to the MsiAssemblyName table for each attribute of the assembly's strong name. All assemblies must have the Name, Version, and Culture attributes that are specified in the MsiAssemblyName table. A publicKeyToken attribute is required for a global assembly

Rule 30: Per-machine Installs are Easier to Manage

Because per-machine packages are installed for every user, there are fewer problems for subsequent repair, patching and uninstall. For example, if a user installs an application only for themselves, another user (even an administrator) cannot later uninstall the application.

Rule 31: Keep the Original Source Files Available


Be aware that when you roll out packages that the original source package may have to remain available. For install-on-demand, repair and some patching situations the Installer returns to the original source of the relevant MSI package to find the required files. If this source is no longer available (e.g the server has been renamed or the original package removed) the user will by default see a prompt such as ‘Please insert CD #2’, or inviting them to browse the network.

Mechanisms for avoiding problems with this include:
• Correctly authoring the SourceList property for all packages. Set this to a semi-colon delimited list of all the possible servers where the software can be found. The clients will failover each server until one that succeeds is found. This can be done with a transform.
• Using a DFS share for the source path. Using a DFS server as the source means that changes to the actual physical location are hidden from the end user and the Installer. It also means that you can take advantage of site-awareness to point the user to a local copy of the source packages, even if they move to a location in another town, country or continent.
• Programmatically altering the registry source list on the client. You can use the Installer sourcelist API calls or a simple VBScript to do this on local or remote clients.

Rule 32: Anticipate Installation on Terminal Server


If your package might be installed on a Terminal Server, keep in mind the following points:
• Administrators and non-administrators can install from Terminal Services console sessions
• Non-administrators cannot install from Terminal Services remote sessions. Non-administrators can only install from a console session
• Administrators can install from a remote session only if the EnableAdminTSRemote system policy has been set
• When an administrator installs from a remote session that is hosted on Windows 2000, the installation must use a UNC path and not a mapped drive letter. This restriction only exists for remote sessions hosted on Windows 2000
• Windows Installer can perform per-machine installations without using install mode. It is unnecessary to place the Terminal Server computer into install mode to perform a per-machine installation. Windows Installer does not automatically place the Terminal Server computer in install mode, regardless of the type of installation
• When installing software for an individual user, placing the computer into install mode may incorrectly copy the application’s shortcuts or data to the profiles of other users, even if the application is not installed for these users. It is recommended that applications installed per-user be installed with the Terminal Server computer in the execute mode

Rule 33: Avoid Black-boxes


Make your packages as customisable as possible. Users like to decide for themselves where and what to install on their systems. Provide valid and sensible defaults, but allow users to over-ride these where possible.

Rule 34: Support Silent Installation


Corporate deployment scenarios almost always demand the ability to deploy with no user interactions. You should design your packages to use public properties for configuration information, which administrators can specify on the command-line.

You should also avoid relying on data gathered from users via dialogues. During silent installs, this data is not going to be availalbe.

Rule 35: Avoid Reboots


Being told to reboot their system is one thing that is sure to annoy users, particularly if there is no obvious reason for it. Even worse is the system automatically rebooting due to a silent install the user was entirely unaware of.

In the case of user applications it is usually not necessary to reboot the entire system. If this is required in your case, your install should handle situations where the application or service must be stopped.

Ideally, you should design your application such that if it is being upgraded or patched it can handle requests to shutdown/restart by saving the current state and restoring it later. this scenario is very much the recommended one for Windows Vista, where eliminating unnecessary reboots is a key objective. You can be ahead of the game by writing your applications to be work this way now.

If an application or service needs to be stopped, the user should be warned. Similarly, in cases where a reboot is completely unavoidable (e.g. replacing the GINA), the user should be warned and given a choice to defer the reboot.

Rule 36: Enable Logging on Clients


Windows Installer supports very detailed logging, which can be invaluable when things go wrong during deployment (and development and testing).

This article gives information on how to enable logging:

How to enable Windows Installer logging

Once you have the log, you obviously need to know what to look for. A very useful resource for this is WiLogUtl.exe, which ships with the SDK. As well as automating the parsing of the log, it also has some documentations describing typical log contents.

Rule 37: Enable the DisableMedia Policy


This policy helps prevent unauthorised installation. When enabled, users cannot install from anything except a local fixed drive or a remote server. This prevents users installing unauthorised MSI packages from removable media such as CD or Floppy Disk.

Rule 38: Avoid Using the AlwaysInstallElevated Policy


While this policy setting will allow a non-administrator to install packages they would otherwise not have right to install, it would allow them to install any package, not just ones provided by their administrator.

Since this policy causes the install to run with SYSTEM privileges, it effectively serves as a possible elevation of privilege security breach. A reasonably smart user could craft an MSI package that would add them to the administrators group, install kernel drivers, alter file and registry permissions and perform other undesirable actions.

As tempting as this policy may seem as a solution to issues of user rights and permissions that are blocking a deployment, do not use it.

Rule 39: Uninstall Cleanly


While most packaging effort goes into the creation and install phases, you should also think carefully about what happens when the user uninstalls the package. Your packages should be authored such that they leave no trace of themselves after uninstall.

Rule 40: Use Installer Version 3.0 or Later


One of the main objectives of MSI3.0 was to improve application servicing. There are many new features such as sequencing and obsolescence that greatly aid in patching installed applications.

In addition, from a system administrators point of view, one key improvement is the ability to remove patches. Prior to Installer v3.0 patches could not be uninstalled without removing the parent application. This is no longer the case.

Note that you should update the Installer before you deploy any applications or patches, as upgrading afterwards does not make already installed patches uninstallable. If this is not possible, then still author patches for MSI3.0. While they won’t be able to take advantage of the new features, they are fully backward compatible and won’t need changed once you do upgrade.

Rule 41: Design and Test Your Servicing Strategy Before You Ship the Initial Product


You must know in advance how you’ll be updating your product, and once it’s shipped it’s too late to change its design. So build and install your product, then create an update (a minor update, a major upgrade, whatever you’ll be using) and make sure that it works.

Rule 42: Author Packages to Avoid Source Requirements During Patching

One major problem with patching previously was that the original source files were required to install the patch. With MSI3.0 this requirement was reduced, but not eliminated. You can help yourself by authoring your packages and patches from the start in a way that reduces dependence on the source files. Here are some tips on doing this:
• Use binary delta patching. You can patch efficiently by just targeting baselines (RTM, SP1, SP2, etc.).
• Ensure that none of your custom actions access the original source location.
• Ensure that the ResolveSource action is conditionalized so that it only runs when needed, or alternatively is not present at all.
• Populate the MsiFileHash Table for all unversioned files. Msifiler.exe can easily do this for you.
• Ensure that all files have the correct version and language information. Msifiler.exe can easily do this for you.

Rule 43: Think carefully about Merge Module Patching


Merge modules provide a standard method for delivering Windows Installer components and setup logic just like any redistributable software. For all practical purposes any MSI package can consume a merge module and start using the functionality exposed by the merge module. However, the servicing story of such an MSI package has a huge flaw. Since the setup author who owns the MSI doesn’t own the merge module, they have no way to deliver fixes to issues that are found in the merge module, unless the merge module owner makes them available. On the other hand, the owner of the merge module has no way of delivering the fixes directly to the clients who installed the MSI built after consuming his/her merge module. The bottom-line of the story here is:
• Do not consume merge modules of vendors who do not promise to fix their merge modules promptly when bugs arrive
• Be prepared to handle the heat when bugs are found in your merge module causing issues for others’ products that have consumed your merge module and you get to put out the flame
Merge modules are great to use for distributing authoring and building of your setup. However, if you intend to redistribute an MSM externally, you should review the servicing implications. If you can tightly control the MSM consumers and identify the relevant products, then it’s possible to service them with a single MSP (multi-product targeting). Otherwise the recommendation is to provide your technology as an MSI that must be chained into the setup

Rule 44: Avoid Patching Aministrative Installs


Administrators are strongly advised to keep administrative install points as pristine images. Here are a summary of problems you can encounter if you decide to use administrative patching:
• Deployment latency. Once the admin image is updated a reinstall command must be invoked on all clients. Before this occurs clients remain in the infamous client/server out of sync state where they can not utilize the admin image.
• Lack of Inventory. Applying a patch to an admin image does not stamp the image with information about the patch. It simply updates the files and MSI. Thus clients have no way to determine and report on which updates they’ve consumed from admin images.
• No Undo Support. Reverting an admin patch from the admin image is not supported by Windows Installer.
• Patch Mixing Issues. Since there is a lack of information on what patches were applied on the admin image, sequencing patches applied on the admin image and those applied on the client side is impossible. Also, the availability of a previous version file is required for binary difference patches to work. But, when a client is synced up to an image that has been admin patched, most binary patch applications fail since, most binary patches target the RTM image.
• Performance. The admin patching model suffers from higher bandwidth usage and lack of change delta information. Bandwidth is increased because an entire new MSI package must be downloaded as well as individual source files, increasing both overall download size and latency as compared to a client patch. Precise change delta information is missing because the admin image is modified in place, forcing clients to perform brute force reinstalls to ensure they are brought up to date.

Rule 45: Use Least Privileged User Account (LUA) Patching if Possible


LUA patching enables you to identify digitally-signed patches that can be applied in the future by non-administrator users, provided the following conditions are met:
• The application was installed on Windows XP using Windows Installer 3.0 or later
• The application was originally installed per-machine
• The application is installed from removable media, such a CD-ROM or DVD disk. This requirement has been removed in Windows Vista.
• The MsiPatchCertificate table is present and populated in the original package
• Patches are digitally signed by a certificate listed in the MsiPatchCertificate table
• Patches can be validated against the digital signature
• LUA patching has not been disabled by setting the MSIDISABLELUAPATCHING property or the DisableLUAPatching policy
The limitation that the original source location must be removable media limits the usefulness of this in a corporate environment and it is really aimed at home users who are patching software they installed from a retail CD. However, if this applies to you, then using this method gives you more flexibility in your patching.

Rule 46: Use the Correct Type of Patch for the Job


The Windows Installer supports a number of classes of patch, each suited for a different task. Use the appropriate one for a given situation. The types are:
• Small Update. These change one or more application files that are too minor to warrant changing the product code. A small update is also commonly referred to as a hotfix.
• Minor Upgrade. These make changes to many resources. A minor upgrade can be used to add new features and components but cannot reorganize the feature-component tree. Minor upgrades provide product differentiation without actually defining a different product. A typical minor upgrade includes all fixes in previous small updates combined into a patch. A minor upgrade is also commonly referred to as a service pack (SP) update.
• Major Upgrade. These are comprehensive updates of a product that needs a change of the ProductCode Property. A typical major upgrade removes a previous version of an application and installs a new version.
There are different limitations and requirements for each patch type. Refer to the SDK for details.

Rule 47: Let the patch authoring environment handle sequencing if possible


In most cases, patches for a product fall in a single patch family, and the sequence values in that patch family increase over time. This design idiom is so common that most authoring environments that support patch sequencing automatically create sequencing metadata that fits this design.

Rule 48: Use a single PatchFamily per product unless specific requirements exist for more than one family


For most products, a single patch family provides enough flexibility to sequence patches. When multiple patch families are used, the complexity of authoring increases. If the additional functionality provided by multiple patch families is not required, you can avoid the unnecessary authoring work.

Rule 49: Follow the guidelines for patch families whenever possible


The guidelines for defining patch families are designed to reduce the complexity of patch authoring and patch management. Following the patch guidelines results in fewer authoring errors and simpler sequencing behavior.

Rule 50: Use meaningful names for patch families

Because patch family names have no meaning except as unique identifiers, use a meaningful name that indicates something about the servicing model (whether it is the target product identity, the updated functionality, the company name, or something meaningful to the patch author.)

Rule 51: Always skip numbers in the fourth field


By skipping numbers in the fourth field between consecutive patches, future patches can be sequenced between existing patches. If patch A is 1.2.3.100 and patch B is sequenced with 1.2.3.200, a future patch could, if required, be sequenced as 1.2.3.150. Using fewer than four fields in the sequence number also ensures that space exists between two consecutive patches.

Rule 52: Do not try to extract data from a patch sequence number


Although sequence numbers may be based on other data about the patch or target product, the algorithm used to generate the sequence number varies from authoring environment to authoring environment. In other cases, the algorithm may be ignored completely to achieve the desired sequencing behavior. Attempting to extract data from a sequence number may result in invalid data.
Testing and Support

Rule 53: Test thoroughly
It is crucial that you test your packages thoroughly before deployment. A classic mistake is for developers and package authors to test only on their own systems where they have full administrator rights and then discover that normal users cannot use their applications. You may want to divide your testing into these phases:

Installation Testing
Test your package under different installation scenarios, including:
• Full user input and silent installation
• Different combinations of selected features
• Different user roles (e.g. normal user, administrator, etc.)
• Different deployment methods (e.g. manual, SMS, GPO, etc.)
• Administrative and advertised installations
Enable installer logging for each test and examine the system event logs for errors after each run. Make sure that all installation initialisation and run-time errors are resolved.

An important, but often overlooked, part of installation testing is to ensure that your setup UI is accessible and can handle different screen resolutions and font sizes.

Patching and Repair
As mentioned in the Patching section, it is very important that you think about patching before deployment. In fact you should not only think about it but test it thoroughly. You should include these scenarios:
• Small update
• Minor upgrade
• Major upgrade
• Product Repair
• Multiple simultaneous patch installs and uninstalls
• Installing and uninstalling patches "out of order"
Uninstall Testing
Many operating system related errors caused by a repackaged application happen during an uninstall session. Therefore it is extremely important to verify that a generated package does not uninstall “too much” and only uninstalls data that belongs to the package itself and not to the operating system or other applications.

Test uninstall as thoroughly as install, including:
• Uninstall under different user contexts (e.g. normal user and administrator)
• Uninstall via different methods (e.g. Control Panel, command line, etc.)
• Make sure that Operating System does not report any integrity problems
• Reboot a test machine after uninstall (even if the uninstall did not require a reboot)
• Verify integrity of common system tools and other standard applications
As with install, use Installer logs and event logs to look for errors, and make sure you resolve all those you find.

Product Functionality Test
A successful installation and uninstall sessions can be insufficient for an application to work correctly. Therefore you need to ensure that the application works as expected. Some things to check:
• Verify each shortcut in the package starts the corresponding application correctly
• Walk through main menu items in the application
• Follow the application functionality test provided by the vendor, if any
• Verify that file associations by launching a file for each registered extension.
• Make sure that application opens, save and prints the files correctly
It is recommended that corporations provide application functionality tests allowing a repackaging engineer to quickly verify major application functionality right away, and make sure that a developed MSI package correctly installs an application, and an application is ready to work properly.

User Acceptance Test (UAT)
Just as important, but often overlooked, is user input into the testing strategy. User Acceptance Testing (UAT) is especially important when re-packaging legacy applications to MSI format.

Some points to note for UAT:
• Application should be deployed for UAT in exactly same manner as it will be deployed in the production environment
• Pilot the deployment with a selection of “typical” users for the application
• Make it easy for users to report problems and provide feedback
• For internal applications, the owning Business Unit must ensure each application passes the UAT to be defined by that Business Unit. Where the package is processed externally an amount of time must be agreed upon, after which it is reasonable to assume the package has been accepted into production
• It is recommended that product functionality tests be provided for all internal applications. Such tests will allow a packaging engineer to verify major application functionality before submitting a migrated package for UAT

Rule 54: Validate Your Packages

Validation is a means by which you can run certain standard checks (called ICEs) against your package to check for inconsistencies and non-compliance with Installer rules.

Many ICEs ship with the Installer SDK in .CUB files and you can write your own. They can be run against your packages using the ORCA or MsiVal2.exe tools from the SDK.

You should always fully validate your packages and, much like when working with application development, correct any reported errors and treat all warnings seriously.

Validation is also a useful troubleshooting technique, particularly if you are working with 3-rd party packages.

Rule 55: Use Installer Logging

As mentioned previously, the Installer supports extensive logging. This is by far the best tool for troubleshooting Installer issues and the time spent understanding the log files will be repaid many times over when problems arise later. At the least logging should be used throughout testing.

You are certain to be asked to provide verbose logs if you contact Microsoft support, so either have these ready or be prepared to reproduce your problem to gather some.

Rule 56: Use Virtual Environments for Testing and Support

Using virtual machines (VMs) for testing rather than real systems has a couple of advantages:
• You can build a VM for each of the different systems in your environment and store these on a single server or workstation to be brought on-line when needed
• With “undo”-style features of VMs you can easily revert to a clean, known state after each test
• The same VM images can be used by testing and helpdesk staff
However, do note that if you think you have discovered a bug in the Installer you may be required to reproduce the problem on real hardware when you contact Microsoft support:

Security Considerations

Rule 57: Install Managed Applications into a Secure Installation Folder

A “secure installation folder” is one for which non-admin users do not have change or modify permissions. There is little point in securing the installation with Group Policy, Software Restriction Policies, Digital Signing, etc., if the user can simply change the application files after install.

You should also secure all of the applications registry data in a similar way.

Rule 58: Be careful When using Properties for Passwords or Other Sensitive Information


The installer may write the value of a property authored into the Property table, or created at run time, into a log or the system registry, which is obviously not good security for things such as passwords. Ideally, avoid using properties in this way, but if you must, then use the MsiHiddenProperties property to makes sure the information is not logged.

Rule 59: Use Restricted Public Properties to Restrict the Public Properties a User Can Change.

In the case of a managed installation, the package author may need to limit which public properties are passed to the server side and can be changed by a user that is not a system administrator. Some restrictions are commonly necessary to maintain a secure environment when the installation requires the installer to use elevated privileges. If all of the following conditions are met, a user that is not a system administrator can only override an approved list of restricted public properties:
• The system is Windows 2000 or later
• The user is not a system administrator
• The application or product is being installed with elevated privileges


Rule 60: Avoid Installing Services That Impersonate the Privileges of a Particular User

This may write security data into a log or the system registry. This creates potential for a security problem, password conflict, or the loss of configuration data when the system is restarted.

Rule 61: Use the LockPermissions Table to Secure Files, Registry Keys, etc.

Every file, registry key, or directory that is listed in the LockPermissions Table receives an explicit security descriptor, whether it replaces an existing object or not. The Windows Installer attempts to preserve the security on objects that already exist on the system, but has these defaults, which may not provide a secure installation:
• If an object is not listed in the LockPermissions table, and replaces an existing object, the replacement gets the security settings of the object that it replaces.
• If an object is not listed in the LockPermissions table, and does not replace an existing object, it receives no explicit security descriptor. The access to the new object is based on the attributes of its parent or container object.
It is recommended that the system administrator’s local group be included in all access control list. This ensures that the system administrator can access and maintain objects.

Rule 62: Add a Digital Signatures to the Installation to Ensure the Integrity of the Files

The Windows Installer version 2.0 and later uses digital signatures to detect corrupted resources. A signer certificate may be compared to the signer certificate of an external resource to be installed by the package to ensure its integrity.

Digital signatures can be used with packages, transforms, patches, merge modules, and external cabinet files.

Rule 63: Fail Securely When the User is Denied Access to Resources

Author your package such that if the user is denied access to resources, the setup fails in a manner that maintains a secure environment. Check the user’s access privileges and determine whether there is sufficient disk space before installation begins. Commonly, the installer should only display a browse dialogue box if the current user is an administrator or if the installation does not require elevated privileges.

Rule 64: Use Secured Transforms to Store Transforms Securely on the Local System


Secured transforms are stored locally on the user’s computer in a location where, on a secure file system, the user does not have write access. Such transforms are cached in this location during the installation or advertisement of the package. Only administrators and local system have write access to this location. A non-admin user would not be able to modify the transform file. During subsequent installation-on-demand or maintenance installations of the package, the installer uses the cached transforms for increased security.

You can enable the use of Secured transforms on the command-line, using Group Policy or by setting the TRANSFORMSSECURE property. See the SDK for details.

Rule 65: Use the Security Summary Property to Indicate Whether the Package Should be Opened as Read-only

This property should be set to read-only recommended for an installation database and to read-only enforced for a transform or patch. Database editing tool should not modify a read-only enforced database and should issue a warning when an attempt is made to modify a read-only recommended database.

Rule 66: Use the DisablePatch and AllowLockdownPatch Policies to Provide Security in Environments Where Patching Must be Restricted.

DisablePatch is a per-machine policy that prevents the installer from installing patches. This policy can be used in high security environments where patching must be restricted.

AllowLockdownPatch is similar but still allows administrators to patch existing products that were installed using elevated privileges.

Rule 67: Make Custom Actions Secure

The installer runs custom actions with user privileges by default in order to limit the access of custom actions to the system. The installer may run custom actions with elevated privileges if a managed application is being installed or if the system policy has been specified for elevated privileges. To ensure better security, follow these guidelines:
• Secure any additional files written by your custom action
• Check buffer lengths and validity of all data read by your custom action. This includes properties that may supply data to your custom action, particularly those that use public properties provided by a user
• Do not rely on external DLLs that are not trusted by the system on all platforms on which your installation package is intended to run
• Carefully consider whether to use custom actions that use elevated privileges or impersonation. If your custom action must run with elevated privileges, be sure that the custom action code guards against buffer overruns and inadvertent loading of unsafe code. Note that during the execution phase of the installation, the installer passes information to a process with elevated privileges and runs the script. Any custom actions that run during the execution phase may run with elevated privileges
• Gather all information provided by the user during the UI sequence. Do not prompt the user for any information that can’t be set using a public property. If your script custom action expands properties, take precautions that the custom action is secured against the possibility of script injection. Scripts may be logged in clear text

Rule 68: Test Your Packages For Locked-down Environments

In corporate deployment scenarios, restricted functionality, rights and permissions are the norm. You should make sure that your packages deploy properly in such situations. These basic guidelines will help ensure that your packages work in a locked-down environment:
• Test your package for compatibility with the Windows Installer machine System Policy
• Make sure you package runs with all user interface levels, none, basic, limited, and full
• Test your package on NTFS partitions, both with elevated and non-elevated privileges
• Test your package with different user roles. That is, make sure it installs and works correctly for normal users as well as highly privileged users
Ideally, you should test your packages on a test platform that exactly matches your live environment in terms of permissions, policies and suer rights.

Rule 69: Use Software Restriction Policies to Restrict Package Execution

Software Restriction Policy (SRP) is a mechanism introduced in Windows XP that allows administrators to restrict the execution of applications based on various criteria such as the file hash, path, URL zone and publisher. The Installer is fully compliant with SRP and you can use it to restrict the execution of MSI packages, patches and transforms.

If a package, patch, or transform is restricted, the Windows Installer displays an error message and logs an entry in the application event log. Software restriction policy is evaluated the first time an application is installed, when a new patch is applied, and when the installation package is re-cached.




Inside the MSI file format.
I've sat down to write this blog no less than four times in the last week. Each time something has come up that has pulled me away from actually getting far enough into writing that it becomes basically self-propelled. Now tonight, I know there is at least one person out there focused on getting her homework done so I thought I'd buckle down and plow through a bit of writing myself.
Let's talk about MSI files. First, MSI files got their name back when we thought Darwin was going to be called the "Microsoft Installer". Thus the file extension MSI made some sense. Unfortunately, it so late when the name changed to "Windows Installer" that it wasn't really feasible to safely change the extension that everyone had come to know and love as MSI.
Anyway, the vision driving development of Darwin was that setup needed to be a transacted set of changes to a target machine that could be aborted and cleaned up if an error occurred or the user cancelled setup. This means the setup logic must be declarative so that an engine can interpret the logic and calculate not only the changes to the target but the changes necessary to undo any of those changes should something go wrong. There are many ways to define data declaratively (XML being my personal favourite these days) but back around 1995 (when Darwin was first started) the team decided the setup logic should be in a database. Unfortunately, all of the database technologies back then required substantial amounts of setup before they could be used. Since a setup technology is kinda' needed before you can setup anything, it wasn't really feasible to use any of database engines that existed. Think classic chicken and egg problem.
So, the Darwin team decided to build a custom relational database. As an aside, in my humble opinion, building this custom relational database to store all the setup logic was unnecessary and generated a lot of overhead over the years (especially for those of us that have to create the flipping MSI files). However, my opinion is based on hindsight and we all know we see better when looking back on history. Anyway, I just wanted to be up front that I can't provide a really strong justification for why MSI files had to be relational databases.
Okay, so say you're in the middle of the 1990's and you need to build a relational database, what do you do? Well, if you're in Office (like the Darwin team was at the time) and you look at the Word and Excel file formats you migh think, "Hey, those structured storage file thingies are really cool! I bet we could use that!"
So, MSI files are actually little databases laid out in a structured storage file. For those of you that haven't played with structured storage files let me talk about them a little. A structured storage file exists on disk as a single file but can contain many "streams" and/or "sub-storages". Streams are essentially just a bunch of bits with a name stored inside a structured storage file. Sub-storages are just structured storage files embedded in another structured storage files. I've seen people compare structured storage files to typical file systems where "files" map to "streams" and "directories" map to "sub-storages". Structured storage files are also often called "compound documents" or sometimes "OLE documents".
There are a few advantages to using structured storage files as the basis for your file format. First, the format provides a very natural way to separate your data with the streams and sub-storages. The MSI file uses separate streams for each of the tables in the database. Second, you can store multiple files in a single structured file which is nice when you want to have a single redistributable. For example, streams are used to store things like UI graphics, CustomAction DLLs, and even the binaries to be installed in many cases. Also, sub-storages are used to nest one MSI file inside another MSI file (note: you should never do this, but I'll talk about nested installs another day). Finally, structured storage files have built in transaction semantics. Having someone else provide the transaction functionality for you is really nice when you're trying to build a database on top of the format.
There are also a few disadvantages to structured storage files. First, the names of streams can only be something around 63 characters. This restriction isn't particularly restrictive but it can cause some really wacky error messages. Second, structured storage files don't shrink. If you add then delete data to a structured storage file, the file maintains its largest size. This design works out okay if you consider the case where a user is writing a document. In those cases, the user spends most of the time adding data and any deletes are often replaced with more data. Editing MSI files does not necessarily follow the same pattern so it is possible to end up with bloated MSI files if you're not careful. Finally, structured storage files don't handle multiple writers well at all. For example, open an MSI file in Orca then try to install the MSI by double clicking on it. You'll get a lovely message box that says something like:
This installation package could not be opened. Verify that the package exists and that you can access it, or contact the application vendor to verify that this is a valid Windows Installer package. [OK]
Okay? No, not okay but whatever. Every time I see that message box I wonder how many hours have been lost trying to figure out what the heck is wrong with an MSI file only to find that it was held open for editing in Orca. K, a buddy of mine at work, was just about pulling his hair out one day trying to figure out what was going wrong with one of his MSI files until I pointed out that he had Orca editing the file on one of his other test machines.
Anyway, there are a couple other things I want to say about the MSI file format.
In the mid-1990's Microsoft was still shipping Office on 3.5" floppies. Granted Office '97 shipped on something like 39 floppy disks but CD-ROMs weren't quite popular enough (i.e. weren't cheap enough). So one of the things the Darwin team needed to do was make the MSI files as small as possible so that the setup logic would fit on a single floppy disk (trying to read a structured storage file the spanned multiple floppy disks was not an option). This need led to the creation of the "string pool" and many dreaded "string pool corruption" bugs.
More detail. If you're familiar with relational databases, you know that primary key identifiers are duplicated everywhere you have a foreign key reference. Well, primary key identifiers in MSI are strings that are recommended to be 72 or less characters long. It's not hard to imagine how quickly all those identifiers could add up to create unnecessarily large MSI files. To combat this bloat there is a single stream in the MSI file that holds all the strings. This stream is calle the string pool contains a single entry for each unique string. That way a string column in a table is just an integer offset into the string pool.
The string pool can save quite a bit of space. It was also pretty tricky to get right. I wasn't directly involved, but I remember quite a few late night bugs when I was an intern where my mentor spent the whole night tracking down why the wrong string or a corrupt string was coming out of the string pool. Then there were the nights trying to figure out why localized strings were coming out corrupt. Anyway, if ever come across a copy of the original msival.exe you'll see a command-line switch that would run tests to detect string pool corruption. Fortunately, the string pool code is stable now and that isn't necessary any more.
On the note of localized strings, I should note that the MSI file format is not Unicode. I'm not an expert on localization and there is a pretty detailed topic in MSDN about localizing MSI files so I'm not going to say much more. Just keep in mind that you have to deal with codepages when storing localized strings in a MSI file.