MSF programs may look alike. But often the case, they differ substantially. Some are Quality Improvement driven while others are Quality Assurance driven. Some only serve as complementary to peer assessment programs.

MSF programs are used in a variety of ways. Some regulatory authorities only review results that fall below set criteria; otherwise reviewees are asked to reflect on their data and make their own decisions regarding practice changes. Other authorities have developed integrated assessment models that combine MSF results with data from other assessments. Both are quality assurance initiatives.

Medical chiefs in hospital-based programs typically review results with the reviewee, provide coaching, and set goals for change as required. In this instance, MSF data has performance measurement objectives.

MSF/360 programs used by medical regulatory authorities that are based on the Alberta’s Physician Achievement Review program require surveys from a minimum of 18 patients, 6 medical colleagues and 6 co-workers.

However, each MSF/360 program is unique and so are the required numbers of completed responses.

Reviewers can be your colleagues, medical peers, patients, parents/guardians of patients, or co-workers in other healthcare fields. Depending on the program, instructions are provided to the participant to ensure they select the most suitable individuals.

Reviewers typically include colleagues, co-workers and patients/clients or parents/guardians of patients. Reviewers must be able to offer an informed opinion. In some programs, the reviewee selects the reviewers while in others the reviewers are selected by program heads or administrators. Provided there are a sufficient number of reviewers there is no evidence of selection bias in either case.

Survey instruments are developed independently or collaboratively by regulatory authorities, academic institutions and research firms like Pivotal Research. Regardless, the construction and validation of instruments must meet accepted scientific practice.

Norms are useful because they provide a context for interpreting results. Suppose that a person receives an average rating of 7 out of 10. Is that rating average or above or below average? The answer is contingent on the normative distribution. If the group average is 6, a score of 7 is favorable. If the group average is 9, then a score of 7 suggests room for improvement.

Norms are based on the performance of a population or a sample of reviewees that is large enough to ensure that the results are representative of the population. Group results are displayed in a distribution against which scores for a single reviewee can be compared.

Frequency varies widely depending upon the resources required to complete a multi-source feedback program. Some regulatory authorities have adopted a 5-year cycle, others a 7-year cycle. Some institutions have used shorter cycle times in order to measure the impact of a significant change in procedures, systems or training. In voluntary programs, the health professional can participate as often as they wish. Once every two years might be optimal to monitor progress while allowing sufficient time for practice changes to take effect.

Some institutions select reviewees within a department. Regulatory authorities use either random selection or non-random selection by focusing on a particular group or type of practice. In some programs a reviewee volunteers to participate.

Data confidentiality and security are top priorities at Pivotal Research. Our applications are programmed, monitored and maintained by our resident programmers using the ASP.NET platform. We have strong security systems in place and we are vigilant about updating security patches and monitoring event logs for suspicious activity.

Pivotal Research uses SQL Server to house data and therefore we are able to use Microsoft SQL Servers securities (i.e. database permissions). Applications are run with the fewest privileges possible. A database user with limited access and privilege is allowed to perform the necessary database queries and nothing more.

Network internet security is provided by ESET, and our Internet Information Server (IIS) is configured to use process throttling to prevent malicious attackers from bringing down the application. Using a Fortigate firewall also enhances our level of security.

Our file, email and web servers are secured, and our physical firewall is locked down. Web applications are hosted on our web server, but data gathered through the portal will be stored on our SQL server. This separation provides a further layer of security for the data. Sensitive information such as usernames, passwords, or PINs are never stored in areas that may be accessible to the user's browser. All of our servers are protected by strong passwords, and complete backups are performed daily.

Pivotal Research's web server is secure and we use our SSL (Secure Socket Layer) certificate to ensure that data is encrypted.

Our Privacy and Confidentiality Protocol ensures that you can be confident in the security and privacy of all data.

Given our extensive experience hosting sensitive and confidential data for various clients, we have developed several strategies to mitigate the effects of unforeseen failures.

All of our data systems are backed up completely on a daily basis, and the backups are stored in a safe location off-site. Backup data is encrypted and can be restored from the previous five days. We maintain redundant software and hardware in case of equipment failure, and we contract same-day technical IT support to respond to software or hardware concerns.

In case of a partial or complete system failure, the first steps are to assess the impact and notify affected clients. Once the impact has been determined, systems have been restored, and data has been recovered from the backups, Pivotal Research will work closely with affected clients to develop a recovery plan.

The functions required for this project can be performed by a minimum of two Pivotal Research staff members. This redundancy ensures cover off staff resources.

Given our experience in administering MSF/360 programs spans almost 20 years, we have developed a pricing model that is quite reasonable for clients. Fees for Pivotal Research vary depending on the nature of the program, the overall number of participants, the number of stakeholders providing feedback to participants, and the data collection methodology.