With the release of the Computer Software Assurance for Production and Quality System Software by the FDA, we get a clear framework to apply for spreadsheet validation.
Spreadsheets are indispensable tools across the pharmaceutical industries, particularly in manufacturing and quality control. They serve as critical systems for collecting, analyzing, and visualizing data, which directly impacts product quality and safety. Especially when they are integral to monitoring processes like tracking nonconformances.
Understanding Spreadsheet Validation
Spreadsheet validation refers to the process of ensuring that a spreadsheet functions as intended, producing accurate and reliable results consistently. It involves testing various aspects such as data input, calculations, formulas, and outputs to confirm they meet predefined requirements and specifications. And, let’s not forget that if it use as to generate some electronic record, the validation should verify the Accuracy, Reliability, Integrity, Availability, and Authenticity of the record.
This process is crucial in the industries, where data integrity and accuracy are non-negotiable, especially in quality systems.
Importance of Spreadsheet Validation
The importance of spreadsheet validation cannot be overstated. Errors or inconsistencies in spreadsheets can lead to flawed decision-making, impacting product quality, safety, and regulatory compliance. For instance, a malfunctioning spreadsheet used for monitoring nonconformances might overlook critical issues, leading to the release of defective products.
Framework for Spreadsheet Validation
1. Define Intended Use
As the FDA regulation 21 CFR 820.70(i) mandates that the industries validate software (encompassing spreadsheet) used within production or quality systems based on its intended use, defining it properly is key. It should clearly outline the purpose and scope of the spreadsheet and assesse if it use directly or to support the production or the quality system.
Intended Use, User and functions :
To properly identify the intended use category, it should clealy outline the users and functions of the spreadsheets.
- Identify Users:
- Determine who will use the spreadsheet (e.g., quality control personnel, QA, etc..).
- Define Functions:
- Outline what the spreadsheet needs to do (e.g., collect, analyze, display data, etc…).
Intended Use, Categories:
Once the user and the functions is clearly defined, it should be determined in the Intended use if the spreadsheet fall in one of this two categories:
- Direct Use:
- Spreadsheets used for data collection, processing, and tracking metrics or performing critical calculations in production or quality systems require validation.
- Supporting Use:
- Spreadsheets aiding in testing or monitoring other software tools fall under supporting use,
often carrying lower risk but still necessitating validation based on a risk-based approach.
- Spreadsheets aiding in testing or monitoring other software tools fall under supporting use,
In short, spreadsheets used directly in production or quality processes must be fully validated. Those supporting other software may have reduced validation efforts under a risk-based approach, ensuring safety and compliance without unnecessary burdens.
2. Conduct Risk-Based Analysis
The FDA recommends using a risk-based analysis to determine appropriate assurance activities for spreadsheet used in manufacturing or quality systems. This approach involves:
Identifying Spreadsheet Risks:
The goal is to assess whether the Spreadsheet feature/function’s potential failures could lead to quality problems that compromise safety of the product or its environment.
Categorizing Process Risk:
Each risk identified previously need to be categorize.
- High Process Risk: Spreadsheet features/functions that, if failed, could result in a quality problem
compromising safety.- Examples include:
- Maintaining critical process parameters affecting product properties or manufacturing processes essential to device safety.
- Measuring/analyzing product/process data with limited human review.
- Automating adjustments to process parameters without human awareness.
- Producing instructions for use or labeling necessary for safe device operation.
- Automating surveillance/trending of data essential to device safety (e.g., cybersecurity).
- Examples include:
- Not High Process Risk: Spreadsheet features/functions where failure would not result in a quality problem compromising safety.
- Examples include:
- Collecting and recording data for monitoring without direct impact on production.
- Supporting quality system functions like CAPA routing or automated logging of complaints.
- Managing data, automating calculations, or providing alerts in established processes.
- Examples include:
Determining Assurance Activities:
- For high process risk Spreadsheet , assurance activities should be commensurate with the medical device risk (e.g., rigorous validation and verification).
- For not high process risk Spreadsheet , assurance activities should align with the process risk (e.g., less frequent or simplified testing).
In summary, manufacturers must evaluate each Spreadsheet feature/function’s potential impact on safety and quality to determine the level of scrutiny required, ensuring that high-risk functions receive greater attention while low-risk functions follow less rigorous processes.
3. Design Validation Tests
Develop test cases that cover all functionalities specified in the intended use. Indeed for a Spreadsheet validation, the intended use is kind of an URS for a “classic” computer system validation in the V cycle.
Types of testing that may be considered:
- Functional Testing: Ensure all features work as expected (create, read, update, delete).
- Data Integrity Checks: Verify data accuracy and consistency across inputs and outputs.
- Error Handling: Test how the spreadsheet handles unexpected inputs or errors.
These testing might be broadly categorized into unscripted and scripted testing :
- Unscripted Testing (exploratory testing): This involves dynamic testing where testers do not follow predefined scripts but instead improvise based on their understanding and experience.
- Scenario Testing (Ad-Hoc Testing): A specification-based method where testers create sequences of interactions between the Spreadsheet and other systems, simulating user behavior in interactions with external systems.
- Experience-Based Testing: Leverages testers’ knowledge and expertise to identify potential issues through techniques like test attacks, tours, and error taxonomies, focusing on areas such as security and performance.
- Scripted Testing: Involves creating detailed test cases that can be executed manually or automatically using tools. This method is structured and crucial for high-risk scenarios where detailed documentation and repeatability are essential.
In summary, unscripted testing offers flexibility and leverages tester expertise to uncover hidden issues, while scripted testing provides structure and detail necessary for managing risks effectively in critical software systems.
4. Execute Validation Tests
Perform both scripted and exploratory testing to thoroughly validate the spreadsheet.
- Scripted Testing: Use predefined test scripts to systematically check functionalities.
- Unscripted Testing: Manually explore the spreadsheet to uncover potential issues that scripted tests might miss.
5. Address Deviations/incidents
Document any deviations encountered during testing and implement corrective actions.
- Identify Deviations/incident: Note any anomalies or failures observed during testing.
- Root Cause Analysis: Investigate the root cause of each deviation.
- Corrective Actions: Implement fixes to resolve identified issues, such as adding validation rules in the spreadsheet.
6. Document Validation Results
Compile and document all findings, including test results, deviations, and corrective actions taken.
- Documentation Template: Use a structured format to record details like intended use, risk analysis, tested version, test type, objectives, testing activities, deviations, conclusion, date, and tester.
- Comprehensive Record: Ensure the documentation is thorough and accessible for future reference or audits.
7. Maintain Validated State
After validation, monitor the spreadsheet to ensure ongoing compliance with its intended use.
- Periodic Revalidation: Revalidate periodically after system changes or updates.
- User Training: Train users on proper spreadsheet usage to prevent errors.
- Continuous Monitoring: Continuously track performance and address any emerging issues.
Benefits of Spreadsheet Validation
Implementing a robust spreadsheet validation framework offers several benefits:
- Data Integrity: Ensures data accuracy and reliability.
- Compliance: Meets regulatory requirements and industry standards.
- Risk Mitigation: Reduces the risk of errors impacting product quality and safety.
- User Confidence: Enhances trust in the spreadsheet’s outputs, leading to informed decision-making.
Challenges and Considerations
While beneficial, spreadsheet validation comes with challenges:
- Resource Intensive: Requires significant time and resources, especially for complex spreadsheets.
- Dynamic Environment: Changes in business processes or system updates necessitate periodic revalidation.
- User Training: Ensuring users understand proper usage can be challenging.
Conclusion
Spreadsheet validation is a vital process to ensure the reliability and accuracy of spreadsheet outputs. By
following this structured framework, organizations can mitigate risks, enhance compliance, and maintain data integrity, ultimately leading to improved product quality and safety. Regularly updating and maintaining validated spreadsheets ensures they continue to meet their intended purposes effectively.
