Sajjadi Consulting
  • About
  • Services
    • Quality Consulting
    • Professional Development
    • Speaking
  • Events
  • Blog
  • Contact
  • Search
  • Menu Menu
 
You are here: Home1 » Blog2 » Quality Quarterly3 » Total Analytical Error (TAE) from Concept to Calculations

Total Analytical Error (TAE) from Concept to Calculations

Quality Quarterly March 1, 2023

Do you struggle with understanding statistical concepts applicable to your role and responsibilities in the development of biologics?

Are you aware of the trend toward Bayesian statistical inference and the shift away from the frequentist approach?

Because I often hear “yes” and “no” respectively to the questions above, I want to explore analytical error as a timely topic. It is highly relevant to the issuance earlier this year of ICH Q14 Analytical procedure development | European Medicines Agency and the introduction of the term “Total Analytical Error” as an “alternative approach to individual assessment of accuracy and precision.”

Without including a specific calculation, interpreting existing assay validation guidance (and understanding regulatory reviewers’ expectations in reporting it), has been confusing. This fall issue of the Quality Quarterly is intended to: (1) raise awareness and (2) offer some clarity to offset (and validate!) any confusion you might be experiencing.

Bench scientists intuitively understand that the accuracy and precision of a test method define its “resolving power”- the ability to discriminate between samples with different amounts of the analyte being measured.

Most are familiar with existing guidance such as: 

  • ICH Q2(R2) Validation of analytical procedures | European Medicines Agency
  • 〈1033〉 Biological Assay Validation

These documents are useful tools for those tasked with designing studies to measure accuracy and precision, but they do not currently use the term “total analytical error.”However, the glossary in the current Bioanalytical Method Validation Guidance for Industry | FDA (BMV) does define total error for a reportable measurement and provides a general calculation:

“Total error is the sum of the absolute value of the errors in accuracy (%) and precision (%). Total error is reported as percent (%) error.”
Elsewhere in the BMV guidance, it specifies that the total error value (%) is derived by adding the % coefficient of variation (%CV) and the % relative error (RE; bias). When I first encountered this equation, I automatically expressed the equation using the corresponding ratios and found it hard to see that the terms add up algebraically given they do not share a common denominator:
Standard Deviation/Measured Mean +
|Measured Mean-Expected Value|
/ Expected Value
Turning to Google and searching the words “total error assay” for clarification, I discovered these types of equations:
Total analytical error = bias + (1.65 x imprecision)
Total Error = %BIAS + (1.96 * %CV)
These equations are similar to those provided in the BMV guidance but they include a “z value multiplier” to the variability term. A further Google search using “variability bias and total error” brings up equations like this:
Total Error = Bias² + Variance + irreducible error
And the figure below from Total error and measurement uncertainty – Finbiosoft:

The latter appears to be using Pythagoras’s theorem to “prove” the addition of Bias2 and variance (standard deviation2 ;SD2) to arrive at total error.  Note that the equation above the hypotenuse in the diagram squares the bias and standard deviation (SD) terms before adding them and then returning to them to their original units of measure by taking the square root.  If this is indeed true, then the BAV guidance equation cannot also be true.

However, you might recognize the addition of bias2 and variance (SD2) as part of the denominator in the Cpm equation referenced in the current USP<1033> chapter. In the Cpm equation, the total measurement uncertainty of a reported value is combined with process variance in the denominator and related to the width of a product specification (USL-LSL) in the numerator. From this value, the probability of an out-of-specification (OOS) result can be derived.

By working backward from a maximum acceptable OOS rate (i.e., manufacturer’s risk tolerance) and an estimated process variance term, the limits on assay precision and bias terms can be established and validation results reported per ICH Q2(R2) Validation of analytical procedures | European Medicines Agency (europa.eu) using a frequentist approach (as evidenced by an expectation of confidence intervals)

  • Precision: The standard deviation, relative standard deviation (coefficient of variation) and confidence interval should be reported for each type of precision investigated and be compatible with the specification limits.
  • Accuracy: An appropriate confidence interval (e.g., 95%) for the mean percent recovery or the difference between the mean and accepted true value (as appropriate) should be compared to the acceptance criterion to evaluate analytical procedure bias. The appropriateness of the confidence interval should be justified.

Complicating matters slightly has been a shift towards implementing a tolerance interval approach to setting lot release specification limits.  This is where the Bayesian inference approach comes in. It establishes limits based on the probability of future results falling within specification limits. It requires another set of concepts and equations that are described in USP <1210> “Statistical Tools for Procedure Validation”.

Statisticians from the FDA have recently co-authored an open-access paper published in AAPS which spells out the advantage of a Bayesian approach over a frequentist approach:  “Comparing a Bayesian Approach (BEST) with the Two One-Sided t-Tests (TOSTs) for Bioequivalence Studies.” AAPS J 24, 97 (2022).

I learned of this paper from a post on LinkedIn and invited clarification on what appears to me, to be a hybrid approach (a Bayesian interval with frequentist bounds?)  There are different camps among Bayesian approach advocates, so I am waiting for more clarity. In the meantime, this open-access paper sent to me by consultant statistician Dr. Janice Callahan, is a good primer on the subject of Bayes or not Bayes, is this the question? (nih.gov).

Although I am not a biostatistician, I have had countless conversations with statistical colleagues over the past 30 years to get me to my level of understanding about TAE. I am still learning about Bayesian ideas and applications and hoping for clear consensus and communication if regulatory expectations shift toward them.

Share this entry
  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn
  • Share by Mail
https://sajjadiconsulting.com/wp-content/uploads/2023/03/26dba071-07fe-881c-40c6-a621742a44dd.png 564 1128 Jill Kurtz https://sajjadiconsulting.com/wp-content/uploads/2021/08/sajjadi-logo-web.png Jill Kurtz2023-03-01 09:42:282023-03-01 09:42:28Total Analytical Error (TAE) from Concept to Calculations

Categories

  • Interviews
  • Presentations and Panels
  • Publications
  • Quality Quarterly

Archives

  • March 2023
  • November 2022
  • September 2021
  • June 2021
  • March 2021
  • October 2020
  • October 2019
  • April 2019
  • August 2018
  • March 2005
  • September 2003
  • January 2003
  • December 2000
  • September 2000
  • September 1997
  • July 1997
  • February 1996
  • March 1995
  • June 1994
Sajjadi Consulting Logo

Email:   nancy@sajjadiconsulting.com
Phone: 703-727-9195

LinkedIn logo

Stay in touch by signing up for our email list. Updates will be sent quarterly. Your information will never be shared or sold.

Privacy Policy

Copyright @ 2023 Sajjadi Consulting • All rights reserved.

Developments in BioassaysNancy Sajjadi
Scroll to top

This site uses cookies to improve your browsing experience. The website may not work as expected without them. By continuing to browse this site, you consent to the use of cookies.

Accept settingsHide notification only

Cookie and Privacy Settings



How we use cookies

We may request cookies to be set on your device. We use cookies to let us know when you visit our websites, how you interact with us, to enrich your user experience, and to customize your relationship with our website.

Click on the different category headings to find out more. You can also change some of your preferences. Note that blocking some types of cookies may impact your experience on our websites and the services we are able to offer.

Essential Website Cookies

These cookies are strictly necessary to provide you with services available through our website and to use some of its features.

Because these cookies are strictly necessary to deliver the website, refusing them will have impact how our site functions. You always can block or delete cookies by changing your browser settings and force blocking all cookies on this website. But this will always prompt you to accept/refuse cookies when revisiting our site.

We fully respect if you want to refuse cookies but to avoid asking you again and again kindly allow us to store a cookie for that. You are free to opt out any time or opt in for other cookies to get a better experience. If you refuse cookies we will remove all set cookies in our domain.

We provide you with a list of stored cookies on your computer in our domain so you can check what we stored. Due to security reasons we are not able to show or modify cookies from other domains. You can check these in your browser security settings.

Other external services

We also use different external services like Google Webfonts, Google Maps, and external Video providers. Since these providers may collect personal data like your IP address we allow you to block them here. Please be aware that this might heavily reduce the functionality and appearance of our site. Changes will take effect once you reload the page.

Google Webfont Settings:

Google Map Settings:

Google reCaptcha Settings:

Vimeo and Youtube video embeds:

Privacy Policy

You can read about our cookies and privacy settings in detail on our Privacy Policy Page.

Privacy Policy
Accept settingsHide notification only