4 replies to this topic

[Helmut Schütz]

Dear all,

I updated the template for the EMA’s reference-scaling method “Average Bioequivalence with Expanding Limits – ABEL”.

New or changed:

• Supported designs:
  • Partial replicate design with sequences RRT|RTR|TRR.¹
  • Full replicate design with sequences RTRT|TRTR.
  • Full replicate design with sequences RTTR|TRRT (new).
  • Full replicate design with sequences RTR|TRT (new).
  • Full replicate design with sequences RTT|TRR (new).
• Not supported designs (and IMHO, should never be):
  • Extra-reference design with sequences RTR|TRR.
    Reason: Since T is not administered in period 3 the models will give a biased treatment estimate in the presence of period effects.
  • Balaam’s design with sequences RT|TR|RR|TT.
    Reason: Very poor power characteristics (sample size ~8× of a 4-period replicate and ~5× of a 3-period replicate). If limited sampling volume is an issue, consider one of the 3-period designs instead.
  • Full replicate design with sequences RTTR|TRRT|RRTT|TTRR. For drawbacks of this design see the FDA’s Guidance for Industry: Statistical Approaches to Establishing Bio­equi­va­lence, Appen­dix B (2001).
• The intra-subject CV(s) are reported to two digits (previously rounded to one digit as in the Q&A document) and used in full precision.
• The expanded limits above the scaling cap of CV_wR 50% are no more hard coded to 69.84–143.19% but calculated in full precision according to
    100*exp(±0.76*sqrt(ln(0.5^2+1))).
  Reason: Rounded limits are not symmetrical around 100% and lead to discontinuities…
  
  
  
  
  
  The inclusion of the 90% CI is still assessed based one the two-digits rounded percentages according to the guideline. The same holds for the GMR-restriction (within 80.00–125.00%).
• Added a data wizard Design to the sub-workflow Prepare Dataset for Analysis:
  • Checks whether the design is a partial replicate.¹
  • Checks whether the design is one of the 3-period full replicates (RTR|TRT or RTT|TRR).

#1 prevents the meaningless² CV_WT in ‘Method C’ to be shown in the Table Final Conclusion.

#2 will be used ‘downstream’ to assess whether at least 12 subjects are in sequence RTR in the RTR|TRT-design (required for a „reliable estimate of the CV_wR” according to the Q&A document Rev. 12 of June 2015). Note: In a pro­perly powered study such an outcome is not realistic anyway (more than 42% dropouts).
Although not mentioned in the Q&A document, the same assessment is per­formed for sequence TRR in the RTT|TRR-design.

• Sub-workflow Analysis|Standard Average BE|Other Methods object BE Method B log changed the Degrees of Freedom to ⦿ Residual in com­pli­ance with the Q&A – expected to be equivalent to SAS DDFM=CONTAIN (new in v1.4.3.1).
• Sub-workflow ABEL|Calculation Steps
  • For validation purposes the Data Wizard Assessment gives the degrees of freedom of the difference and the log halfwidth  (new in v1.4.3.1).
  • If the Table Final Conclusion is executed, the calculation of all Methods (A, B, and C) is triggered.
  • The Table Final Conclusion has additional columns:
    90% CI (pass|fail), GMR (pass|fail), and overall (pass|fail).
  • If the design was RTR|TRT or RTT|TRR, an addi­tio­nal column Reliable estimate of CV_wR (yes|no) will be given right to the column CV_wR (%).
  • If the design was a partial replicate¹ the column CV_wT (%) will not be displayed since due to the over-spe­cified model the value esti­mated by ‘Method C’ is meaning­less.²
• Sub-workflow ABEL|Outlier Analysis
  • The filter for sequences and periods was hard-coded for RTRT|TRTR and RRT|RTR|TRR. There­fore, RTR|TRT and RTTR|TRRT did not work. Now the work­flow accepts any of the supported designs.
  • Nit-picky: Changed the outlier-flag from ±2 to ±1.959964.
  • Removed the 3px-border of the Box Plot.
  • PHX7-version of the template: Increased the Title Area Size of the Box Plot from 30 to 40.
• Documents folder.
  • Added the current version of the EMA’s Q&A document.
  • Updated the Instructions.

The template was developed in PHX 6.4.0.768 and tested in PHX 7.0.0.2535 as well. The template was cross-validated with 11 data sets (all designs, up to 360 subjects) against SAS 9.4 (Methods A, B, C), R 3.3.2 (A, B), STATISTICA (A), and partly (Q&A data sets I and II; Methods A, B) against STaTa 14, SPSS 22, and JMP 10. As usual, use at your own risk.

If you become aware of any defects, please report here.

 Comments:

• The partial replicate is – in statistical terms – a lousy design. If you want to avoid trouble, please forget it or report here why you want to use it.
• ‘Method C’ is an over-specified model. It is essentially forcing the soft­ware to esti­mate the within-sub­ject vari­ability of T, which the data cannot provide (since T was not repeated!). Hence, Phoenix – cor­rectly – throws a warning:
    Newton's algorithm converged with modified Hessian. Output is suspect.
    Model may be over-specified. A simpler model could be tried.
  Similar in SAS:
    Convergence criteria met but final hessian is not positive definite.
  F.i. the Q&A data set II gives a CV_WT 8.65% (Phoenix) or 3.87% (SAS). Both are meaningless.

 I removed the 1.4.3 templates from this post. Please download v1.4.3.1 provided in this post or in the previous versions.use the workaround as described.

Edited by Helmut Schütz, 13 December 2016 - 04:21 PM.

[Simon Davis]

Thanks for the early Christmas presents Helmut ;0)

 

Unfortunately some other projects will prevent me unwrapping them to look at for a few weeks!

 

 Simon.

[Helmut Schütz]

Hi Simon,

 

Thanks for the early Christmas presents Helmut ;0)
Unfortunately some other projects will prevent me unwrapping them to look at for a few weeks!

Welcome!

When I tried to shrink the size of the file (export template/clearing the history and import again) I made a mistake. The workflow Analysis of the 3 Dec. versions was a sub-workflow of Prepare Dataset for Analysis

and not as described in the instructions (and as it was in previous versions):

Corrected in the meantime.
If someone downloaded the versions of 3 Dec. already, please use the updated versions instead or:

• Drag/drop Analysis before Prepare Dataset for Analysis.
• Drag/drop Prepare Dataset before Analysis before Analysis.

In actual projects delete Example Data EMA 618604 2008 Rev 7 from the Data folder and the PDFs in the Documents folder in order to minimize the size of the file.

[mittyright]

Hi Helmut!

 

excellent templates, thank you!

 

BR,

Mittyright

PS I removed my remark regarding design object, everything is fine there

Edited by mittyright, 05 December 2016 - 10:35 PM.

[Helmut Schütz]

Hi Simon,

 

Thanks for the early Christmas presents Helmut ;0)

Maybe this is a Trojan Horse…

The EMA’s Q&A document gives this SAS-code for Method B:

proc mixed data=replicate;
  class formulation subject period sequence;
  model logDATA= sequence period formulation;
  random subject(sequence);
  estimate "test-ref" formulation -1 1 / CL alpha=0.10;
run;

Since no method for the DFs is specified in the model statement, SAS falls back to its default, which is

DDFM=CONTAIN

Splendid. In Phoenix We have Satterthwaite (default) and Residual.

Currently we are working on cross-validation of different software packages. Our target is agreement of the CV_wR and the 90% CI in percent, rounded to two decimals (according to the GL).
In one of our test data sets (RTRT|TRTR, unbalanced 38|39, incomplete period data 0|1|7|2, extreme CV_wR 222%) we discovered the first disagreement in 90% CI:

SAS (contain):          90.34 – 157.88

R package nlme:         90.34 – 157.88

R package lmer (Satt.): 90.35 – 157.88

Phoenix (Sattertwaite): 90.35 – 157.88

Phoenix (Residual):     90.34 – 157.88

AFAIK, nlme was designed to reproduce SAS DDFM=BETWEENWITHIN. Duno.

See the attached comparison of both DF-options in Phoenix. As expected DFs are identical if the design is balanced and the data complete. However, I don’t see a consistent pattern for the other data sets.

• In two cases Satterthwaite is more conservative than Residual (DFs lower).
• In five cases it is the other way ’round.
• Interesting the unbalanced DS11: identical DFs.

If we want to take the Q&A literally, Satterthwaite’s DFs should not be used.
Any ideas? Change the DF option in the template to Residual? Is PHX’s Residual equivalent to SAS’s DDFM=CONTAIN (and why not DDFM=RESIDUAL)?

 Edit: Here is a comparison of Phoenix’ Degrees of Freedom with SAS (THX to Detlew Labes!):
          Phoenix                 SAS
            ─────────────    ───────────────────
       Satt.  Residual    SATT  CONTAIN RESIDUAL
      ─────── ────────  ─────── ─────── ────────
DS01  216.939   217     216.939   217      292
DS02   45        45      45        45       66
DS03  143.267   143     143.267   143      218
DS04   99        99      99        99      147
DS05   74        74      74        74       98
DS06  216.939   217     216.939   217      292
DS07  717       717     717       717     1074
DS08  662       662     662       662      882
DS09  662       662     662       662      882
DS10   33        33      33        33       49
DS11  107       107     107       107      142
DS12  291.173   217     291.173   217      292
DS13  554.657   550     554.657   550      770
DS14  197.440   192     197.440   192      267
DS15  545.657   550     545.657   550      770

Hence, I believe that Phoenix’ Residual Degrees of Freedom equal SAS DDFM=CONTAIN. I attached the updated templates. If you want to take the EMA’s Q&A literally in previous versions: Navigate in the sub-workflow Analysis|Standard Average BE|Other Methods to object BE Method B log and change in the tab General Options > Degrees of Freedom from ⦿ Satterthwaite to ⦿ Residual.
Note that Satterthwaite’s DFs can be lower/equal/higher than the Residual DFs. This has consequences on the CI. In our data sets we had balanced (equal number of subjects in each sequence) and unbalanced (unequal number of subjects / sequence) ones, complete (data of all periods available) and incomplete (some period data missing) ones, and all of their combinations. An example of an unbalanced and incomplete data set is DS01 given in the Q&A. In comparing the outcome based on Satt. and Residual I observed this pattern of the CI:
    balanced + complete: identical
  unbalanced + complete: identical
  balanced + incomplete: liberal
unbalanced + incomplete: liberal or conservative

Attached Files

•  MethodB_and_PhoenixDFs.xls   33KB   533 downloads
•  EMA ABEL Project Template v1.4.3.1 PHX7.0.phxproj   3.36MB   453 downloads
•  EMA ABEL Project Template v1.4.3.1 PHX6.4.phxproj   3.25MB   403 downloads

Edited by Helmut Schütz, 13 December 2016 - 04:02 PM.