Jump to content


Photo

successful covariance step in bootstrap


  • Please log in to reply
10 replies to this topic

#1 Teodora

Teodora

    Member

  • Members
  • PipPip
  • 13 posts

Posted 05 March 2014 - 08:10 PM

Hi,

Is there a quick way to find out the number of bootstrap runs that had a successful covariance step, outside of checking out each individual run in the core status worksheet?

 

Thanks,

Dora



#2 serge guzy

serge guzy

    Advanced Member

  • Members
  • PipPipPip
  • 485 posts

Posted 05 March 2014 - 08:47 PM

I am not sure but I think you should get a thetabootstrap table and then sort by se.

If the table is there, every non successful covariance step should have se cells empty.

best

Serge



#3 Samer Mouksassi

Samer Mouksassi

    Advanced Member

  • Members
  • PipPipPip
  • 90 posts

Posted 05 March 2014 - 08:55 PM

the raw output file should have the return code for each boot model here you can know if the model has converged then as Serge suggested you can check which one had standard errors in theta boot worksheet that means the variance covaraince matrix could be computed and then it had a successful covariance step.

 

Many times I turn off stanard error computation when I run bootstrap and I just use all models that provided parameters estimates.



#4 Teodora

Teodora

    Member

  • Members
  • PipPip
  • 13 posts

Posted 06 March 2014 - 03:01 AM

Hi Serge,

 

There is a BootTheta worksheet that is outputted in Phoenix, but this is a summary table of the fixed parameters over all replicates. It does not contain the SE values of each individual replicate run. Should I have set up the run in a certain way to get the table you are talking about? I checked the help and I haven't found any info about this worksheet. Am I missing something?

Thanks,

Dora



#5 Teodora

Teodora

    Member

  • Members
  • PipPip
  • 13 posts

Posted 06 March 2014 - 03:15 AM

Hi Smouksassi,

How do you turn off the standard error computation when you run bootstrap?

 

I know that there are different schools of thought re using bootstrap estimates of parameter CI based on only the runs with successful covariance step vs based on all runs. All I could find in my search is that there is anecdotal evidence that it doesn't matter what you use, as the CI values between the two approaches are pretty close. But I am not sure that this is the case with my model.

 

Smouksassi, are there any articles that address this issue? What is the rationale of your approach?

 

Thanks,

Dora



#6 serge guzy

serge guzy

    Advanced Member

  • Members
  • PipPipPip
  • 485 posts

Posted 06 March 2014 - 03:58 AM

OK I understand. It is not there but only the summary statistics of all the thetas. The only way I see you could do it if you are able to take the output called bootsubj.csv (external view only), export and then import and then find a way to merge that with your data set. I know how to do it in Fortran but not with the data manipulation tools. The idea is that each row of bootsubj.csv gives you a patient number and the sample number. For each subject is associated a certain number of raw data and that is what you need to map with each subject. Then you would have the overall bootstrap data set you can then run using the simple mode but sorting by bootstrap number(called Samp in bootsubj.csv).

Best

Serge



#7 Teodora

Teodora

    Member

  • Members
  • PipPip
  • 13 posts

Posted 06 March 2014 - 05:53 AM

Thank you Serge, that's a good idea.

Best, Dora



#8 Teodora

Teodora

    Member

  • Members
  • PipPip
  • 13 posts

Posted 06 March 2014 - 06:37 AM

Serge,

If I follow the approach you suggested, would NLME get confused by the fact that some of the subjects are duplicated in one replicate? Do I have to make an additional variable for SUBJID whereby each subj has a different ID?

Thanks,

Dora



#9 Samer Mouksassi

Samer Mouksassi

    Advanced Member

  • Members
  • PipPipPip
  • 90 posts

Posted 06 March 2014 - 03:03 PM

Hi Dora,

Please find some responses below

 

In run options you chose standard error to be none

 

 

I am aware that there is different opinions regarding this subject one case study is at :

http://www.nature.com/clpt/journal/v77/n2/abs/clpt200514a.html ( it shows there is no difference)

 

We run the bootstrap to get the gold standard confidence interval (uncertainty distribution).

As such I see no need to request standard errors on each replicates.

Standard error computations are very time consuming. The only use I see is that if you want to try to exclude runs without standard errors and then compare to when you use all replicates.

 

One important plot to look at is the distribution of your likelihood value to make sure that you don't have multi model ( multiple minimum) over your replicates. Another is to make sure that your bootstrap distribution converged

http://www.page-meeting.org/pdf_assets/2508-Wilkins%202008%20PAGE%20BEQ.pdf

 

 

The rationale of my approach is based on several scientific discussion and on experience of hundreds of POP PK projects that I performed. I agree with you that it is still anecdotal and we need more scientific solid proof.

 

Also see the pfizer guidance and appendix it is free they mention any model that provide parameter estimates is to be used (successful minimization or termination due to rounding errors) without paying attention to covaraince step and I agree with them. What is more important is to run enough replicates and to stratify correctly.

 

http://www.nature.com/psp/journal/v2/n7/full/psp201326a.html

 

Samer



#10 Teodora

Teodora

    Member

  • Members
  • PipPip
  • 13 posts

Posted 06 March 2014 - 04:24 PM

Thanks Samer! Much appreciated.

 

I have another question: I am trying to run the bootstrap on a model that has both dermal PK and IV PK (from different subjects). I would like to set up the bootstrap so that the number of subjects within each group is identical to the original dataset. Is there a way to set this up?

I tried to add the route of admin as a covariate in the model (without actually using it as a covariate) just so I can bring up the stratification option in bootstrap. However, it didn't work.

 

Any suggestions?

 

Thanks,

Dora



#11 Samer Mouksassi

Samer Mouksassi

    Advanced Member

  • Members
  • PipPipPip
  • 90 posts

Posted 06 March 2014 - 04:31 PM

Hi Dora,

 

The covariate should be defined as categorical in covariate types and then it should show up in starification.

 

Samer






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users