1、Analysis and evaluation of the evidence of diagnostic test Clinical Trail Study CenterCao SumeiDiagnostic test are not just about diagnosisl Screeningl Determining severityl Optimally therapyl Prognosisl Monitor Examplel Carotid ultrasound can tell you the severity of the patients carotid stenosisl
2、Carotid ultrasound can tell you the patients prognosis for stroke and deathl Carotid ultrasound can predict your patients likely responsiveness to therapyBasic principles of conducting diagnostic studiesl Apply the gold standard to determine whether or not the target condition is present Gold standa
3、rd:The most recognized standard for clinician to diagnose the target conditionlPathological measurementlOperation findinglSpecial imaging detectionlLong-term follow-up lRecognized standardWhat if your test is more gold than the standardl May lead to underestimate of the diagnostic power of the evalu
4、atingl One strategy for dealing with this problem is to use long-term follow-up as a gold standardTo Whom Should the Gold Standard Be Applied?to everyoneselective performing the gold standard on patients may result in“verification bias”or“workup bias”Recruit your participantsl Recruit the target-neg
5、ative and target-positive participants identified by gold standard l characteristic of those to whom you will want to apply the test in clinical practicel Including a broad spectrum of the diseasedcase：from mildly to severelycontrol：a broad spectrum of competing conditionsl An alternative approach i
6、s that recruiting a consecutive sample of patientsMeasurement proceduresl Specifying test techniquel Reproducibilityl Blinding of the individual conducting or interpreting the test to the gold standardSelect statistical procedurelCalculating sample size212(1)Z SenSenn222(1)Z SpeSpen Example:Assuming
7、 a sensitivity of 80%,specificity of 6 0%o f u l t r a s o n o g r a p h y f o r d i a g n o s i s o f cholecystolithiasis.How many samples are needed?2122220.05,1.96(0.80,0.60,0.10(1.96)(0.80)(10.80)62(0.10)(1.96)(0.60)(10.60)93(0.10)ZSenSpenn 双侧），设例例Result evaluation indexl Example:126 patients un
8、derwent independent,blind BNP measurement and echocardiography for diagnosis of LVD.l sensitivity：a/(a+c)=35/40=0.88,or 88%l specificity：d/(b+d)=29/86=0.34,or 34%l positive predictive value(PPV)：a/(a+b)=35/92=0.38,or 38%l negative predictive value(NPV)：d/(c+d)=29/34=0.85,or 85%l prevalence:(a+c)/(a+
9、b+c+d)=40/126=0.32,or 32%l Pre-test odds:pre-test probability/(1-pre-test probability)=32%/68%=0.47l positive likelihood ratio(LR+):Sen/1-Spe=88%/(100%-34%)=1.3Multilevel likelihood ratiosStability of the indexlStable:Sen,SpelRelatively stable:LR+,LR-lUnstable:PPV,NPV,prevalence:Receiver operating c
10、haracteristic curves(ROC)l It illustrates the performance of a diagnostic test when you select different cut-points to distinguish“normal”from“abnormal”l It demonstrates the fact that any increase in sensitivity will be accompanied by a decrease in specificityl The closer the curve gets to the upper
11、 left corner of the display,the more the overall accuracy of the testl The closer the curve comes to the 45-degree diagonal of the ROC space,the less accurate the testl The area under the curve provides an overall measure of a tests accuracy Fig A ROC for BNP as a diagnostic test for LVD Parallel te
12、st l A test B test Resultl +l +l +l -lReduction miss diagnosislExclude some disease lWhen prevalence is low,as the primary screening method(1)SenSenASenASenBSpeSpeASpeBSerial testl A test B test Resultl +l +-l +-l -Sen=Sen A SenBSpe=Spe A+(1-Spe A)Spe Bl Misdiagnosis may cause nuisance effect l Conf
13、irmatory diagnosis Serial test with enzyme labeled compound assay for diagnosis of myocardial infarctionEnzyme labeled compound assaySenSpeCPK57SGOT9174LDH87910.76(1)10.57(10.57)0.740.8920.89(10.89)0.910.99SenSenASenBSenCSpeSpeASpeASpeBSpeSpeMultivariate analysislSEN lSPE single variable analysismar
14、kermethodsSEN(%)spe(%)cutoffAREAaELISA90.788.80.19150.926b bELISA77.373.20.20350.802cELISA74.270.90.09050.762dELISA78.481.61.080.836eELISA90.784.40.3560.932fELISA84.581.60.7990.899multivariate analysis using logistic regression Combined markersSEN（）（）SPE（）（）AREAa and b b88.882.50.926a and c87.782.50
15、.927a and e91.690.070.974a and f95.590.070.967a and d87.285.6.0936b b and c78.876.30.837b b and d87.786.60.934b b and e83.882.50.900b b and f82.781.40.863C and d87.785.60.946C and e88.385.60.926C and f81.679.40.854d and e89.486.60.946d and f88.386.60.952e and f87.785.60.933Prediction the probability
16、 of a diseasel Logit(P)=-0.934+4.797 x a+2.203 x eAvoiding overfittingl Overfitting occurs when a computer model identifies a“chance”pattern that discriminates cancer patients from non-cancer patients,perfectly fitting that dataset but not reproducible in other data sets.l One way to avoiding overfi
17、tting is to randomly split the data into separate training and test samples.The EBM steps for diagnostic testsl Looking for the most suitable study papers according to the clinical question Bring forward the question in cliniclExample 2:if detection of serum forritin can diagnose Iron deficiency ane
18、mia?Search the computer information using the apposite key wordl“diagnose Iron deficiency anemia”and“diagnostic test”and“human”Evaluation of the scientificity of the papersl If compared with the gold standard independently and blindly Example 2:Iron stain with myeloid biopsyl If detected with the co
19、ntrol test for every quizzeeGold standardTotal（No.）Results+-New diagnostic test+351550sensitivity=46%New diagnostic test-40460500specificity=96.8%Verification biasGold standardTotal（No.）Results+-New diagnostic test+3515 50sensitivity=90%New diagnostic test-446 50specificity=75%If the patient samples
20、 included a broad spectrum of the diseasel If the disease spectrum uniform Whats the objective question that the investigator concerned aboutIf the study sample and the quizzee is uniformSpectrum bias：overstate the performance parameter of the diagnostic test because of excluding the“grey zone”patie
21、ntsThe precision of the diagnostic tests l Definition:in the same condition,degree of stability of achieving the same result when repeating the operation Detailed introduction the procedure standards of reporting diagnostic accuracy,STARDEstimate the significance of clinical application Estimate the
22、 pre-test probability Pre-test probability：Estimating the suffering probability before diagnostic testl Medical recordl Medical examination l Individual experience l Epidemiology dataThe predictive value depend on the pre-test probability of illnessExplain and use of sensitivity and specificityl Hig
23、h sensitivity test(negative test,rule out)l High specificity test(positive test,rule in)Use likelihood ratio,LRl Likelihood ratio：provides a direct estimate of how much a test result will change the probability of having a diseaselLR+=sensitivity/(1-specificity)l Likelihood ratio unchanged with the
24、prevalence ratel l Overcoming deficiency with only sensitivity or specificity to expresspost-test probability l post-test probability=post-test odds/（1+post-test odds）l post-test odds=pre-test odds LR l pre-test odds=pre-test probability（1-pre-test probability）l Crude principle:l LR 10 or 0.1 affirm
25、 or denial l 5 LR 10 or 0.1 LR 0.2 moderate probabilityl 2 LR 5 or 0.2 LR 0.5 minor probabilityl 1 LR 2 or 0.5 LR 100 81080.13denial45-100 7270.45minor probability18-45 23133.10moderate probability=18 47242.5 affirmTotal 85150The LR with serum ferritin for diagnosis of iron-deficiency anemiaApply th
26、e results to our own patientsl If the results applicable to my own patients If possess the qualification to develop the test in my own unit If the vital statistical information is uniform if the significance of the disease is uniformIf the result change the treatment to patientsl Example:A 45 years
27、old male patient came to the clinic on foot.The probability if he suffer from stenosis of coronary orifice is 6%.He told doctor his typical heartstroke history.The LR+of heartstroke medical record as a diagnostic test is 13.Then,he did ECG exercise test,showed that ST-segment depression(LR+=11)l Pre
28、-test probability=6%l Pre-test odds=0.06/0.94=0.064l Pro-test odds=0.064 13 11=9.13l Pro-test probability=9.13/(1+9.13)=90%Combined diagnosis:Pre-test odds x LR1 x LR2=Post-test odds Thank youl筑质量大堤，迎世纪挑战。23.1.1923.1.19Thursday,January 19,2023l不接受不良品，不制造不良品。11:56:3811:56:3811:561/19/2023 11:56:38 AM
29、l安全促进生产，生产必须安全。23.1.1911:56:3811:56Jan-2319-Jan-23l以管理保质量、以质量保进度、以进度求效益。11:56:3811:56:3811:56Thursday,January 19,2023l正确使用它、事故就少了。23.1.1923.1.1911:56:3811:56:38January 19,2023l管理靠制度，制度在落实。2023年1月19日上午11时56分23.1.1923.1.19l质量意识加强早，明天一定会更好。2023年1月19日星期四上午11时56分38秒11:56:3823.1.19l安全操作不马虎，分分秒秒惜生命。2023年1月
30、上午11时56分23.1.1911:56January 19,2023l品质管理标准化，ISO成功靠大家。2023年1月19日星期四11时56分38秒11:56:3819 January 2023l消防安全和减灾关系到全民的幸福和安宁。上午11时56分38秒上午11时56分11:56:3823.1.19l交通规则是您的生命之友。23.1.1923.1.1911:5611:56:3811:56:38Jan-23l从坏处着想，向好处努力。2023年1月19日星期四11时56分38秒Thursday,January 19,2023l科学管理、精心施工。23.1.192023年1月19日星期四11时56分38秒23.1.19谢谢大家！谢谢大家！
