The purpose of double-blind studies and why they are needed sometimes, and not others

[Heddi]

There seems to be some confusion concerning the ideas of double-blind studies, particularly that EVERY scientific study NEEDS to have a double blind study in order to be valid.

This is wrong on many levels, and I will try to explain in simple terms later on.

For the record, I'm not a scientist. I'm not a Doctor. I'm not a statistician.

However, before I became a nurse, I worked for a leading center of cancer research, and I was involved in 2 studies that dealt with cancer.

Both studies (one dealt with Ovarian, the other with Breast cancer) were aimed at trying to find a better way to diagnose ovarian and breast cancer earlier than our current detection allows. Especially for ovarian cancer, which is known as a silent killer because it often manifests no symptoms or very vague symptoms (bloating, gas, abdominal upset, etc) that can be attributed to other, mostly benign, causes.

For breast cancer, there is a genetic marker--CA-125, but this is known to have false positives and false negatives and can be a guiding point to warrant further diagnostics (in the case of a high CA-125), but it isn't foolproof.

The research study I worked on had multiple components:

1) enrolling women who had known 1st and 2nd degree relatives with breast and or ovarian cancer (since the two are very closely related), but had no breast or ovarian cancer themselves

2) enrolling women who had known 1st and 2nd degree relatives with breast and or ovarian cancer, but did have breast and or ovarian cancer themselves

3) enrolling women whose 1st or 2nd degree relatives had no breast and or ovarian cancer, and the participants had no breast or ovarian cancer

4) enrolling women whose 1st or 2nd degree relatives had no breast and or ovarian cancer, yet the participants DID have breast and or ovarian cancer.

Participants in the study agreed to long-term monitoring through blood draws over the course of 2 years, each blood draw 6 months apart.

Participants also all filled out a standardized multi-page questionnaire which went through their family history (going beyond first and second degree relatives), their ethnic background (Azenkazi Jews have a higher rate of certain cancers relative to the general population), the number of pregnancies they have had (as well as the number of term births, miscarriages, still births, and abortions), when they started to menstruate, the average length of menstruation, and when (if) they started menopause, and the symptoms (if any) of menopause. Also we were interested in medications they have taken, especially those with hormonal and steroidal components. What kind of birth control did they use, and how long did they use it? Have they ever had any other kind of cancer? What treatment did they get? Any chemo or radiation? What about adjuvant therapy? HRT?

All of the women enrolled in the study were given a randomly-generated number that was attached to their questionnaire. The questionaire did not have a spot for their name or any other identifying information aside from the randomly generated number.

Their blood samples were also labeled with just this number (and the date and time of the blood draw) but no other identifying information.

One of my roles in the study was to keep a database of the name of the study participant, and the randomly generated number that was assigned to them. This database was not able to be accessed by any of the researchers.

Then, in a separate database that WAS accessable to one set of the researchers, I entered the information from the questionnaires.

Then, in a THIRD database that was accessable to another set (but not the first set) of researchers, there was information about the blood samples obtained from the patients, again, using the same randomly generated number that was attached to the pt's name and questionnaire.

All three of these databases were separate, and only accessable to those working on THAT part of the study: the questionnaires, or the blood, or the names and randomly generated numbers.

THen, another component of the study was for women who had filled out the questionnaires, and who had given blood, but were also undergoing surgery to remove tissue---either through a mastectomy in the event of breast cancer, or a hysterectomy (removal of only the uterus and cervix) or a total abdomominal hysterectomy, salpingo-oophorectomy (removal of the uterus, cervix, ovaries, and fallopian tubes).

My job was to take the tissue removed from the operation and dissect parts of the tissue, cut it up, and freeze it in a variety of medium for study. Each tissue sample was marked with the same randomly generated number that was assigned to the participant, the blood, and the questionnaire.

Each surgery would land us about 10 different samples of ovary and fallopian tube or breast tissue. They were labeled, put in boxes, and put in liquid nitrogen for study.

The location, box, and date of collection was recorded (by moi) in a FOURTH database that was only accessable to the scientists working on the tissue.

----

So this is a double-blind, and this is why double-blind is needed in this kind of study. The scientists working on the blood had no idea what the risk factors were for the women whose blood we were taking. The scientists working on the tissue had no idea what the bloodwork showed, and the statiticians had no idea what correlation the questionnaire information had on the results of blood or tissue samples.

Each researcher was working independently of any pre-conceived notion of the woman who was part of the study. The only information the researchers had prior was:

1) this was a woman
2) this woman was above the age of 18

---

Then, say a researcher found a genetic marker in several samples of blood that may or may not have pointed to a certain cancer. They would enter that into the database. Later, the separate researcher would find something in the tissue that may or may not have pointed to a certain kind of cancer. They'd enter THAT into the database. The people working with the study will have rated the patient's information based on risk factor from KNOWN genetic or environmental risk factors.

ANOTHER set of people would go through the information entered by the blood people, the tissue people, and the questionnaire people and would parse that out to see what that information meant.

FINALLY, at the end of the study (we're not there yet, and I was working on the study in 2004-2005), someone will go through and put ALL the information together---the name, the number, the answers to the questions and what the researchers parsed from that, the blood and what the researchers parsed from that, and the tissue and what those researchers parsed from that. These FINAL people put it all together, find the statistical relevant information, find out what the outliers are and why they are there, and the study is published.

THIS is a double blind study. No one knows anything EXCEPT for the raw data they are processing and finding.

We were also involved with about 5 other Cancer Research Centers around the world that were doing the EXACT SAME STUDY with the EXACT SAME METHODOLOGY so that it would be a combined, large-sample comprehensive study on risk factors (hopefully updated from what we currently know) and HOPEFULLY genetic markers that can be more easily, and correctly, identified so that women can get early treatment.

---

Okay----so that's when a double-blind is necessary. It keeps the researchers from being biased (in this case) from extrapolating information that they may not have extrapolated had they known something about the research participant. If a researcher knew that the woman's blood he was testing was positive for breast cancer, he may look at that blood a little differently, or reach pre-conceived conclusions based on his knowledge of the pt's cancer status. However, not knowing how many blood samples came from people with or without cancer, they are able to look at ALL blood with the same non-bias.

The same with medicine---a double blind in the case of testing medicine gets rid of the bias of the researcher who knows that Medicine A is a placebo, but Medicine B has an anti-inflammatory process. That knowledge could bias the resarcher as far as followup questions, or how the information at the end of the study is extrapolated.

Same with the patient taking Medicine A---if they know it's a placebo, they will define their answers of questions based on the knowledge that they know the medicine is useless (therapeutically). If they know that they are receiving a medication with anti-inflammatory process, they will define their answers to questions on effecacy based on this knowledge.

---

But a study doesn't HAVE to be double-blind in order to be valid.

If I am a researcher and want to know how people view the quality of mail service in their area, I don't need to double-blind the study. I can ask the questions, the participants can know what they're being asked, and answer appropriately. I don't necessarily need to TELL Them that I am doing a study on mail satisfaction, and I can add "innocent" questions in the questionnaire about their garbage service, or the quality of the roads, just so they don't find the survey as a way to "rant" about mail service but rather to give honest answers about their feelings.

If I wanted to know what percentage of people in a given area code vote Republican, I can ask that question right out without hiding the reason for the study, or being hidden from knowing what the study was about.

---
Just because it's double-blind doesn't mean it's invalid. There *ARE* types of studies that can BE invalid because they weren't double blind, but double-blind is not a REQUIREMENT for every single type of study that is conducted.

:falling on deaf ears:

[beam me up scottie]

1. Happy to be the first k&r.

Thank you. :thumbsup:

[Heddi]

2. Danke shoen, dahlink, danke shoen

thank you for
all the joy and pain
i recall
central park in fall
how you tore your dress
what i mess
i confess
that's not all

la la la

[beam me up scottie]

3. ack! delete

way too much information

bmus out

[uppityperson]

4. when in trouble

when in doubt
run in circles
scream and shout

[Heddi]

5. That's one outrageous dude

The name's Poochie D,
And I rock the telly.
I'm half Joe Camel,
And a third Fonzarelli.
I'm the kung fu hippie,
From gangsta city.
I'm a rappin' surfer,
You the fool I pity

[mr blur]

6. Well said and well worth saying.

:thumbsup:

[LaurenG]

7. I thought this was an excellent article with easy to understand explanations

What was the outcome of the CA study or is it a current study?

[Heddi]

8. It is a current study

I get updates on progress now and again even though I'm not involved with the institute anymore.

One of the sister studies we were doing with a couple of other centers in the US dealt with shift workers and their risk of cancer. Some of the results have been published, one of which pointed to a greater risk of breast cancer in those women who worked 3rd shift, or overnight shifts. One of our researchers was studying factory and garment workers in China, and the other studies were based in the US. The significance is that Asian women, by in large, have one of the (if not THE) lowest risks of breast cancer in the world. Yet women who did shift work in China were disproportionately more likely to get breast cancer than their non-shiftworking female counterparts.

Most of the women in the US that were studied were RN's (like me) because that is the job that most women have that has them work 3rd, or overnight, shifts.

The theory is that the lack of melatonin may be a cause. Those that work overnight have lower levels of melatonin than those that work more traditional schedules. It's interesting to see what the long-term results are for the population, and whether taking melatonin supplements, or participating in light therapy, offers any benefits.

The studies I was working on, however, are still in progress and I don't think anything other than very preliminary risk factors have been identified. One result of the study on Ovarian Cancer, however, is that women who get Ovarian Cancer do suffer vague symptoms such as abdominal bloating, gas, abdominal discomfort, so the push is to educate MD's that women that present with these generally benign symptoms should be examined more thoroughly. I've also seen recently articles in women's magazines encouraging women to talk to their MD about possibility of Ovarian CA if they suffer abdominal issues on a frequent basis.

[LaurenG]

9. Fascinating that the 3rd shifters would have higher rates

My mind automatically went to low levels of Vitamin D/sun/light therapy. I am very interested in reading any updates you may have and would love to see the the completed study.

As an aside I just scheduled a young woman for an appointment with her pcp who asked for a CA 125 as the main reason for her appointment.

[Heddi]

10. The CA-125 can be a pointer for many women

It should not be used alone, though, and is always best when used in conjunction with family history, environmental risk factors, and diagnostic tests.

I saw a 28 year old woman with a CA125 of over 100,000 (ridiculously high) who had no breast cancer. I've seen women with multiple lymph node involvement, metastasis, and stage IV breast cancer with CA 125's in normal ranges.

The ultimate goal of those studies was to find, hopefully, a "rapid" blood test that could be done in a doctors office, like a pregnancy test, or maybe even one that was a send-off that would take a few days to get back but still would have a better true positive/true negative rate than CA-125 does.

And just as an aside, the researchers were not doing these studies to so that they could make high-cost pharmaceuticals to treat the cancer. They are looking for early detection so that women can have more options related to treatment, that cancers can be detected earlier and the earlier they are detected, the easier it is to remove them and remove them for good.

By the time my Aunt was diagnosed with Ovarian Cancer, she was stage IV with mets throughout her body. They did the hysterectomy but that only removed the reproductive organs. It did nothing for the mets that had gone to her bones and thyroid and lymph system. Radiation and chemo did help her go into remission for a while, but eventually it came back and was untreatable at that point. Had her cancer been detected earlier, she could have possibly saved herself chemo and radiation and had the cancer removed through TAH-BSO (total abdominal hysterectomy, bilateral salpingo-oophorectomy).

If you're interested, check out the research center's website: www.fhcrc.org.
A list of the "2008" research highlights: http://fhcrc.org/research/highlights.html


***HEY check this out. When looking at the highlights I saw this, which deals with the study I was working on and one of the researchers, Dr Andersen:

http://fhcrc.org/about/ne/news/2008/06/23/symptom_screening.html

SEATTLE — June 23, 2008 — Women's reports of persistent, recent-onset symptoms linked to ovarian cancer — abdominal or pelvic pain, difficulty eating or feeling full quickly and abdominal bloating — when combined with the CA125 blood test may improve the early detection of ovarian cancer by 20 percent, according to new findings by researchers at Fred Hutchinson Cancer Research Center published online today in CANCER.

Research has found that when used alone, a simple four-question symptom-screening questionnaire and the CA125 ovarian-cancer blood test each detect about 60 percent of women with early-stage ovarian cancer and 80 percent of those with late-stage disease. This study found that when used together, the questionnaire and blood test may boost early-detection rates to more than 80 percent and late-stage detection rates to more than 95 percent.

snip

For the just-published study, the researchers administered the symptom questionnaire to 75 women about to undergo surgery for pelvic masses who were later diagnosed with ovarian cancer (the case group), and 254 healthy women at high risk for ovarian cancer due to a family history of the disease (the control, or comparison, group). The cases were recruited through Pacific Gynecology Specialists at Swedish Medical Center in Seattle, and the controls were recruited through the Ovarian Cancer Early Detection Study, a joint project of the Hutchinson Center and the Marsha Rivkin Center for Ovarian Cancer Research.

------

In the last paragraph, OCEDS (ovarian cancer early detection study) through swedish and fred hutch was the study I was referencing in the original post, and they are referring to the questionnaires and blood and tissue samples I was talking about, too.

Neat!

Oh, and I did find in one of the preliminary studies that was published my name is an official part of the record since I was involved in record-keeping for the study. So my name is attached to some scientific study out there. Woot!

[LaurenG]

11. Wow - I am impressed and the results are a great step toward

early detection and combating what is often a fatal form of the disease. I am all for anything that can be done to prevent/stop any form of illness but especially something as "silent" as ovarian cancer. A 20 percent jump in early detection of ovarian cancer is excellent!

Good job and this is a great post!

Congrats!

Oh and this is good to see as well. "What's good for the heart may be good for the prostate
Study finds a diet high in vegetables and lean protein, and low in fat and red meat, combined with moderate alcohol use reduces the risk of symptomatic benign enlargement of the prostate."

http://fhcrc.org/about/ne/news/2008/02/12/heart_prostate.html

[Heddi]

12. Fred Hutch is a great faclity and I am honoured to have been able to work there

I find that as a nurse, my specialty is in cardiac, acute, and critical care. I have worked a few nights on the oncology floor and that is just not a place I could work for night after night. The nurses that are oncology nurses love their jobs (as I love mine), but that is just too emotional for me, especially since many of the patients are there for hospice, end of life, and comfort care. :( Same with burns....most people on the burn unit aren't there for end of life care, but the way that burns, especially partial and full thickness (2nd and 3rd degree) burns are cared for is beyond my ability. Many people go to the "scrubbing tub" on a daily basis where they are soaked in a tub and then have the wounds scrubbed vigorously. Not my cup of tea at all. Not for the gross factor (I love gross things), but the pain factor. Especially since the burn unit at my hospital also does pediatric burns....yikes. no thank you.

There is a link somewhere on the fred hutch page for signing up for research studies, and I think that they link to sister-studies done by other institutions outside of Seattle. I always encourage people to see if they fit any profile for any of the studies and become involved if that's something they're interested in.

The OCEDS study that is referenced above was blood draws and a questionnaire. For people that lived out of state, we paid for the blood draw and sent them the tubes and supplies where they could just go to a lab or dr's office, have the blood drawn and mailed back to us in a bio-hazard container. So we had people all over the country doing the blood and questionnaire for our Seattle-based study. The tissue removal was only for those that 1)had a disease process where the surgery was going to be done anyway and 2) wanted to take part in that aspect of the study. We didn't sign women up to get unnnecessary hysterectomies or mastectomies....they were getting the surgery anyways, and they (the pt) invited us to have samples of their tissue for the research.

I did a saliva test for some kind of cancer...I'm not even sure what it was, but they had me fill out some questions, spit in a cup, and did a cheek scraping with a Qtip for DNA. It took 2 minutes and I got free parking!! :D

[uppityperson]

27. Relative of mine was one of those responsible for investigating

and publishing on those studies done at Hutch yrs ago that were, oh what's the word I'm looking for? Wrong? Inaccurate? Lies? That said, I am very glad that FHutch is here and doing what they do, aside from that group that was unethical.

[bluedawg12]

38. See, you took a buccal swab, got free parking

and you're smiling. :)

The placebo effect! :evilgrin:

[TZ]

13. Excellent summation

Most of the clinical studies I work on are double blind but a few aren't (mostly the pre clinical stuff). There is debate how much I as a assay specialist (PK and IM, I think you know what those are) need to know...Time points of infusion usually is about it...although dosage is also something I know from time to time.
Designing and running a clinical trial is alot more complicated than folks (especially here) make it out to be.Nothing makes me madder than someone reading a study from Google and deciding they know better than the phD's who work for years to learn how to design them efficiently and effectively...

[HamdenRice]

14. Actually another reason is the scientifically inexplicable "placebo effect"

You've focused on researcher bias, which is one reason for double blind studies. But another aspect of double blind studies is that in clinical trials of drugs, vaccines and certain other treatments, the test subjects, or patients, themselves don't know whether they are getting the drug or a placebo.

The reason is the "placebo effect" -- namely, that if people think they are being treated, a certain percentage of them mysteriously will get better. So in order to measure the effect of a drug you can't measure it against the completely untreated; you have to measure it against people who think they are being treated and aren't. That way your statistical comparison will automatically be adjusted for the placebo effect.

No one knows why the placebo effect works, but the most likely explanation is some form of mind over body phenomenon.

It's a bit ironic that some people who insist that only double blind studies are valid tend to dismiss as woo woo any possibility of mind over body healing.

[trotsky]

17. Actually, the most likely explanation is perception over mind.

As evidence, we only need observe that there is no placebo treatment/effect for broken bones or smallpox or any serious condition. Chronic pain or similar phenomena are bad case studies for placebo, since those conditions tend to wax/wane over time anyway. But how one *perceives* the pain is a big issue. We know that we can sometimes treat phantom limb pain using a mirror to give the mind the *perception* that a limb is still there. In any study where a pill or treatment is administered, there is an innate human bias - a subject generally WANTS to please a researcher and in the process will report (and as a result perhaps even sense) a lessening of symptoms.

But if you jump right away to "mind over body" without a shred of evidence (as there is none), it's not unreasonable to call it woo.

[beam me up scottie]

19. It's more ironic that the poster who insisted that "only double blind studies are valid" was a woo.

Evil skeptic trotsky tried to explain it:

"valid studies can use entirely different methodologies, that "double blind" is only logical in certain kinds of investigations, or that a "control group" is meaningless when you're not changing anything in the subject's environment"

[trotsky]

26. Ahem.

That's MR. Evil Skeptic to you.

[beam me up scottie]

31. My humble apologies, sir.

I get so excited I sometimes forget my level.

I know that's no excuse, please don't smite me!

[TZ]

25. Placebo effect is OFTEN overstated, especially here.

Will it cure cancer? No. Will it fight off MRSA? No. Will it cure minor aches and pains--yes, sometimes.
Placebo effect has one big influence though..It helps us figure out which drugs are effective, because when the difference between a placebo group and a drug group is small, the likelyhood of continued research is slim to none.
Placebo effect is useful but its NOT A GODDAMNED MIRACLE TREATMENT.

[Why Syzygy]

32. .nt

:boring: .. completely unrelated ..

Something that skeptics would find amusing. Did you know that in homeopathy, there is a specific usage for the placebo? It's considered a remedy in its own standing, and is not made from any substance. I'm pretty sure my homeopath gave it to me once.

Conversely, a medical doctor once told me I was experiencing a "anti-placebo" effect. He made that determination because I had side effects from the Rx. IOW, I didn't want it to work, so I made myself bleed for two months, among other things that are well established side effects, which I was completely unaware of until AFTER I had a biopsy. That messy mind.

[TZ]

34. Okay.

See how much of a placebo effect helps you when you contract MRSA. Placebo effect=scientifically interesting, not that useful.
By the way since much of homeopathy has no scientific basis thats the only use for it. Mild placebo effect. I have yet to see homeopathy cure a disease. Makes someone feel better perhaps.
Just proves that homeopathy is NOT scientific. That messy mind works so well when you encounter nasty pathogens NOT! Or else how did people who never even heard of small pox die from it?

[uppityperson]

39. Because they really wanted to be covered with painful nasty sores and ooze to death

they felt guilty about something and made themselves have high fevers. See?

:sarcasm: just in the odd chance someone takes me seriously

[beam me up scottie]

41. Alt-med believers' anecdotes and testimonials are equivalent to (if not superior to) your data.

#9 of http://www.geocities.com/healthbase/altmed_debate_laidler.html">The Rules to Debating with "Alt-med" Believers:

Their anecdotes and testimonials are equivalent to (if not superior to) your data.
After all, everybody knows that data can be faked (and that people never, ever lie).

[populistdriven]

15. thank you! nt

[uberllama42]

16. Who argues that a study is invalid if it is double-blind?

Anyone who says that has a sad misconception, and I hope your post will remedy that. Thank you.

[beam me up scottie]

20. The same kind of person who repeatedly demands others disprove their ridiculous claims.

;)

[trotsky]

18. Excellent post & explanation.

Unfortunately those who need to read and understand it the most will likely ignore it. Recommended.

[LaurenG]

21. I know that you are wrong about that and

I think there was a misunderstanding. *peace*

[trotsky]

23. Um, I wasn't referring to you if that's what you thought. n/t

[LaurenG]

24. No I know you weren't.

I was just trying to sneak in and say I think there was a misunderstanding. Thanks for indulging me. :hi:

[Tumbulu]

22. Thank you for sharing so much of your time and knowledge with us (nt)

[Why Syzygy]

28. Thanks, Heddi!

The confusion was actually about reading comprehension and recognition of literary device.
Add in a failure to adequately quantify ability, and a big splash of researcher bias, and you've
just made one big wrong conclusion!

But, we always enjoy your lively descriptive passages. You're a hit! :thumbsup:

[omega minimo]

29. .

http://www.democraticunderground.com/discuss/duboard.php?az=view_all&address=247x23802

[Why Syzygy]

30. .

:crazy:

[TZ]

35. Yes. us evil scientists with our messy ol facts aren't welcome

here in the health scare forum where science and logic are NOT WELCOME! WOO-hoo.

[Why Syzygy]

37. I have yet to

see any skeptics act rationally. It is ALL pure emotion. That was my first observation six weeks ago.
It was true then and it is true now.

All I see is malicious vandalism. Not interested. Game over.

Get acquainted with mathematics >

http://en.wikipedia.org/wiki/John_Forbes_Nash

:hi:

[omega minimo]

40. "The Health Scare Forum"

Your behavior speaks for itself. :hi:

[Name removed]

36. Deleted sub-thread

Sub-thread removed by moderator. Click here to review the message board rules.

[bluedawg12]

33. Thank for your hard work on this and devoting the time

to explain it! :thumbsup:

It points to how complex research is and literally what an infinite variety of questions might be asked by researchers and why each project has to be set up with careful planning for statisical validity.

The take home points for me are:

Randomness

Reproducibility

And efforts to minimize bias, of which, a randomized, prospective, double blind study is usually considered the gold standard in evidence based medicine, BUT, keeping in mind that it is not always possible to do so, sometimes for ethical reasons, or sometimes, it is simply not the intent of the study, as in a literature meta analysis,

The first thing in a standard evaluation is to decide what type of study has been performed.
Is it a case series?
Is it a randomized clinical trial?
Is it a prospective cohort series?

The research protocol best suited for the task is then carefully designed, often with the aid of statisticians.

............
The Journal of Bone and Joint Surgery (American) 85:1-3 (2003)
© 2003 The Journal of Bone and Joint Surgery, Inc.

Editorial

Introducing Levels of Evidence to The Journal
James G. Wright, MD, MPH, FRCSC, Marc F. Swiontkowski, MD and James D. Heckman, MD
Levels of Evidence for Primary Research Question

Types of Studies

--------------------------------------------------------------------------------

Therapeutic Studies—Investigating the Results of Treatment Prognostic Studies—Investigating the Outcome of Disease Diagnostic Studies—Investigating a Diagnostic Test Economic and Decision Analyses—Developing an Economic or Decision Model

--------------------------------------------------------------------------------

Level I 1. Randomized controlled trial
a. Significant difference
b. No significant difference but narrow confidence intervals
2. Systematic review 2 of Level-I randomized controlled trials (studies were homogeneous) 1. Prospective study 1
2. Systematic review 2 of Level-I studies 1. Testing of previously developed diagnostic criteria in series of consecutive patients (with universally applied reference "gold" standard)
2. Systematic review 2 of Level-I studies 1. Clinically sensible costs and alternatives; values obtained from many studies; multiway sensitivity analyses
2. Systematic review 2 of Level-I studies
Level II 1. Prospective cohort study 3
2. Poor-quality randomized controlled trial (e.g., <80% follow-up)
3. Systematic review 2
a. Level-II studies
b. nonhomogeneous Level-I studies 1. Retrospective study 4
2. Study of untreated controls from a previous randomized controlled trial
3. Systematic review 2 of Level-II studies 1. Development of diagnostic criteria on basis of consecutive patients (with universally applied reference "gold" standard)
2. Systematic review 2 of Level-II studies 1. Clinically sensible costs and alternatives; values obtained from limited studies; multiway sensitivity analyses
2. Systematic review 2 of Level-II studies
Level III 1. Case-control study 5
2. Retrospective cohort study 4
3. Systematic review 2 of Level-III studies 1. Study of nonconsecutive patients (no consistently applied reference "gold" standard)
2. Systematic review 2 of Level-III studies 1. Limited alternatives and costs; poor estimates
2. Systematic review 2 of Level-III studies
Level IV Case series (no, or historical, control group) Case series 1. Case-control study
2. Poor reference standard No sensitivity analyses
Level V Expert opinion Expert opinion Expert opinion Expert opinion



1. All patients were enrolled at the same point in their disease course (inception cohort) with 80% follow-up of enrolled patients.
2. A study of results from two or more previous studies.
3. Patients were compared with a control group of patients treated at the same time and institution.
4. The study was initiated after treatment was performed.
5. Patients with a particular outcome ("cases" with, for example, a failed total arthroplasty) were compared with those who did not have the outcome ("controls" with, for example, a total hip arthroplasty that did not fail).

[xchrom]

42. kick