I think this conflates poorly designed studies and preliminary or hypothesis-generating studies. If not for uncontrolled proof-of-concept studies, or pseudorandomized designs such as those used for post marketing studies, we would a) not have any new medications and b) rarely learn about unanticipated toxicities. It turns out that us clinical trial folks are more Bayesian than the cult of p<0.05 would lead you to believe.
At one end of the spectrum, we rely heavily on uncontrolled single- or multiple-ascending dose studies to prove to ourselves that a treatment is likely to be safe in next-step studies, and to guess at optimal dose. At the other, we learn a great deal from post marketing surveillance about unanticipated toxicities - because our priors based on big phase 3 studies may still be insufficient to accurately estimate risk. Neither of these designs are randomized or placebo-controlled - and in neither case is it an indication that 'something is wrong', even though an RCT would be better in both contexts. Better, if cost were no object and patient safety were not a concern.
I realize my fellow Brunonian does include some offhand caveats - but it worries me to read comments about how this negates most social science research.
Not at all... He states merely that if someone has the option to have a control and chooses not to, you should be concerned. Specifically he hits on the fact that some fields can't have a control (sometimes just due to ethical reasons).
I think the distinction is that we /often/ choose not to have a control, for a multitude of reasons, so as a cause for concern it's highly nonspecific. Much like the critique of small samples - there are underpowered 5000-patient multi center trials, and well-powered 30-patient trials.
As I said, I think this is a nice introduction but can lead one to discount, well, anything other than large randomized placebo-controlled trials.
At one end of the spectrum, we rely heavily on uncontrolled single- or multiple-ascending dose studies to prove to ourselves that a treatment is likely to be safe in next-step studies, and to guess at optimal dose. At the other, we learn a great deal from post marketing surveillance about unanticipated toxicities - because our priors based on big phase 3 studies may still be insufficient to accurately estimate risk. Neither of these designs are randomized or placebo-controlled - and in neither case is it an indication that 'something is wrong', even though an RCT would be better in both contexts. Better, if cost were no object and patient safety were not a concern.
I realize my fellow Brunonian does include some offhand caveats - but it worries me to read comments about how this negates most social science research.