Background: Dabigatran, an oral direct thrombin inhibitor, has now been available for 2 years in the US for the prevention of stroke in patients with nonvalvular atrial fibrillation, and direct Xa inhibitors are also starting to enter the market. Studies examining the effects of new oral anticoagulants in health care databases are beginning to emerge. The purpose of this study was to describe the validity of early published observational studies on the comparative safety and effectiveness of new oral anticoagulants in patients with atrial fibrillation. Methods: We identified published nonrandomized post-marketing studies (articles or conference abstracts or posters) and critically appraised their internal validity, with a particular focus on their ability to control confounding and other biases. Results: Two full-length journal articles, three conference posters, two conference presentation abstracts, and a US Food and Drug Administration analysis form the basis of the early comparative effectiveness and safety experience with new oral anticoagulants. Some published studies exhibit substantial biases and have insufficient precision for several important endpoints. Several studies suffer from biases arising from comparing ongoing users of the older drug, warfarin, who seem to tolerate it, to initiators of the new treatment who may have switched from warfarin or have had no prior experience with anticoagulants. Analyses tended to not adjust or not adjust adequately for confounding, and unsound propensity score application was also observed. Several studies introduced selection bias by excluding patients who died during follow-up and by restricting the study population to those with continuous database enrollment following cohort entry. We describe how these deficiencies can be avoided when studying new drugs. Conclusion:The first published post-marketing observational studies may not be sufficient for decision-makers to assess fully the comparative effectiveness and safety of new oral anticoagulants. These studies have methodologic challenges that can be avoided by using sound pharmacoepidemiologic design and analysis strategies.