Objective: To examine the interobserver agreement of commonly used clinical tests and diagnoses in patients with shoulder pain, and the accuracy of these tests and ultrasonographic findings in comparison with arthroscopic findings. Methods: Eighty six patients with longstanding shoulder joint pain were "blindly" examined by two trained doctors using several clinical tests. In all patients an ultrasonographic examination was performed, and in 42 (49%) an arthroscopy. Results: Tests for impingement showed poor to moderate agreement. Tenderness of muscles, muscle weakness, and tests for labral lesion also showed poor agreement. Pain during muscle contraction showed moderate agreement. The agreement of clinical diagnoses was poor and the accuracy was low in comparison with arthroscopy. Ultrasonography was accurate in full thickness supraspinatus tendon tears, but inaccurate for partial tears and labral lesions. Conclusions: Most clinical tests showed poor agreement. Clinical and ultrasonographic diagnoses had low accuracy in comparison with arthroscopy.