Filly's 8th Law
There are a few important things to know about statistics. Statistical significance.
Statistical significance means that the result was probably not a matter of chance (see below). It is unlikely that a scientific manuscript will be published if the results are “not statistically significant.” My point here is not that statistical significance is irrelevant. Of course, it is. However, and of critical importance to you and me as diagnosticians, it does not mean that the result will be clinically useful. Statistical validity, while important, has never meant that an observation will consistently point you in the right diagnostic direction. The classical example here is Doppler resistance of arteries in an ovarian mass to discriminate benign from malignant ovarian lesions. Every paper, and there are many, shows the finding to be statistically significant, but using the resistive index to judge the benignancy or malignancy of an ovarian mass is fraught with problems.
Always remember Filly's Law #6, “Never turn off the main computer bank.” If your experienced eye says, “worry,” but “the number” says don’t worry… WORRY!
A p-value less than 0.05 is statistically significant. Usually a research effort must show this level of statistical significance to be published. This means that of 20 papers that showed this level of statistical significance the results of one of those papers could have been achieved by flipping a coin.