A “post turtle” is a turtle resting atop a fence post. The turtle didn’t get up there by himself, he doesn’t belong there, he can’t get anything done while he’s up there and you just want to help the poor, dumb thing down.
Kind of describes our current president, doesn’t it?
Examine your guess. Your guess may be right, but it is foolish to accept a vivid guess as a proven truth — as primitive people often do. Your guess may be wrong. But it is also foolish to disregard a vivid guess altogether — as pedantic people sometimes do. Guesses of a certain kind deserve to be examined and taken seriously: those which occur to us after we have attentively considered and really understood a problem in which we are genuinely interested. Such guesses usually contain at least a fragment of the truth although, of course, they very seldom show the whole truth. Yet there is a chance to extract the whole truth if we examine such a guess appropriately.
Many a guess has turned out to be wrong but nevertheless useful in leading to a better one.
No idea is really bad, unless we are uncritical. What is really bad is to have no idea at all.
From: Observational Epidemiology
“If the texts seem incomprehensible, it is for the excellent reason that they mean precisely nothing.”
Most of us run sample size calculations when a granting agency or committee requires it. That’s reason 1.
That is a very good reason. But there are others, and it can be helpful to keep these in mind when you’re tempted to skip this step or are grumbling through the calculations you’re required to do.
It’s easy to base your sample size on what is customary in your field (”I’ll use 20 subjects per condition”) or to just use the number of subjects in a similar study (”They used 150, so I will too”).
Sometimes you can get away with doing that.
However, there really are some good reasons beyond funding to do some sample size estimates. And since they’re not especially time-consuming, it’s worth doing them.
Often the most time consuming part is figuring out and writing the data analysis plan to base the calculations on, but that’s another step you should do anyway.
Reason 2: Many, many published studies have very low power, and are bad sources for basing your sample size on.
As reported in Keppel, Cohen calculated the power of every study in a psychology journal for a year. The average power was just under 50%.
If power is 50% for a study, it basically means that that study had a 50% chance of finding significant results given the sample size, the effect size, and the statistical test. Because these were published studies, they must have had significant results. But there were probably a lot of other studies (just as many) that never got published because they just didn’t have adequate power.
If you now attempt to build on that study and you use the same sample size, you only have a 50% change of replicating it with significant results. Do your own power calculation and raise the sample size, if needed.
Reason 3: A power calculation estimates not only how many participants you need, but how many you don’t need.
You don’t want to spend any more resources–time, money, and energy–collecting more data than you need. Save those resources for a follow-up study.
Especially if your study creates any risk, or even inconvenience, for your human or animal participants, you don’t want to oversize your study either. You don’t want to expose more participants than necessary to the risk.
Reason 4: When sample size calculations tell you you’re close, but have not quite enough subjects, you can make adjustments to the study that will increase the power in other ways.
Maybe you can adjust the way you’re measuring some of your variables to add precision or switch your design to something that will give you a little more power. Or make sure you include some controls that will control some of the random error. All of these increase power without increasing sample size.
Reason 5: The biggest benefit of doing these calculations is to not waste years and thousands of dollars in grants or tuition pursuing an impossible analysis.
If sample size calculations indicate you need a thousand subjects to find significant results but time, money, or ethical constraints limit you to 50, don’t do that study.
I know it’s painful to go back to square 1, but it’s much better to do it now than after 3 years of work.
From: The Analysis Factor
Princeton University’s list of skills that make an “educated person”:
- The ability to think, speak, and write clearly.
- The ability to reason critically and systematically.
- The ability to conceptualize and solve problems.
- The ability to think independently.
- The ability to take initiative and work independently.
- The ability to work in cooperation with others and learn collaboratively.
- The ability to judge what it means to understand something thoroughly.
- The ability to distinguish the important from the trivial, the enduring from the ephemeral.
- Familiarity with the different modes of thought (including quantitative, historical, scientific, and aesthetic.)
- Depth of knowledge in a particular field.
- The ability to see connections among disciplines, ideas and cultures.
- The ability to pursue life long learning.
Here’s Harvard University’s list of skills that make an “educated person”:
- The ability to define problems without a guide.
- The ability to ask hard questions which challenge prevailing assumptions.
- The ability to quickly assimilate needed data from masses of irrelevant information.
- The ability to work in teams without guidance.
- The ability to work absolutely alone.
- The ability to persuade others that your course is the right one.
- The ability to conceptualize and reorganize information into new patterns.
- The ability to discuss ideas with an eye toward application.
- The ability to think inductively, deductively and dialectically.
- The ability to attack problems heuristically.
There are many effect size statistics for ANOVA and regression, and as you may have noticed, journal editors are now requiring you include one.
Unfortunately, the one your editor wants or is the one most appropriate to your research may not be the one your software makes available (SPSS, for example, reports Partial Eta Squared only, although it labels it Eta Squared in early versions).
Luckily, all the effect size measures are relatively easy to calculate from information in the ANOVA table on your output. Here are a few common ones:
Eta Squared, Partial Eta Squared, and Omega Squared Formulas
Cohen’s d formula
You have to be careful, if you’re using SPSS, to use the correct values, as SPSS labels aren’t always what we think. For example, for SSTotal, use what SPSS labels SS Corrected Total.
What SPSS labels SS Total actually also includes SS for the Intercept, which is redundant to other information in the model.
This is a nice page that walks you through some of these calcuations using SPSS output:
Measures of Effect Size in SPSS
The denominator for Cohen’s d is always some measure of standard deviation. I’ve shown s pooled here, but you often see different options, including just using one sample’s s. This is the one I see used most commonly.
From: The Analysis Factor
Source: Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing moderator and mediator effects in counseling psychology research. Journal of Counseling Psychology, 51(1), 115-134. doi:10.1037/0022-022.214.171.124
The probability of any first significant digit n occurring in a dataset is:
From: Amazing Applications of Probability and Statistics
Source: Page, L. A., & Henderson, M. (2008). Appraising the evidence: What is measurement bias? Evidence Based Mental Health, 11(2), 36-37. doi:10.1136/ebmh.11.2.36
Source: Bethlehem, J. (in press). Selection bias in web surveys. International Statistical Review. doi:10.1111/j.1751-5823.2010.00112.x
The Department of Methodology and Evaluation Research at Universität Jena has made available a set of free online video courses on data analysis. They cover topics that are particularly relevant to psychology and social science researchers, including SEM, causal inference, regression, R, and psychometrics. Some courses are in German, but many are in English, and the language of the course is clearly marked. Some require that you register, but registration is free. Their website allows you to filter just for English Language courses. Below are some courses that Jeremy Anglin found particularly appealing.
From: R Bloggers
Here is a newly discovered blog with lots of useful articles about practical statistics.
From: Jeromy Anglim’s Blog: Psychology and Statistics
Below is a categorized list of the articles currently offered in the R Tutorial Series.
Introduction to R
From: R bloggers
- Baskerville – cultured, dignified and just and admirable
- Bodoni – elegant, subjective, both classic and modern
- Caslon – dignified, formal, vigorous but graceful
- Franklin Gothic – classic sans serif
- Frutiger – strong, readable, simple, smooth
- Futura – elgante sans serif
- Garamond – classically elegant, adult
- Gill Sans – sans serif font with a strong, warm and friendly personality
- Helvetica – neutral without being boring, simple, contemporary
- Optima – smooth, refined, calming, elegant, in my view, however, partially limited line width for presentations
- Rockwell – owned, bold, confident
From: Warum ignorieren Sie Typographie in Präsentationen? Die wichtigsten Regeln!
Read the passage carefully. Determine its structure. Identify the author’s purpose in writing. Make a note in the margin when you get confused, or when you think something is important.
Reread. This time divide the passage into sections or stages of thought. The author’s use of paragraphing will often be a useful guide. Label, on the passage itself, each section or stage of thought. Underlying key ideas and terms. Write notes in the margin.
Write one sentence summaries, on a separate sheet of paper, of each stage of thought.
Write a thesis: a one or two sentence summary of the entire passage. The thesis should express the central idea of the passage, as you have determined it from the preceding steps. You may find it useful to keep in mind the what, who, why, where, when, and how of the matter. For persuasive passages, summarize in a sentence the author’s conclusion. For descriptive passages, indicate the subject of the description and its key feature. Note: in some cases, a suitable thesis may already be in the original passage. If so, you may want to quote it directly in your summary.
Write the first draft of your summary by (1) combining the thesis with your list of one sentence summaries or (2)combining the thesis with one sentence summaries plus significant details from the passage. In either case, you eliminate repetition and less important information. Disregard minor details or generalize them. Use as few words as possible to convey the main ideas.
Check your summary against the original passage and make whatever adjustments are necessary for accuracy and completeness.
Revise your summary, inserting traditional words and phrases where necessary to ensure coherence. Check for style. Avoid a series of short, choppy sentences. Combine sentences for a smooth logical flow of ideas. Check for grammatical correctness, punctuation, and spelling.
Behrens, L., Rosen, L. J., & Beedles, B. (2002). A Sequence For Academic Writing. New York: Longman.
A new Web site, Open Educational Resources Center for California, brings together information on free and open textbooks and course materials in one location. Though the Web site was designed for California’s community-college faculty members, it could be a useful resource for anyone trying to find learning materials in the public domain.
The site links to more than 400 open textbooks and peer reviews of open textbooks.
APA Style Blog: Books That Provoke Our Thanks
Writers in psychology and the social and behavioral sciences reach for the APA Publication Manual first when they have style and usage questions. But inevitably, some topics are outside its scope, while others are covered in less detail than one might like.
At APA, there are a few resource books we turn to in gratitude on a daily basis. You’ll find them on the desk of almost every editor here, and they just might have the answers you’re looking for.
Grammar, Usage, and Style
The Publication Manual contains valuable basic information on writing clearly and correctly but by no means covers all the bases. When I’ve been wrestling with a thorny sentence for the last half hour and it still looks wrong, I reach for Words Into Type. If you’re looking for guidance on coordinate conjunctions or collective nouns, this is the place.
Some issues may arise infrequently in psychology and the social sciences (e.g., “In what context should I capitalize Platonic ideas?” “Where can I find a conversion chart from Wade-Giles to Pinyin?”), but they do arise. For questions with a humanities slant, the Chicago Manual of Style can be helpful. It has extensive information on foreign language references, titles of historical persons, and other topics that are beyond APA’s purview.
Finally, a lighter approach sometimes helps when you’re trying to get your head around language problems. In The Elephants of Style and Lapsing Into a Comma, Bill Walsh offers tips on contemporary usage in a humorous and thoroughly approachable way. Another classic in this vein is The Careful Writer, by Theodore M. Bernstein.
Spelling and Word Division
For everyday spelling issues, APA uses Merriam-Webster’s Collegiate Dictionary. But when you need to know whether it’s “Alzheimer syndrome” or “Alzheimer’s disease,” we (naturally) recommend the APA Dictionary of Psychology: a thousand pages of terminology specific to the psychological sciences.
Appendix 7.1 of the Publication Manual gives a good introduction to the use of legal materials in APA style. For most authors, this is all you’ll ever need. But researchers in some fields, such as forensic psychology, may need a more comprehensive guide. In that case, go straight to the source: The Bluebook: A Uniform System of Citation. (Then head over to the nearest law school librarian for help, because legal citation is an art unto itself.)
The Internet Public Library has lots of really useful links. I find this set particularly useful when writing.
Internet Public Library: Style & Writing Guides