Most of us run sample size calculations when a granting agency or committee requires it. That’s reason 1.
That is a very good reason. But there are others, and it can be helpful to keep these in mind when you’re tempted to skip this step or are grumbling through the calculations you’re required to do.
It’s easy to base your sample size on what is customary in your field (”I’ll use 20 subjects per condition”) or to just use the number of subjects in a similar study (”They used 150, so I will too”).
Sometimes you can get away with doing that.
However, there really are some good reasons beyond funding to do some sample size estimates. And since they’re not especially time-consuming, it’s worth doing them.
Often the most time consuming part is figuring out and writing the data analysis plan to base the calculations on, but that’s another step you should do anyway.
Reason 2: Many, many published studies have very low power, and are bad sources for basing your sample size on.
As reported in Keppel, Cohen calculated the power of every study in a psychology journal for a year. The average power was just under 50%.
If power is 50% for a study, it basically means that that study had a 50% chance of finding significant results given the sample size, the effect size, and the statistical test. Because these were published studies, they must have had significant results. But there were probably a lot of other studies (just as many) that never got published because they just didn’t have adequate power.
If you now attempt to build on that study and you use the same sample size, you only have a 50% change of replicating it with significant results. Do your own power calculation and raise the sample size, if needed.
Reason 3: A power calculation estimates not only how many participants you need, but how many you don’t need.
You don’t want to spend any more resources–time, money, and energy–collecting more data than you need. Save those resources for a follow-up study.
Especially if your study creates any risk, or even inconvenience, for your human or animal participants, you don’t want to oversize your study either. You don’t want to expose more participants than necessary to the risk.
Reason 4: When sample size calculations tell you you’re close, but have not quite enough subjects, you can make adjustments to the study that will increase the power in other ways.
Maybe you can adjust the way you’re measuring some of your variables to add precision or switch your design to something that will give you a little more power. Or make sure you include some controls that will control some of the random error. All of these increase power without increasing sample size.
Reason 5: The biggest benefit of doing these calculations is to not waste years and thousands of dollars in grants or tuition pursuing an impossible analysis.
If sample size calculations indicate you need a thousand subjects to find significant results but time, money, or ethical constraints limit you to 50, don’t do that study.
I know it’s painful to go back to square 1, but it’s much better to do it now than after 3 years of work.
From: The Analysis Factor
Princeton University’s list of skills that make an “educated person”:
- The ability to think, speak, and write clearly.
- The ability to reason critically and systematically.
- The ability to conceptualize and solve problems.
- The ability to think independently.
- The ability to take initiative and work independently.
- The ability to work in cooperation with others and learn collaboratively.
- The ability to judge what it means to understand something thoroughly.
- The ability to distinguish the important from the trivial, the enduring from the ephemeral.
- Familiarity with the different modes of thought (including quantitative, historical, scientific, and aesthetic.)
- Depth of knowledge in a particular field.
- The ability to see connections among disciplines, ideas and cultures.
- The ability to pursue life long learning.
Here’s Harvard University’s list of skills that make an “educated person”:
- The ability to define problems without a guide.
- The ability to ask hard questions which challenge prevailing assumptions.
- The ability to quickly assimilate needed data from masses of irrelevant information.
- The ability to work in teams without guidance.
- The ability to work absolutely alone.
- The ability to persuade others that your course is the right one.
- The ability to conceptualize and reorganize information into new patterns.
- The ability to discuss ideas with an eye toward application.
- The ability to think inductively, deductively and dialectically.
- The ability to attack problems heuristically.
To avoid making purchases I’ll only regret later, I created a list of 14 questions I ask myself before I buy. They’re designed to cut through the consumer noise and get at the true value of the item or the true motivation behind purchasing behavior. The questions don’t fit every buying situation, but they work most of the time. Amend as needed and watch your regrets and credit card bills shrink.
1. Can I get this at a better price somewhere else?
Are you shopping at the right place to get the best price? Is there a less-expensive retailer in your area that you haven’t checked out?
2. Can I get this at a better price at some other time?
Are you buying your lawn furniture in the spring or waiting until autumn when there are deals to be had? If your timing is predictable, retailers have you right where they want you.
3. Do I already own it?
This may seem like an odd question, but often we buy replacement items when we can’t locate things we already own. Does your home need an organizational makeover? Is it time to declutter and discover the wealth of items you already own but can’t find in all the chaos?
4. Is this product about to be improved upon?
This is a tricky one. On one hand, if an object is about to be improved upon, you may want to wait for the more feature-rich model to come out and avoid upgrades later. On the other hand, older models tend to decrease in price when new versions are released and there are deals to be had if you can postpone the purchase. In either case, biding your time can pay off.
5. Can I borrow this product from someone else or buy it used?
Is it even necessary to buy an item or, at the very least, buy it new? Can you borrow a vacuum from a neighbor for that single room in your home that’s carpeted? If you hit a few yard sales this summer, could you find a good used vacuum for pennies on the dollar?
6. Does this product make my life easier or more complicated?
It took a brilliantly evil mind to reinvent the broom by attaching a glorified paper towel to a stick and making a simple device something that required constant refills. Any product that promises to simplify your life by eliminating a single object you already own and replacing it with an object you must “feed” should be relegated to the dustbin of history. It’s not cost-effective and definitely not simple.
7. Does this product function independently or require add-ons to work?
Are you buying a single product that works on its own or one that requires more features, attachments, and upgrades to do the job? Avoiding wallet-hungry products is the best way to go.
8. Do I have to use credit to pay for this item and is it worth it?
Even the best deals are soured once you factor in credit card interest. If you have to use credit to pay for an item (and can’t pay that credit card bill off completely during your next billing cycle), the deal better be worthy of a Facebook status update.
9. Am I an educated consumer of this product?
It’s about time the Information Age did something more than show us funny videos of dancing cats and giggling babies. Use the tools available at your fingertips to research consumer data on products before you buy. Leverage the power of communication to make smarter buying decisions.
10. Is there another product that’s just as good and less expensive?
This question requires an understanding of how you’ll use the product based upon your habits and behavior. Do you need the deluxe MP3 player if you’re just using it while you jog for 30 minutes a day? What’s a simpler, less expensive solution that would fulfill your specific needs just as well?
11. Will this product still be useful in six months or a year?
Fast-forward through the life cycle of this product and see where you envision it in six months or a year. Be realistic — Is it still being used? Has it become abstract lawn art? Is it in the garage with the rest of the yard sale stuff?
12. If I waited two weeks, would I want this item just as badly?
In other words, is this an impulse buy? Is this something that you truly need or just feel compelled to buy at this moment?
13. Does this product carry a warranty?
What’s your recourse as a consumer if this item doesn’t work or malfunctions after purchase? Is the manufacturer confident enough to provide a warranty, or are you on your own the minute you hand over your cash?
14. Is this product disposable when there is a non-disposable solution?
This question focuses on the environmental and budgetary impact of what we buy. Opting for reuse and rejecting single-use items whenever possible may be slightly less convenient, but more beneficial for our planet and our pocketbooks.
As we all try to leverage the power of our dollars, screening our purchases through the filter of tough questions can be the smartest thing we do for our budgets.
From: Wisebread
There are many sources of Earned Value Management training. Some good, some not so good. Here is the training materials from Department of Energy, that is one of the better resources.
Each of these modules provides an indepth view of EVM from a practical point of view. When someone speaking about EV, its applicability in a domain, its application to specific projects, and the like – take care to vet the source of the advice, the domain and context where that advice is applicable, and the credentials of the adviser.
From: Herding Cats
There are many effect size statistics for ANOVA and regression, and as you may have noticed, journal editors are now requiring you include one.
Unfortunately, the one your editor wants or is the one most appropriate to your research may not be the one your software makes available (SPSS, for example, reports Partial Eta Squared only, although it labels it Eta Squared in early versions).
Luckily, all the effect size measures are relatively easy to calculate from information in the ANOVA table on your output. Here are a few common ones:
 Eta Squared, Partial Eta Squared, and Omega Squared Formulas
 Cohen’s d formula
You have to be careful, if you’re using SPSS, to use the correct values, as SPSS labels aren’t always what we think. For example, for SSTotal, use what SPSS labels SS Corrected Total.
What SPSS labels SS Total actually also includes SS for the Intercept, which is redundant to other information in the model.
This is a nice page that walks you through some of these calcuations using SPSS output:
Measures of Effect Size in SPSS
The denominator for Cohen’s d is always some measure of standard deviation. I’ve shown s pooled here, but you often see different options, including just using one sample’s s. This is the one I see used most commonly.
From: The Analysis Factor
If you can’t explain it to a six year old, you don’t understand it yourself.
It is, of course, the plan plus the action that explains success, not the action or the plan alone.
Better to light a candle, than to curse the darkness.
Just because the plan seems complete and you think you’re ready to go doesn’t necessarily mean that you are. Apparently small details left unattended as the project is poised for execution can become the source of re-work, frustration, delays, conflict and dysfunctional team behaviors later on in the project.
16 Team Readiness Checks
Here are some of those often forgotten pre-launch checks:
- Have the overall project objective and scope boundaries been shared with all team members?
- Have all known gaps in resource expertise been resolved?
- Have clear roles and responsibilities been defined for each individual?
- Has real availability been validated with each team member and relevant line managers?
- Have time and effort estimates involved input from the team?
- Have the team agreed on who owns which deliverables?
- Have those owners specified completion criteria for each of their deliverables?
- Is the team aligned on deadlines, dependencies, constraints and risks?
- Is the project team ready, willing and able to execute the project according to the baseline plan?
- Have initial work priorities been communicated to the project team?
- Has a procedure for issuing weekly WBS task lists, actions and priorities to the team been set?
- Is the team aware of which tasks are critical and will actual slack values be communicated to task owners each week?
- Has the team been informed of how and when they should provide status updates?
- Has the team been involved in identifying risks and formulating response strategies?
- Have procedures for raising, escalating and resolving issues been defined and communicated?
- Does the team know how often project review meetings will be held and who should attend?
From: pm-lotus
A successful team is a group of individuals who hold each other accountable for a shared outcome.
This is a pretty good article with a practical methodology for web site information architecture.
Here is my simple, patent pending method to use to help users design good SharePoint sites. It combines two very effective IA methods into one and its amazing how it turns people from wanting 1990’s era sites complete with horizontal scrolling banners with animated GIF’s into usability and IA gurus within minutes.
The tools of the trade you need for this method is:
So now you know the ingredients, let’s run through the recipe
- Put key stakeholders into a room (ensure the ones with poor taste are there together)
- Visit websitesthatsuck.com and review the 2010 contenders for worst websites of the year. (For what its worth, my personal vote is Yale School of Art)
- Have a good laugh and discuss all the crappy aspects to those sites – make particular note of the write-up on websitesthatsuck for each contender
- With the group’s sucky website radar now primed, have them load up their existing intranet (if they are really big organisation, go around to various departmental sites around the intranet). This time they will not laugh, due to the effect of your “crapness calibration” ™ exercise, they will see many faults in the existing site straight away.
- At this point, crank out Balsamiq and start to wireframe what the site should look like while you have the fleeting moment of clarity (crapness calibration fades with time and needs to be re-primed). The wisdom of the crowd should ensure that most of the common mistakes will be avoided there and then.
- Statistically, one of every three times you do this, there is always one user who’s taste is so bad that calibration will take another round of deprogramming. So if you have someone that persists with crap taste or has ideas that 99% of the user base would balk at, move to the 2009 hall of shame for sucky sites. Faced with the reaction from their peers, as well as the parallels that can be drawn between their current site and the contenders, it usually does the trick.
- Also be sure to draw attention to sites that have similar underlying concepts, but where one works well and the other has agonising lameness. For example, the New York Times compared to Havenworks. Discuss the layout, colours, fonts, images, navigation, search and the like and relate back to the site being envisioned.
In about 30-90 minutes, one of two things will happen.
- You will have a pretty good wireframe or three
- The group will realise that they have more soul searching to do.
Although your business development manager will whine at you if outcome 2 happens, consider it a good thing. You will be saving yourself and the participants a mountain of stress later and have them thinking more holistically about the outcomes they are trying to achieve.
(Final serious bit at the end alert)
What you will notice when performing this process, is that with a recent and clear frame of reference, some of the biases that people carry with them can be temporarily lifted. In some ways, this exercise is very similar to the “down the pub” calibration of estimates exercise that I wrote about previously. The trick is to find ways to change the lens people look through to see other aspects or facets to the problem at hand.
From: Clever Workarounds
Source: Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing moderator and mediator effects in counseling psychology research. Journal of Counseling Psychology, 51(1), 115-134. doi:10.1037/0022-0167.51.1.115
The probability of any first significant digit n occurring in a dataset is:
From: Amazing Applications of Probability and Statistics
Source: Page, L. A., & Henderson, M. (2008). Appraising the evidence: What is measurement bias? Evidence Based Mental Health, 11(2), 36-37. doi:10.1136/ebmh.11.2.36
Source: Bethlehem, J. (in press). Selection bias in web surveys. International Statistical Review. doi:10.1111/j.1751-5823.2010.00112.x
“It is what you learn after you know it all that counts.”
- Collect: Grab everything that needs your attention. Whether you use David Allen’s mind sweep or you prefer a list format, go through your messages, e-mail, missed actions, etc. and capture all the items that require some action.
- Meet: Sit down with co-workers. This is the people version of collecting. Find out what was managed was you were away, what new issues have arisen and add these to your mind-sweep list. This is also a good time to thank them for covering your unexpected absence.
- Process: Once you’ve collected all the open loops, figure out what you need to do to close them. Whether it’s as simple as throwing a brochure in the garbage or as complex as planning a management retreat, you need to identify the steps needed to move the item forward.
- Prioritize: Next, you can organize the action steps into lists of what you’re going to do.
- Get it done: Now that you know what you need to do, get started. It may take time and effort to get things reorganized to move forward, but don’t stop at the end of step four. Here’s where you can pull things back on track.
From: Ian’s Messy Desk
- Do I heckle my subordinates or strengthen and encourage them?
- Do I use moral courage in getting rid of subordinates who have proven themselves beyond doubt to be unfit?
- Have I done all in my power by encouragement, incentive and spur to salvage the weak and erring?
- Do I know by NAME and CHARACTER a maximum number of subordinates for whom I am responsible? Do I know them intimately?
- Am I thoroughly familiar with the technique, necessities, objectives and administration of my job?
Do I lose my temper at individuals?
- Do I act in such a way as to make my subordinates WANT to follow me?
- Do I delegate tasks that should be mine?
- Do I arrogate everything to myself and delegate nothing?
- Do I develop my subordinates by placing on each one as much responsibility as he can stand?
- Am I interested in the personal welfare of each of my subordinates, as if he were a member of my family?
- Have I the calmness of voice and manner to inspire confidence, or am I inclined to irascibility and excitability?
- Am I a constant example to my subordinates in character, dress, deportment and courtesy?
- Am I inclined to be nice to my superiors and mean to my subordinates?
- Is my door open to my subordinates?
- Do I think more of POSITION than JOB?
- Do I correct a subordinate in the presence of others?
From: Leading Blog: A Leadership Blog
|
|
