Posted by: Jeremy Fox | January 12, 2012

Advice: choose the right tool for the job

As an editor, reviewer, and committee member, I have seen many authors and students give the following rationale for their chosen research approach (e.g., choice of study design, analytical method, or response variable):

“This approach has proven useful in other contexts, so we chose to use it in our context.”

This rationale is weak. A skeptic might rephrase it as follows:

“This hammer has proven useful for hammering nails, so we tried hammering on other things to see if any of them were also nails.”

It is not enough to argue that a particular approach is useful in general, or even that it is useful in contexts that vaguely resemble your context. Lots of things vaguely resemble nails, such as screws and one’s own fingers. You should be able to make a very specific case that you have chosen the right tools for your particular job. Choosing your tool first, independent of the job, is a good way to end up using the wrong tool for the job. Your rationale for your research approach should be “We needed to hammer a nail, and so we chose to use a hammer.”

Advertisements

Responses

  1. Oh man, literally laughing out loud, that was great.

    The number of instances of people using some method or other because Joe Schmoe (insert string of publication dates here) did so, is frightening.

  2. Rather:

    “This hammer has proven useful for hammering nails, so we tried hammering on other things in the off chance that any of them were also nails.”

  3. I especially enjoy papers that conclude, pretty much straight up, that the methods have some potentially pretty serious problems, then insert a bunch of irrelevant justifications and other stuff and go ahead and use them away. And how authors respond in print when challenged on it.

  4. tendency to use an established method by a famous somebody overwhelms whether that tool is applicable or appropriate, or not especially if you’re not yet a household name in the field


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Categories

%d bloggers like this: