Disparate Measures

I’m what you call a self-employed person. I’ve been that way since I left my last clinical job in 2013.

Self-employed people have a few perks: We often work out of our homes, tailor work hours to our lifestyles and don’t have bosses in the conventional sense.

We rely on customers—in my case, you and my editor—to tell us how we’re doing. That feedback is always appreciated and usually helpful, but it lacks a quantitative component that would allow us to judge our long-term performance and spot trends.

If those of you who work for healthcare companies ever get letters from patients, you know what I mean. Consider this note received by Jimmy, an AEMT and good friend, from the daughter of one of his patients:

I have worked in the ER for the past 20 years, and seldom have I seen such an impressive medical response… I very much appreciated his calm skills in the midst of a life-threatening situation… He is a wonderful professional who makes the world a better place.

Way to go, Jimmy! Such praise should count in some positive way when his employer evaluates him, but Jimmy’s supervisor faces the same problem with customer comments that I do: There isn’t a good way to combine qualitative feedback, like compliments, with other performance measures to yield all-inclusive evaluations. Nice letters are hard to beat for instant gratification and are much more fun to receive than anything from a law firm, but they don’t really tell us enough about the quality of our work.

I’m betting Jimmy gets graded at least annually on some sort of scale. You know the kind I mean: On a scale of 1–5, with 5 the highest, how would you rate the employee’s effort/attention to detail/dedication to world peace?

If so, is Jimmy’s letter worth a “+1,” or will it just show up as a memo entry? Does he have to be perfect to get a 5 or just excellent? Is 3 average or not very good? Would your boss answer those questions the same way as his boss? Where’s the consistency, the fairness?

Simply adding a comment about Jimmy’s letter to his evaluation—perhaps as a statement supporting a generally favorable numerical assessment—is hardly an ideal solution. Such prose would likely be interpreted differently by Jimmy’s current and future managers. I don’t think it’s possible to set standards for combining words with numbers.

In the absence of policy on this matter, what if some supervisors raised Jimmy’s score to accommodate his nice letter and others didn’t? I faced that sort of unpredictability when I was employed by someone other than “self.” A few of my reviews catalogued and coded specific aspects of my performance, but one simply graded my smile. I earned near-perfect scores marginalized by negative observations and average scores buoyed by effusive compliments. That’s because numerical ratings are often meaningless—nothing more than subjective appraisals force-fit onto arithmetic scales.

In the words of noted vocalist and part-time philosopher Dionne Warwick, “What’s it all about, Alfie?”

First, I think our industry has to decide what we should be rated on. I’m a big believer in focusing on results, rather than effort or affect. If I were running an EMS agency, I’d want to know how well my medics were recognizing manageable conditions, like acute bronchospasm or hypoglycemia, and then treating them.

Second, we need quantitative ways to evaluate outcomes. Customer feedback is nice to have, but it’s almost impossible to merge well-meaning prose into the kinds of performance appraisals that can be compared. We have to be able to do that—compare and judge individuals objectively—or we’ll never know what’s above or below average.

Third, after we review documented weaknesses with employees, we should seek improvement in their performance. That doesn’t mean negative reinforcement is all that matters; we just have to recognize that evaluation strictly for evaluation’s sake won’t close the QA/QI loop.

There are ways to quantify and compare clinical outcomes meaningfully. Just ask me if you want to know more. I’m not sure most people in our business have thought much about measuring quality that way. It’s harder work than reading letters.

Thank-you notes are nice. If you get one, enjoy the moment, but understand that quality improvement comes from constructive changes based on comprehensive, measureable results. Getting there requires much more than nice letters.

Mike Rubin is a paramedic in Nashville and a member of EMS World’s editorial advisory board. Contact him at mgr22@prodigy.net.