Go To Search
Click to Home
Auditing in the Dark Corners
By Gary Blackmer

A police officer sees a drunken man closely searching the ground near a lamppost and asks if he can help. The man replies that he is looking for his keys.

After a few minutes of looking the officer asks whether the man is certain he dropped his keys near the lamppost.

“No,” he says, “I lost the keys somewhere across the street.” “Then why are we looking here?” asks the officer. “The light is much better,” the man responds.

shutterstock_635378759_web.jpg 

That’s a very old joke, and it’s also a parable for auditors.

Do you audit where the light is better? Where you know the data is reliable? Where procedures are established? Where clear criteria exist? Where you’ve audited before? Do you choose your audit topics sitting at your desk without exploring around the agency?

In any organization, think about what percentage of its work is recorded and measured by its managers. A few important things are measured, but the rest is mostly detailed stuff that has little impact on the lives of the public. You can count physical things and discrete actions, but you can’t easily gauge the success of a city’s parks or emergency medical responses or many other public services. If I were asked to put a percentage on it, only about 15 percent of important actions are measured.

If you were the director of that organization, would you just be concerned about that 15 percent, while ignoring the necessary coordination and qualitative assessment of the other 85 percent? And if you received an audit that found some flaws in that data and recommended that you allocate more of your attention and resources to the measurable 15 percent, would you consider that a wise move?

Now consider procedures. Think about the kinds of actions and decisions that can be programmed to be faithfully followed by public servants. I would argue for 15 percent again, especially in the areas like counseling, policing, land use planning, teaching, and prosecution. Multivariate factors require workers to make judgments that can sometimes produce unpredictable and wrong results. Yet, auditors spend a lot of time in these organizations looking at procurement, payroll, and asset management because they have procedures that must be followed.

Here is another example in our audit kitchen. During every peer review I have received or participated in, the examination of CPE hours was a major attraction. One of the reviewers would spend a full day or more checking to make sure every auditor has the required hours for each year. It is such a measurable criterion and plays to the auditor’s lust for data.
I should add that the latest Yellow Book version contains expanded verbiage and expectations about CPE, which will surely worsen the situation. If a staffer comes up two hours short, will that threaten the integrity of the audits? I consider supervision much more critical, but without measures, we can only rely upon peer reviewer judgment. Is there some presumption that when an audit organization or agency can accomplish the simple tasks, then it will get the more complex operations right?

Again, if you were the director looking at the bigger picture, you might be unenthusiastic about greater attention to compliance in these areas, or for writing additional procedures—another classic auditor recommendation.

Okay, a pause here to assure you that I’m not rejecting audits limited to procedures or measures. I don’t reject audits with obvious criteria either—a mandate, a law, a prohibition. Those areas need attention, but they should be balanced against all the other things that could go wrong in the delivery of public services. Sadly, the worst problems sometimes have criterion that must be found in the darker corners.

Auditors can develop a list of comparable jurisdictions to develop costs for services or measurable service quality. They can survey recipients of the services. Academic research may point to best practices. Or public management textbooks can guide agencies to better outcomes. These criteria may lack the specificity of a mandate, but they expand the scope of auditors to where they can produce findings with greater public impact. They can also be as persuasive as concrete criteria if the condition and effect elements of the finding are solid.

I think there is also a temptation to replicate past audits, especially if auditors previously found something. I get it; there are places where I know a fish can be caught, and I go back and cast more flies there.

At entrance conferences, I’ve had agency heads boast about their actions to address a previous audit’s recommendations and invite us to look at the topic again. But I see audits as “rattling doors”—trying to find a new weak spot in an agency that encourages vigilance in management. We can certainly include a brief section in our report to follow up on agency actions, but that’s probably not where the impact will be.

I’ve also been asked at ALGA conferences for our “audit plan” as if it could be applied in another jurisdiction. The field work steps may be enlightening about our methods to address the weaknesses in my city’s bureau, but the findings may not be relevant elsewhere. Following a Portland street map won’t help you find your way in Tucson or Philadelphia.

Sometimes auditors just peer into the darkness from afar and make recommendations. These audits will state that no problem was found, but it would be good to write a procedure or start gathering data. If it is a critical issue, that might be worthwhile; but then again, maybe not.

If auditors have not found a serious problem, it’s difficult to convince an agency to commit its limited resources to modify their information systems, train personnel in its use, and routinely review the data. It’s the equivalent of wanting another streetlight installed in case someone loses their keys nearby.

There is no doubt that groping around in the dark is difficult and unpleasant, but it often produces the biggest audit impacts. Conducting surprise inspections of adult care homes was emotionally difficult, but resulted in several negligent homes getting shut down. Tabulating deployment of 500 patrol officers for four months was pure drudgery, but it pointed out the mismatch with workload. It required patience and perseverance to interview a dozen agencies to learn why they were accomplishing more, but showed a pathway to success for the collections agency.

Yes, in the dark, terrors may also be found. Write down those terrors and make a plan for addressing each one. And later, you may awake in the night, thinking of another that requires some extra planning. Caution and preparedness (e.g. audit risk controls) will reduce your fears, and may not even be needed.

This is my most important question for you: The public doesn’t let agency managers do just the easy stuff, so why should auditors just choose easy audits? When you go into those dark corners, you may find serious public harms. But our objective, cool-headed audit procedures can bring them into the light.


ABOUT THE AUTHOR

Gary Blackmer has been conducting audits for 30 years and recently retired from his position as Director of the Oregon Audits Division. The Division conducts performance, financial, and information technology audits, monitors financial audits of local governments, and responds to hotline allegations. Previously, Blackmer served 10 years as the elected Portland City Auditor, eight years as elected Multnomah County Auditor, a management auditor, and analyst for a variety of state and local agencies. Blackmer is a past-Chair of the Pacific Northwest Intergovernmental Audit Forum, and past-President of the Association of Local Government Auditors. He received the ALGA Lifetime Achievement Award in 2015.