Opportunities for Improvement: relentless auditing
By Gary Blackmer
You believe that every recommendation is precious because it took your time, talent, and effort to produce. Yes; but not every recommendation has been implemented. What considerations should be made to follow up on your recommendations?
Setting aside the personal stake auditors may have in their recommendations, the public can benefit when auditors follow up on their recommendations. Most obviously, follow-up encourages action, because managers know that agency implementation efforts will be publicly reported. There could also be a “meta” message here for the public: the follow-up report may show that government is not broken and does indeed take steps to improve its services.
Yet our audit resources are limited and auditors must balance the need for follow-up with the need for audits. If we find that agencies are following through on 85 percent of recommendations, more cursory follow-up may be better, with more auditing efforts spent on finding new problems and writing new recommendations.
One quick anecdote: The state legislators wanted a report on the status of our recommendations because there seemed to be a perception that our audit work had little effect. We had been reporting the general status, as one of our performance measures, running in the range of 80 to 85 percent implemented. When legislators saw our report on the specific recommendations and their status, they lost interest.
Auditors can also benefit with follow-up because it may suggest some weaknesses in the audit itself, which is a learning opportunity. An auditor might learn when some recommendations could be better written to be more detailed, or more general. The level of specificity is directly related to the confidence that an auditor can establish that a particular solution is correct. I have found that more generally worded recommendations give the agency latitude in determining the right strategy, which may be better than any specific direction the auditor might conceive.
An auditor might also learn that a recommendation was not implemented because the audit was not very persuasive. An auditor who can establish only a weak or minimal effect may produce insufficient reason for an agency to invest the time and attention to modify its practices. The auditor should learn that future audits need to be either more persuasive, or to search for a more compelling potential finding.
Auditors must also accept that good public managers will evaluate whether the cost of implementation is simply more than the savings or the risk they can tolerate. Auditors seldom report on the cost to implement recommendations. Follow-up work could better inform auditors about some of the cost considerations.
In some cases, a manager might say that the recommendation is reasonable but of less public benefit than other pressing matters. The manager might have 20 or 30 priority objectives and actions on their plate in the coming year, when the auditor identifies an issue that might be ranked
17th on that list. Should an auditor insist that the manager redirect attention and resources to this less beneficial matter? I have found agencies are more likely to implement recommendations when we choose an audit topic that relates directly to the agency’s mission, not some back-office procedure.
I think some of these back-office topics should be part of the audit mix each year, especially because most recommendations can be easily implemented and shouldn’t interfere with higher priorities. But auditors should selectively examine agency compliance with good practices and procedures and push for corrections where higher-risk fixes can be easily made. But be aware that an agency director may express some frustration that they are trying to hold back the floodwaters and you are telling them how to improve the landscaping.
When the director submits a written response to the audit, I have always considered that memo to be a contract with the public. The memo is the promise made by a public official, asserting that action will be taken to resolve the finding. That also makes auditors duty-bound to escalate the force of our response if we discover during follow-up that no action was taken. Auditors must not allow false promises or deceit to go unchallenged.
Shrewd managers will instead report that the recommendation is still in process of being implemented. For that reason I have wished that managers would publicly and forthrightly state that they decline to implement a recommendation. That declaration, along with a well-reasoned explanation, would close the matter. After all, recommendations have a shelf life and we can’t expect laws, procedures and conditions to remain unchanged for years.
It’s important that everyone understands that auditors make recommendations; we don’t issue commands. I’ve had legislators and citizens asking if my office could benefit from a law or ordinance that mandated departments to implement our recommendations. I thanked them for the support but stated that such a requirement would violate Government Auditing Standards. We present our best case for change in the audit report and later we can follow up and report any actions taken. There is a fine line between advocating for our recommendations and demanding action, as if we were directors of agencies.
In my experience managers never evaded critical issues or serious financial risks. They might try to minimize the audit findings but knew they would lose. Yet, if enormous public risks are at stake, I am a strong advocate for pushing hard, using briefings of elected officials, time with reporters, and sit-downs with editorial boards to describe the importance of the recommendations.
That raises another caution for auditors, something I’ve heard described as “malicious compliance.” A provoked manager can make sure the best audit recommendation fails. In that way the manager may try to discredit your work by showing how impractical your recommendations were. All I can suggest is that you try to establish a good working relationship and conduct your audit work in a manner above reproach. A manager who hates being audited should be given no legitimate reason to complain, and no rationale for falsely proving you wrong.
The timing of follow-up reviews is not a simple matter. Some recommendations require only small changes that can be accomplished by the agency in a matter of days. Other recommendations can require substantial time and effort — modifying enterprise software, getting clarification on federal guidance, developing and administering organization-wide training, coordinating efforts with another department, or hiring and training additional employees to fill vacancies. For the more difficult tasks, follow up at six months may only find the beginnings of implementation, and perhaps no action if new budgeting is required.
You can conduct follow up at a particular time of year, after a specific time has elapsed, or as part of subsequent audits. We found that an annual process, producing a follow-up report at a useful time, can be cost-effective. A quick survey of outstanding recommendations (on paper early in my career, now using something like Survey Monkey) can provide a low-cost update. In addition to a check-box on status, we found a brief narrative from the department of steps taken was good. (The status boxes could include something like fully implemented, in process, decline to implement.) We also found that some skepticism about progress was worthwhile and a phone call to the department could result in a change in status on recommendations we felt needed questioning.
Conducting follow up efforts six months or one year after the audit is done is a good concept but it has practical challenges. First, the interruption of other audit work to conduct follow-up can delay audits underway. Second, issuing a memo or follow-up report on a single set of recommendations is unlikely to attract a reader. It has value, but to a very small audience, in contrast to a single annual report.
Keep in mind that both of these options commit major audit resources if they follow Government Auditing Standards. Of course, you can conduct your work professionally but note in the report those aspects of the standards that were not followed. If the implementation rate is high and audit resources are scarce then auditors should evaluate the payback for all the extra hours to meet Standards. With a high rate of action on our recommendations, I found that combining an annual survey with targeted reviews can ensure continued efforts as well as more honesty in agency reporting.
The third option is to incorporate follow up into the work plan of subsequent audits, which offers an efficient means of conducting the work according to Standards, and as set forth in the planning section of those standards. The big drawback, in this case, is that the follow-up may not be as timely or frequent as needed.
Here is a follow-up problem that was very difficult to address at the state level: Sometimes we discovered that the language in a law produced inefficiencies or counterproductive work for the agencies. I believed that it was important to identify those situations as a cause, with a recommendation to revise the language. I hoped that agencies would take up the matter with the legislators and we could testify at hearings about the finding and recommendation to encourage action. That didn’t happen, and as auditors we were reluctant to draft legislation or appear to be the primary advocate for the change. Perhaps that was the reason legislators lost interest in our report on unimplemented recommendations — because it showed their inaction. (I still wonder whether I overemphasized my role of the independent auditor, reluctant to engage in a political process.)
If and when you abandon hope that a recommendation will be implemented, you still have several options. In my experience, if the recommendation was a money-saver we revived it when financial times were tough. You can issue a quick report titled something like, “Missed Opportunities” by gathering your past unimplemented recommendations that promise savings. Governments with a desperate need for money can often overcome any institutional resistance.
There is another way to revive an unimplemented recommendation. When something goes badly in an area where your recommendation went unheeded, you possess the unassailable, exquisite response: “I told you so.” When a contract went badly, clients suffered, money was lost, service was slowed despite your work...you should feel vindicated. Dig out that audit and that specific recommendation and feel free to share it with elected officials and departments. Managers throughout your jurisdiction will learn two lessons: that they should take steps to ensure that the problem can’t occur in their area, and that there is no escape when they ignore an auditor’s recommendation. But be professional about it, and be careful not to gloat because it is the public that lost, which is more important than your jubilation.
One final anecdote. When I was a rookie auditor we issued a report on substandard paving practices in Portland. We conducted follow up regularly for several years. Then 17 years later, after we had stopped our follow-up, we conducted another audit and found the same problem. In both cases, in an unusual practice, we were invited out to the maintenance building to speak with the paving crews about our audit results. I was in the unique position of telling them that now, as the city auditor, I had to return and tell them the same thing, again. I told them that I expected better. But in my mind I suspected that their attention to quality paving lapsed over the years and would probably lapse again in the future. The ultimate cause is entropy, which increases everywhere, and which auditors continually battle.
ABOUT THE AUTHOR
Gary Blackmer has been conducting audits for 30 years and recently retired from his position as Director of the Oregon Audits Division. The Division conducts performance, financial, and information technology audits, monitors financial audits of local governments, and responds to hotline allegations. Previously, Blackmer served 10 years as the elected Portland City Auditor, eight years as elected Multnomah County Auditor, a management auditor, and analyst for a variety of state and local agencies. Blackmer is a past-Chair of the Pacific Northwest Intergovernmental Audit Forum, and past-President of the Association of Local Government Auditors. He received the ALGA Lifetime Achievement Award in 2015.