From the Editor
A New Way to Risk Assess
Vehicle Misuse in City Government
Milwaukee County's Approach
Fraud Risk in a Community Agency
Benefits of Establishing a Fraud Hotline
Raising the Alarm
Register for Training
You are here:
Publications & Media
Local Government Auditing Quarterly
Opportunities for Improvement: We Need to Talk
By Gary Blackmer
I’m convinced that good communications are critical for effective auditors and the managers they audit.
Audits shouldn’t be a furtive activity with a big reveal at the end. Communications must be ongoing. Surprising a department head with your audit results will make it more difficult to get recommendations implemented.
Here I will state the big exception, which are suspicions of criminal wrongdoing or serious violations of personnel policies. Auditors need to re-evaluate their communications strategy based upon the particulars of perilous situations, perhaps contacting ALGA peers for advice.
The Yellow Book addresses communication of audit scope and objective at the beginning of the audit, and audit results at the end, but much communication happens, or should, during the audit.
8.23 Determining the form, content, and frequency of the communication with management or those charged with governance is a matter of professional judgment, although written communication is preferred. Auditors may use an engagement letter to communicate key information early in the engagement.
“Written communication is preferred”? Of course an engagement letter and discussion draft are written, and at the federal level, written is probably preferred, but the federal government is astronomically larger than any local audit office, like Jupiter is to Earth. Working under the general assumption that communications must be written, I think, will limit interaction that is critical to the ultimate success of an audit.
Because you all are auditing in a large variety of jurisdictions, I am cautious about recommending universal practices and just urge you to develop your briefing (and listening) procedures around the who, what, when, where, and why appropriate to your government.
I do have a suggestion about the “where,” though. It sends an important signal to be seen and known in the department you are auditing, so go there for a meeting whenever possible. I don’t necessarily endorse a desk or work area in the place for performance audits because you should be most concerned about frontline activities, which may occur far from that location. Onsite can also limit candid discussions about plans, observations and results, if the audit team members are concerned about being overheard.
On the other hand, working onsite allows agency staff to slip into the office and divulge bad stuff to the auditors, so circumstances and judgment are again important.
I‘ll organize my thoughts about these communications around the “why” because I can emphasize their purpose and importance.
Audit start letters are official notice to the department and contain several key messages: the audit is starting; the team members and in-charge; and request for a meeting time. You can mention your interest in relevant information like annual reports, contact information of the management team, and key data systems.
The entrance conference is a meeting to introduce the team, describe the general audit steps, seek contacts to access data systems, discuss other logistics, and to start scheduling interviews with managers and onsite visits to work locations. Generally describing the phases and their involvement to auditees often reduces their anxiety, especially if they know they will have a chance to review and discuss a draft before it is released.
When the director asked what we would be auditing, I generally said that we would like to get an understanding of the agency and its issues before deciding where an audit could be worthwhile. In all my years, I have never set the scope and objective at this stage of the audit because I haven’t yet learned to be omniscient.
Instead, this is an opportunity to ask the director what large issues were problematic for the agency. Good managers actively seek help from us on complex problems because we can bring powerful analytical tools and a singular focus on a problem. We shouldn’t feel obliged to tackle one of the director’s problems, but they sometimes prove to be the best use of our efforts. When we choose topics that a director actively solicited, we should note that in the report.
COMMUNICATIONS TO ENGAGE THE AGENCY IN SOLVING BIG PROBLEMS
The question of finding big problems to audit doesn’t stop with the director. Managers and line staff have good, if not better, perceptions of the obstacles and frustrations to accomplish their mission. Ask them what is the most difficult barrier to getting their work done. Ask them if there are better delivery methods out there. Ask what worries them the most.
When our interviews and research suggest potential findings, we meet with management to outline the general area and issue. Without getting into details, because we haven’t gathered sufficient evidence or put it through our internal quality control, we describe the narrower field of inquiry and may seek more specific information of them, such as an extract of their management data. We may describe criteria, or discuss methods for obtaining a set of comparable agencies.
The astute director can often discern what auditors could find, by starting an internal conversation and gathering information about any problems in that area. Bad managers will develop strategies to resist. Good managers will understand that the auditors will likely find the same thing and any resistance will be futile.
SENSING COMMUNICATION WITHIN THE AUDITED AGENCY
Audits are structured to start with meetings of top management, then meetings down the rest of the organization. This is a courtesy to the director but also gets auditors access to the rest of the organization and its information. Then when we have developed our findings, we meet again with top management.
We don’t always think about the communications going on within the organization about the audit, but it can be a critical element of success. It’s better for your preliminary findings to be discussed within the organization as your work proceeds because personnel can understand and accept them if they have time to verify the problem and to develop solutions and implementation plans. Auditors always want to shorten the reporting phase, but I’m convinced that organizations are more likely to adopt our recommendations if they have some time.
I’ve learned that bad news doesn’t travel up a poorly managed organization, which is why frontline personnel have to work around problems that are never solved. For various reasons, the bad news gets filtered, diminished, or ignored through the layers of supervision and management. Auditors hear about these problems and, if solved, can produce big benefits for the public. If the findings are a surprise to the director, that’s a sign of weak management, and you and the director may be in for a rough ride.
Here’s an example situation. Management has explained that its procedures are based upon some generalized needs or strategies for solving them. Your data analysis finds that those perceptions are wrong. Talk with one or more frontline supervisors as if it’s an odd puzzle you can’t understand. They may be surprised, or they may agree and even elaborate on the problem. If it’s a well-functioning organization, that information will go to their boss, and on up to the agency head. When auditors point out those problems and managers do their own checking, they will learn the hard truth from their own employees. That means you may get little resistance to your findings at the exit conference. You may not know the extent of information flows until the exit conference, but that’s why the most careful audits can still have fun surprises.
My worst-case situation involved the police where its rank structure interfered with information flows. The chief didn’t hear, or didn’t want to hear, our findings, which we had discussed with many personnel, including the audit liaison, a lieutenant. Many of the officers and supervisors, and even a commander, acknowledged the problem—until the exit conference when they all snapped to alignment with the chief to deny a problem existed. That audit didn’t end well for the chief.
COMMUNICATIONS TO SIZE EACH OTHER UP
These are not formal meetings or planned communications, but your antenna should be up throughout the audit. Auditors need to be alert to signals that agency management will fight an audit. I’ve experienced overt criticisms and questions of competence, slow-walking responses to information requests, and suspended communications. In contrast, I’ve received early acknowledgment of a finding from the director who has already acted to resolve the problems. (The director expressed a faint hope that the auditors would go away but also an acceptance that the work had to be completed and reported out.)
Resistance will require you to make an extra effort to ensure your elements and narrative are backed up with solid evidence. You also need to act carefully and deliberatively, consistent with your standard procedures, to ensure that your competence is unquestioned. Always remember that, as you appear to question the competence of agency management in the audit, they could be looking for opportunities to question yours.
COMMUNICATIONS FOR CONTEXT
Only after many audits did I learn that problems and solutions are not as obvious as I thought. Sometimes the simplest questions reveal surprising root causes of problems and barriers to solutions. Asking, “How did this come about?” may enlighten you on how to understand the context of a program.
And sometimes those root causes make you rethink your approach to an audit. Often, government programs are intended to avoid certain costs, or offer a more effective solution for a smaller subgroup. A good example is mental health services and alcohol and drug treatment. Clients may not all recover and sustain themselves in the long term, but treatment efforts can be cheaper and more effective than a jail bed. Concluding that these treatment efforts are not all effective may miss the broader considerations.
Another common problem is caused by overcorrection. Sometimes people (I’m thinking politicians and editorial writers) think that moderate solutions aren’t good enough, and “turning it up to 11” would be better. As a result, programs may careen between too tolerant and too harsh, too much and too little, too rigid and too flexible. Audits should move the organization to “just right.” Asking larger questions can reveal these kinds of causes, which can be described as fine-tuning service delivery.
Context that explains the reason behind the agency’s situation may exonerate the leadership. It is much better than just recommending that a problem needs solving without understanding the context.
COMMUNICATIONS TO SPOT AUDIT FLAWS
Complex performance audits often contain multiple risk points related to methods and calculations. You cannot develop a sound recommendation if your audit is wrong, and the members of the agency can be the best early sounding board for your work.
Audits may hinge upon a data analysis that runs counter to prevailing beliefs. This kind of enlightenment can produce money savings or better outcomes. If you find some surprising results, consulting with agency analysts can help you distinguish between whether there is a flaw in your analysis or whether you have a eureka finding. I don’t think it’s wise to share workpapers with people in the agency, but you can avoid some pitfalls if you discuss your preliminary results and approach with people in the organization with firsthand knowledge.
I wouldn’t limit your feedback to analysts, but seek out people who can offer insights into the data reliability and relevant front-line perceptions. You should always be alert to puzzlement or disagreement and explore their reasoning. You may be convinced they are wrong, but the arguments they make are likely to arise later. The better you know the arguments, the better you can be prepared to dissuade management. Consider a few extra sentences or charts in your report if you have evidence to head off their claims.
Feasible recommendations are very important. The worst thing an auditor can do is point the agency in the wrong direction. First, it undermines auditor credibility to propose something wrong. Second, it wastes public resources to push a bad solution.
Auditors can’t be expected to know the details of operations and shouldn’t be so prescriptive that better solutions are not considered by the agency. Early communications on recommendation-testing can be critical. Posing questions to staff and managers can also give the agency some beginning thoughts on solutions.
Again, it’s important for you to hear doubts and objections about practicality. I remember the written response to someone else’s poorly done audit, where the agency said the audit recommendation was illegal and cited the unambiguous law. The auditors either missed this issue or ignored the pushback.
Even more terrifying to me is “malicious compliance.” Be very concerned if you hear an agency director tell you through clenched jaw, “Okay, we’ll implement this recommendation exactly as you’ve written, and we’ll see how much better things are.” You know it will not be a sincere effort to improve, but a sincere effort to discredit the audit.
So, be aware of all the information flows. The bigger the organization, and the bigger the needed change, the more communication and patience are required if you want your recommendations to be implemented.
ABOUT THE AUTHOR
has been conducting audits for 30 years and recently retired from his position as Director of the Oregon Audits Division. The Division conducts performance, financial, and information technology audits, monitors financial audits of local governments, and responds to hotline allegations. Previously, Blackmer served 10 years as the elected Portland City Auditor, eight years as elected Multnomah County Auditor, a management auditor, and analyst for a variety of state and local agencies. Blackmer is a past-Chair of the Pacific Northwest Intergovernmental Audit Forum, and past-President of the Association of Local Government Auditors. He received the ALGA Lifetime Achievement Award in 2015.
<<Previous: From the Editor
Next: Decoding COSO>>
Ph: (859) 276-0686
Powered by CivicPlus