Go To Search
Click to Home
How Police Inspired Us to Write Effective Recommendations
By JoJo Cruz

One of the most important events for the City of Austin this year is the labor agreement negotiations for the Police Department. Our office conducted work to support Council in decision-making. One of the projects we did was the follow-up on the police department’s handling of complaints against officers. As we worked on this project, we learned a few lessons about writing recommendations, and in this article we are sharing the framework that can support auditors in writing effective recommendations.
Cruz_1_sm.jpg 


Always remember that the ultimate goal of audit work is to address the issues and improve operations/services. As such, auditors must be able to confirm that actions were taken to implement the recommendations, and the intended results were achieved. In Austin, we even have it in our City Auditor’s definition of success—“Improve Austin City Government.” Through writing effective recommendations, we increase the likelihood of positive change.


BIG PICTURE

Operational management of the audited entity is responsible for addressing and implementing the recommendations. Do we, as auditors, have an interest in seeing through the implementation? Yes we do! What can we do to motivate management to implement recommendations? Quite a few things:
      
  • we can build productive relationship with auditees throughout the audit;
  • we can explain our findings and how we got there at the completion of testing; and 
  • we can involve management to make sure the recommendations are understood, can be achieved, and will address the issue as we are developing our audit recommendations.

Although, all of those items are important, the last part is going to be the focus of the article.


THREE C'S OF AN EFFECTIVE RECOMMENDATION

In our shop, we had a number of audits where our recommendations were never acted upon or implemented. During our follow-up process, as we read the recommendations a few years later, we sometimes had a hard time understanding them ourselves! Concise, Clear, and Constructive recommendations are now the first lens we use to look at our recommendations. Recommendations that contain audit jargon or unnecessary technical wording, abbreviations, and vagueness make recommendations hard to understand, and thus, reduce the likelihood of management implementation. Section A7.01 of GAGAS supplemental guidance has definitions of clarity and conciseness that can be applied to the recommendation language. 

To illustrate our point on not so clear and concise recommendation, we have an example below that came from an actual recommendation we issued:

“To address the first finding, the Public Works Director should ensure that policies and procedures are reviewed and revised to assure sufficient and appropriate documentation is collected, reviewed, and maintained by staff in order to provide reasonable assurance that goods and services were delivered in accordance with contract terms and paid at the correct amount.”

The statements “to assure sufficient and appropriate documentation” and “provide reasonable assurance” are auditors’ language and will certainly confuse management more than help them. For our audit of police department complaints, our recommendations were in line with the three Cs. For example:

“The Police Chief should ensure that all staff are aware of, and comply with, the requirement that all complaints should be sent to Internal Affairs for inclusion in the complaint database.”

Also, imagine if there was a turnover in management. The new management would be busy figuring out their operations and keeping up with its workload. They do want to take action on the implementation of your recommendation from their list, but want to take the shortest way possible. Using the three Cs helps with the succession planning of the implementation.


SMART RECOMMENDATIONS

In the City Auditor’s Office, we adopted the SMART objectives framework to address the 3 Cs of an effective recommendation. The SMART framework is the recognized standard for internal control objectives and includes the elements laid out below. Using the examples from the follow-up on the police handling of complaints, we are presenting how those elements would look in practice.

shutterstock_436032571_med.jpg

  • Specific - Who is/are the parties responsible and accountable for the implementation of the recommendation? The specific department, unit, or function should be identified. GAGAS standard 7.29 also states that “recommendations are effective when they are addressed to parties that have the authority to act.” In the case of the police audit, we addressed most of the recommendations to the Police Chief, coupled with the Police Monitor, who is charged with police oversight, and to the City Manager, who has the level of authority needed to address the interactions between the departments.

  • Measurable - How management will know it has achieved what the recommendation is aiming to accomplish, and how to implement the recommendation to accomplish the desired outcome of the recommendation. As such, the recommendation should be clear about its requirements in addressing the deficiencies identified during the audit. For recommendations to be measureable, we also should be careful with words like “improve,” “consider,” “discuss,” “expand,” and “review.” These words are not “standalone” words if we wanted our recommendations to be measurable. These words should be accompanied by specific requirements on how management should correct or improve the critical deficiencies or risks identified during the audit. It is very easy to meet the letter of the recommendation without achieving the intent of the recommendation. In the audit we followed up on, we had examples of both easily measurable and harder to measure recommendations. The example below shows a measurable recommendation:

“The Police Chief should ensure that data is accurate, complete and consistent. This may include working with the City’s Communication and Technology Management Department to identify and implement updates to the complaint database, including: required fields that cannot be blank when cases are closed; and field level controls to ensure dates are reasonable.” [Underscoring supplied for emphasis]

In this case, management did make multiple improvements, but they did not include the two specific improvements that the audit team had recommended, which are the “required field that cannot be blank when cases are closed”; and “field level controls to ensure dates are reasonable.” Had we not included those requirements, we would have concluded that the letter of the recommendation was implemented. By writing the recommendation containing the requirements for implementation, we were able to insist on the desired improvement.

  • Attainable - What are the existing constraints that could prevent the implementation of the recommendation? Recommendations should consider the financial, staff resources, technology, and/or legal constraints that would make their implementation unlikely or impossible. As such, it is important to recognize and deal with those constraints when developing the recommendation. One example of the perfectly attainable recommendation:

“The Police Chief should revise the record retention schedule to ensure that evidence that could be used in complaint investigations is available to Internal Affairs investigators for at least 180 days.”

  • Relevant - What is the recommendation aiming to correct or improve? Why is it important to do so?

It is important for operational management to see and be convinced that the recommendations will correct the identified deficiency or issues, and that operations will clearly benefit from the recommendations. Therefore, recommendations should be well-supported by facts and clearly connected from these facts. One example is:

“The Police Monitor should review the complaint process, identify barriers people may face when attempting to make a complaint, and implement methods to reduce or eliminate those barriers.” [Underscoring supplied for emphasis]

The recommendation clearly shows relevance because the recommendation is connected to and resolving the identified deficiency, which is to remove or reduce barriers for making complaints.

  • Time-bound - What is the agreed timeline that operational management will act on the recommendations? Usually the timeline is included in the management response to the recommendations. While this is not normally a part of the recommendation’s narrative, an auditor should communicate the benefits of timely addressing the risk, as well as give feedback on the corrective plan proposed by management. Additionally, follow-up work to test the timely implementation is a part of the SMART approach. The example from this audit was the recommendation about entering all complaints that was implemented at the time the audit was presented due to its criticality and attainability.

In summary, putting thought into writing recommendations pays off. It helps to create an “enforceable agreement” with management to improve the services they in turn provide to public. A few basic things to remember: management are not auditors and also change over time, so write in clear language and put enough detail to help them make the improvement. Include the why, who, and what of implementation in your recommendations. Lastly, your recommendations should be SMART – specific, measurable, attainable, relevant, and time-bound. To the side are the results of our follow up of recommendations on the Police Department Handling of Complaints Audit. The ongoing labor negotiations affected the implementation of some recommendations; thus, they were classified as underway.

Cruz_2_thumb.jpg

Writing recommendations using the three Cs and the SMART frameworks contributed to seeing a positive change in this important function.

For reference, the original report can be accessed here:
http://austintexas.gov/sites/default/files/files/Auditor/Audit_Reports/APD_Handling_of_Complaints__September_2016_.pdf

And the follow-up report here:
http://austintexas.gov/sites/default/files/files/Auditor/Audit_Reports/APD_Complaints_Follow-Up_June_2018__v1_.pdf


ABOUT THE AUTHOR

JoJo Cruz brings with him over 35 years of experience in internal audit where his technical qualifications include strengths in operations, performance, financial, risk assessments, project management, information technology, and fraud audits. JoJo gained his first auditing experience as financial audit associate of Price Waterhouse Co., Makati, Philippines. He has worked with the City of Austin - Office of the City Auditor since August 2005. JoJo led the Emergency Medical Services Collection Controls Audit that was awarded the 2012 Knighton Bronze Award–Large Audit Shop by the Association of Local Government Auditors. Prior to joining the City, JoJo was a Senior Supervising Auditor of Ayala Corporation in Makati, Philippines.