Lessons Learned Integrating Interactive Dashboards into the Audit Process

By Julian Metcalf

Creating an interactive dashboard can be a powerful tool to share your audit results and make information more accessible. I published my first interactive dashboard with Harvey M. Rose Associates for the San Francisco Budget and Legislative Analyst Office in 2015. Since then, I’ve made several more, but not for every audit or study. Making a dashboard can take extra resources and planning. Sometimes it isn’t the right fit, and knowing why you are making a dashboard and what you want out of it is a critical first step. Once you know what you want to achieve with the dashboard and why, you can plan accordingly. I’ve outlined the benefits and challenges of making a dashboard and the key considerations, such as the tools you can use, the workflow impacts, and privacy and security.

Why Bother with an Interactive Dashboard?

Most readers don’t truly read a full report. Some readers skim and many others just look at headings and graphics. This may make a case for short reports full of graphs, but interactive dashboards offer more. Their interactivity means they are more engaging, and users can manipulate and view the data of most importance to them. For example, a department director can isolate just the data related to their staff, or a member of the public can zoom in on a map to an area they care about. This means the data in the dashboard can have rich dimensionality, without taking up printed page space for each iteration and dimension. See Figure 1 below.

Another benefit of interactive dashboards is the access and transparency they offer, though as I discuss further below, this is a risk too. Since most stakeholders never see the extensive data or files that make up our workpapers, there is a lot of information left in the dark. Sometimes this information could help inform operations or public understanding. Instead of publishing hundreds of pages of appendix tables in a report, we can produce an interactive dashboard to collect and share this information. For example, we produced an analysis of San Francisco’s vehicle telematics system and made a dashboard showing the locations of thousands of recorded speeding incidents of staff driving City vehicles (see Figure 2 below). This gave officials and the public a way to see and assess information that was previously only loosely monitored. Due, at least in part, to our analysis and interactive dashboard explaining it, the San Francisco Board of Supervisors passed legislation to improve the monitoring of driving behavior of City-owned vehicles.

The Best Tool for Your Audit Shop

There are three categories that I use to describe dashboard tools. The first, and the one I would recommend to most audit shops, is using a business intelligence or BI application. There are hundreds of software and service choices. Choosing one will be a matter of your budget, goals, and technology environment. I’ll say this again later, but it pays to first talk to your IT department. Your jurisdiction may already have licenses for a BI application or there may be some technical constraints to consider.

The second category of dashboard tools is custom-made visualizations using JavaScript libraries. Large media publications often use these. They offer the ability to make highly customized dashboards and visuals on the publications’ websites and apps, but often require advanced web development skills.

Third are Geographic Information Systems or GIS. Many BI applications have basic map and geospatial features, but a GIS is dedicated to geospatial data and advanced analysis. Many GIS applications offer interactive dashboard outputs focused on spatial information.

Logistics and Setup… Talk to Your IT Department

Before launching an audit that will have an interactive dashboard, you should first talk with your IT department. They can help you plan and answer some important questions such as:

• Is there already a BI application available?
• What BI applications are compatible with your jurisdiction’s enterprise systems, such as finance, human resources, or enterprise resource planning systems, or will you require additional data exporting?
• Where and how will you host any dashboards you issue?
• What security and privacy considerations do you need to consider?

The answer to these varies and you may find there are unexpected limitations. For example, the website where you plan to embed or host your dashboards may have some size constraints. Our dashboards for the San Francisco Budget and Legislative Analyst’s Office cannot exceed 800 pixels wide because of the maximum width allowed on the City’s website, but it can be infinitely long (see Figure 3 below). Once you know any maximum dimensions, it shapes the design of your dashboard and how to use that limited screen real estate.

Fitting Dashboards into Your Workflow

An interactive dashboard is an investment in time and resources, and if you plan to do it in addition to a written report, you need to plan to add it into your existing workflow. There are three main types of workflow integration models that I’ve encountered:

1. Parallel production using the same tool. Use a BI application to make both the interactive dashboard and the charts and materials used in your text-based report. This can save you time if you plan to do it upfront, but if members of the audit team are not familiar with the BI application it will take initial time to train them on the software. This will have a short-term cost but will save time in the long run.
2. After-the-fact or different tools for each. You might get to the end of a project and then realize that it would benefit from an interactive dashboard, or staff on an audit don’t have time to train on new software. Assuming someone can make a dashboard, they can use data from the audit after the fact. That means the audit team will make information and charts in their report using traditional tools, and separately you’ll use the data to make a dashboard at the end. You don’t have to invest in training everyone, but it risks costing more time in the long run.
3. Digital-only audit reports. Instead of making a print or PDF copy of an audit report, you can make the report digital-only where it is hosted as a webpage. The advantage is that dashboards can be embedded directly into the webpage of the report. As readers scroll through the report, all of the charts and graphics they see can be interactive and they can explore the data without leaving the body of the report. This method can be quicker to produce and could draw more people to the audit report. However, it risks limiting access to the information for members of the public without access to the internet, a computer, or a smartphone.

Whichever model you use, it pays to consider it in advance and accept and plan for the time, costs, and risks of any approach.

Format Considerations

I’ve found that there are four main types of interactive dashboards that make sense in our work. See Figure 4 below. The first is an executive summary format, which can share the same summary text and charts that you put in a printed executive summary. However, with an interactive dashboard executive summary, the data in the charts can have rich dimensionality that lets the user zoom into details and trends. The second type is a digital appendix. This format works well when you have a wealth of detailed data that may be valuable to some but would otherwise take too many pages to put in a print appendix. The third is a digital repository of all charts. If you made the dashboard in parallel with the charts for the print report, then the dashboard can serve as a feature-rich and interactive mirror to the main report. The fourth format is the digital-only report with in-line interactive charts using the process described previously.

Static or Ongoing Updates?

You’ll need to decide if the information in the dashboard will be static or updated regularly. Static may make sense for many audit scenarios. The audit will reflect a point in time, and your access to the related data may be temporary. The data you use may be manually generated from observations, surveys, or review of paper records, and you may only intend to gather the information during the point in time of the audit’s scope. On the other hand, there may be circumstances where you want a dashboard with information that will receive future updates. This could include annual risk assessments, audits with planned follow-ups, or situations where the dashboard can serve as an ongoing monitoring tool. To make a dashboard with ongoing updates, you can either manually update the data with new files at future dates or build the dashboard to connect automatically to an existing enterprise system. In this latter case, you’ll again want to work with your IT department.

Security and Privacy

A risk of making dashboards is that you are often releasing more information than normal. This is good for transparency but risks unintentionally releasing sensitive or private information. To prevent this, you need a system in your audit workflow to check all aspects of the dashboard’s data for inclusion of the following:

• Protected Personally Identifiable Information (PII) such as names, addresses, phone numbers, and Social Security numbers,
• Health information that can be associated with individual identifiers,
• Security-sensitive information such as vulnerable physical or IT infrastructure,
• Any draft or incomplete information and documents, and
• Any statutorily protected data or information.

You may need to see and work with sensitive data during your analysis. This is fine, but you need to take precautions when issuing the dashboard. Once your analysis and visuals are complete, I recommend a review process with the following steps: (1) duplicate the dashboard files, (2) save one copy as workpapers with sensitive materials, (3) make the other a publication copy, (4) if sensitive, transform data to only the aggregate level, meaning individual records are not stored, and finally (5) have someone who didn’t make the dashboard review the publication copy thoroughly.

Sharing the Draft Dashboard

Before you issue the dashboard publicly, you may need auditees, counsel, or third parties to review draft audit materials including dashboards. These parties may not have access to your BI application and posting it publicly at this stage would be inappropriate. How you share the draft dashboard privately with select stakeholders will depend on the BI application you used and your IT environment. Here are four approaches that may work:

1. Share files using a BI application that offers a free reader version for viewing.
2. Develop a password-protected site for hosting draft confidential dashboards.
3. Make a view-only account in a web-based BI application to share your hosted draft confidential dashboard.
4. Create a PDF of all frames of your dashboard and export a spreadsheet file of all data to be published.

Final Tips

• Know why you are making a dashboard and the intended audience.
• Expect workflow changes and added time needs.
• Start small with a simple project.
• Train staff and give opportunities for hands-on practice.
• Work with your IT department to determine solutions.
• Keep data security and privacy top of mind throughout.

ABOUT THE AUTHOR

Julian Metcalf is a Principal Analyst with Harvey M. Rose Associates. He manages audits and studies with the Management Audit Division of Santa Clara County, the Budget and Legislative Analyst Office of San Francisco, and other local government clients. He is a Certified Internal Auditor, a Tableau Desktop Specialist, holds an ITIL Foundation Certification in IT Service Management, and earned a Master of Public Administration from New York University.