Subscribe to Email Updates

Quality Creates...Perspective

Metrics to Help You Gauge Your Safety Management System's Effectiveness

Rachel Beavins Tracy
by Rachel Beavins Tracy on Thu, Oct 20, 2016

Since 1998, the aviation industry and the U.S. Federal AviationMetrics to help you guage SMS effectiveness.jpg Administration (FAA) have reduced the fatality risk of air travel by a full 83 percent. The FAA’s goal is to cut that risk in half by 2025, one reason why the agency is requiring most commercial airlines to implement a Safety Management System (SMS) by 2018.

Achieving such ambitious safety goals calls for more than just a “check the box” approach, requiring companies take a hard look at whether or not their SMS is truly effective.

With that in mind, this post offers a variety of metrics you can use to measure SMS effectiveness.

Employee Training

Employee training is the bedrock of your safety program. Human behavior may seem impossible to measure, however there are some key measures that reveal a lot about the effectiveness of your employee training program:

  • Test results: Using the SMS to analyze the results of post-training assessments or quizzes can help you identify weaknesses in your training program.
  • Number of training course updates: If you aren’t updating employee training course requirements regularly, you’re missing opportunities to share lessons learned.
  • Number of changes that trigger new training requirements: It’s essential that employees are trained up on any process changes. Use your SMS to compare the number of change management initiatives in the past year against how many of them involved new training requirements.

Incident Management

Many safety professionals treat incident management as a rear-facing process. A truly effective SMS takes a proactive approach to incidents, using data and lessons learned to minimize the chance of recurrence.

Metrics that indicate whether your incident management program is working include:

  • Repeat incidents: Repeat occurrences are a sure sign you’re missing something, and likely not identifying the true root cause of incidents.
  • Proportion of incidents assessed for risk: Without identifying high-risk incidents, you have no way of knowing which ones need action first.
  • Time required for corrective action: Taking too long to resolve problems just increases the likelihood of recurrence. An automated SMS routes corrective action requests for you, providing alerts when tasks are overdue.

Audits

It’s not enough to have safety plans, procedures and training. You also need an effective audit program to see where things are (and are not) working so you can make adjustments. This is the all-important “check” step in the plan-do-check-act process.

The following audit metrics are particularly revealing:

  • Number of repeat noncompliances: Recurring noncompliances indicate a problem with your audit follow-up system. Your SMS should allow you to launch corrective actions directly from the audit record to prevent this issue.
  • Proportion of high-risk findings: If you consistently identify lots of high-risk noncompliances, it’s a sign that something is missing in your policies or training process.
  • Consistency between internal and external audits: If external audits regularly uncover more noncompliances than internal audits, you probably need to reevaluate your audit procedures and checklists.

Risk Management

Many companies mistakenly equate risk assessment with risk management, when the truth is risk assessment is just one part of a continuous cycle.

Metrics to help you analyze your risk management efforts include:

  • Hazards or observations reported in the SMS: This tells you whether people are engaged in the process, or if you need to offer incentives or quotas to boost SMS participation.
  • Number of new controls implemented: Looking at the number of new controls instituted over the past quarter or year says a lot about your investment in risk management.
  • Proportion of changes that triggered a risk assessment: Change introduces significant risk for an organization, which is why it’s critical each change management initiative involve risk assessments throughout the process.

At the end of the day, an SMS is just a tool—what really matters is how you use it. To make real progress, aviation companies must dig deeper into their metrics and ask if their safety programs are achieving their intended goals.

Learn how an integrated airline management system can improve your organization

Leave a comment

Rachel Beavins Tracy
Rachel Tracy is a writer for ETQ with expertise in environmental, healthcare and technology topics. She has a master’s degree from Vanderbilt University and has been writing for businesses since 2008.
Written by Author

Related posts

How Wearable Technology Can Improve Aviation Safety

According to Forbes, the wearable tech market will be worth $34 billion by 2020, with experts predicting over 400 million...

Rachel Beavins Tracy
By Rachel Beavins Tracy - October 17, 2017
How Disruptive Technology Can Transform Aviation Safety

Uber recently announced its intention to develop Uber Elevate, a new air taxi service using helicopters to transport people...

Rachel Beavins Tracy
By Rachel Beavins Tracy - June 13, 2017
Using Employee Perception Surveys to Measure Aviation Safety Culture

Many companies mistakenly believe compliance alone controls safety performance. In reality, many studies have proven that...

Rachel Beavins Tracy
By Rachel Beavins Tracy - April 18, 2017