Mastering the Data Game – Which Market Surveys do you choose, and why does it matter?

Home / compACT Insights - Compensation Management Research and Case Studies / Mastering the Data Game – Which Market Surveys do you choose, and why does it matter?

masteringdata

Slogging through the job matches and other data in annual market surveys – or even just figuring out which surveys to use – is a daunting and very often confusing task.

This post shows why you may often get mired in a guessing game when it comes to sorting through multiple variations on job titles and salary benchmarks.

By learning the right questions to ask, you will understand how to use market survey data like a veteran, and how to apply your best professional judgment to all the variables.

 

Decisions to Make Before You Look at the Data

 

  • Who is your contact at the survey company if you have questions?

 

  •  Which data element will be the important one for your organization?

 

  1. Does it vary by functional area or employee level?
  2. Are you going to look at the weighted or unweighted results?
  3. Percentiles or averages?

 

  •  Do you have a threshold for the number of participating companies or incumbents before you will include the data point in your analysis?

 

  • If your incentive plan is based on financial results, what do you compare to market when those results are extremely bad, or extremely good? Plan ahead for these circumstances and ensure that you have management’s agreement on the approach.

 

  •  Are you looking at Total Cash Compensation for everyone, or just those who are eligible for an incentive program? Many surveys provide both, so be sure you know what you want.

 

  •  How big a difference from market will be of concern to you? If your pay falls five percent below market, will you consider that a problem? Only for certain key jobs?  

What will you consider significantly above market?

 

  •  Which of your jobs do you consider “benchmarks”? It is key in your analysis to know which jobs you can count on for good market data.

 

Those with the highest number of incumbents—across the broadest breadth of companies—will give you the most reliable results.

 

If you’re an insurance company, you would choose job titles that are consistent across your industry, for example.

 

Remembering Last Year’s Survey – How  & Who in Your Organization Participated

 

  •   Was someone familiar with the jobs involved in the matching?

 

  •   If not, did you at least have good job descriptions?

 

  •   Did you include as many quality data points as possible?

 

  •   Did the managers of the employees in the jobs agree with your matching?

 

Know the Surveys; Separate Wheat From Chaff

Once you’ve evaluated your own participation, take a look at the reputation of the survey—is it typically consistent? Are the results typically higher or lower overall than in your industry?

First, don’t use self-reported data on the internet or any organization’s data claiming to include “major surveys” but doesn’t tell you what they are.

Second, understand that different surveys are relevant to different jobs. Finally, your competitors are likely not the only ones hiring for these roles.

Surveys aren’t always perfect, so cast a critical eye before you bank too much reliance on them. In many cases, they could:

 

  •   Have wacky data and analysis (for example, take a look at year-over-year results for jobs that have not changed descriptions);
  •   Only include a particular type of employer (not at all close to your organization in size, employee demographics or business results);
  •   Not include anyone in your industry when it matters (an engineer in one industry can be significantly different than an engineer in another);
  •   Be new to the scene, with little history to tell if they are any good;
  •   Be too industry-specific (Don’t look ONLY at your competitors when deciding on pay for a general job, like administrative assistant, hr generalist, actuary, etc.); or
  •   Be sloppy at checking the matches from participants.

It’s up to you to ensure that you’re using the right surveys and the right data.

Compensation surveys are hardly foolproof. Because of the subjective decision making that goes into every job match, getting a handle on the big picture and understanding how data is selected will help make the process less mysterious and daunting.

By spending a little time and thought before completing or utilizing a market survey, you can take command of the data with greater confidence, and take the guesswork out of future survey participation. It takes a lot of scrutiny, discernment and a healthy dose of skepticism to master the data game.