Findings from the Data Analytics Member Survey

The Actuaries Institute’s goal is for members collectively to be at an intermediate level of proficiency in modern data analytics (including machine learning). This would put existing members in line with newly qualified members after recent changes to the examinations. As a membership group, we are progressing to this goal with the greatest strengths in ethics and coding ability. An area to improve is our ability to communicate machine learning concepts to others.

Unsurprisingly, those who practice in data analytics are significantly more proficient than those that do not. Even after allowing for an overlap with data analytics, general insurance members are more proficient than life insurance and other members.

Five out of six members are interested in learning more about data analytics with reduced interest as the time and expense commitment increases. The Institute has suggested DataCamp courses for members who are interested in an online experience and there are other good providers as well. Self-sourced learning can work for data science but requires commitment and time. The Institute is partnering with Macquarie University to offer a certificate course and enrolments have opened for interested members. Many universities offer Masters of Data Science noting these require a significant commitment to complete.

Results

On the proficiency questions, members assessed whether they had a basic level of understanding, an interested level, an intermediate level or an advanced level. A copy of the survey is at the end of the article to assist in interpreting these levels. To provide a numerical average for the results, the levels have been converted to scores with basic having a score of zero and advanced a score of three.

The mean score was 1.3 averaged over all five areas of proficiency. This compares to a target level of intermediate where the score would be two. Members were strongest in coding and ethics with the results shown in the following chart. The score was lowest for the business questions which were about explaining machine learning to others.

Members shared their interest in different educational offerings as follows:

• 117 selected self-paced online courses at a cost of ~$40 per month;
• 57 selected a certificate course at a cost of ~$3,000 for 80 hours training over ten weeks;
• 35 selected a Master of Data Science at a cost of ~$35,000 over two years part time;
• 32 had no interest in courses; and
• 19 provided another response.

Many of the 20 members who provided another response preferred free resources (sometimes provided by their employers).

Member profiles

We asked members as to their practice area and their level of experience so we could identify groups of members with different proficiency and preferences. The charts below show the different responses.

Variation in member groups

To help understand how responses varied across different member groups, we regressed the responses against the features. The analysis was repeated where each member was included in only a single practice group category with no overlap (e.g. those who are data analytics and general insurance).

For data analytics proficiency, the findings were:

• Members working in either general insurance or data analytics were more proficient than other members.
• When single practice groups were used, those with data analytics in their group were significantly better than average. Of those not identifying as practicing in data analytics, members in general insurance were at the average score and those with other practice areas below average.
• There was no significant variation in score based on years of experience since qualification. Some significant variation was observed when experience was used as a single explanatory variable but this effect is actually explained by the variation in practice area.

For interest in a certificate course, we observed similar responses across all groups except for less interest from those with more than 15 years experience. The author has over 25 years post qualification experience and is counting himself as an outlier. The lack of significant variation by practice area is interesting.

There was little variation for interest in Masters courses. In respect of interest in online learning, the two groups who were least interested are a) those that practiced in none of data analytics, life insurance or general insurance and b) those who practiced in data analytics but not general insurance.

Survey Questions

Machine Learning Concepts – tick the level that best applies

• BASIC – I have read about machine learning in the media.
• INTERESTED – I can compare machine learning and traditional techniques.
• INTERMEDIATE – I have applied machine learning in my work.
• ADVANCED – I have worked with a range of machine learning models and can recommend a solution to a colleague or a client.

Machine Learning in Business – tick the level that best applies

• BASIC – I can follow a machine learning discussion at work but would not contribute.
• INTERESTED – I would contribute to a machine learning discussion at work.
• INTERMEDIATE – I can explain to colleagues how machine learning contributes to business objectives.
• ADVANCED – I lead machine learning discussions and am part of a machine learning implementation.

Coding – tick the level that best applies

• BASIC – I know what a programming language does but use a spreadsheet for analysis.
• INTERESTED – I have trained in a programming language and can follow code written by others.
• INTERMEDIATE – I can code in SQL, R or Python and can manipulate data using code.
• ADVANCED – I use functions and objects in my coding, understand database schemas and use version control.

Modelling Techniques – tick the level that best applies

• BASIC – I use traditional actuarial techniques
• INTERESTED – I know the difference between regression and classification models.
• INTERMEDIATE – I have built machine learning models using pre-developed packages.
• ADVANCED – I can customise packages to provide additional modelling features or statistics.

Ethics of Data Analysis – tick the level that best applies

• BASIC – I know that data can be biased and models can produce unexpected outcomes.
• LEARNING – I understand the precautions needed to protect against data leaks which impact individual’s privacy.
• INTERMEDIATE – I understand how a model trained on biased data absorbs those biases.
• ADVANCED – I know the difficulties in making data fully anonymous. I can identify biases in models and remove them.

Interest in data analytics CPD – tick each one that applies

• I am interested in self-paced online courses at a cost of ~$40 per month
• I am interested in a certificate course at a cost of ~$3,000 for 80 hours training over ten weeks.
• I am interested in a Masters of Data Science at a cost of ~$35,000 over 2 years part time
• No interest
• Other – free entry

How many years has it been since you qualified?

• Zero – I am studying now
• 1 to 5 years
• 6 to 10 years
• 11 to 15 years
• 16 to 20 years
• More than 20 years

What are your practice areas?

• Banking
• Data Analytics
• General Insurance
• Health
• Life Insurance and Wealth Management
• Risk Management
• Superannuation
• Other (please specify)

Please provide any comments you have on the questions or potential CPD offerings.

CPD: Actuaries Institute Members can claim two CPD points for every hour of reading articles on Actuaries Digital.