[This article was first published on Data Analytics and AI Archives - Giles, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.TLDR: This post estimates potential labor market transition pathways based on the similarity of skills, abilities and knowledge required by occupations. The approach broadly follows Bratanova et al. (2026). Like their research, the considered transition pathways are for workers currently employed as truck drivers. Unlike their approach, we’ve used a more recent version of the O*NET database and a different SOC-to-ANZSCO crosswalk. Although the point wasn’t to precisely replicate their results, our approach largely did, with differences in similarity scores likely stemming from our different mapping of the SOC to the ANZSCO rather than from the method itself.Citation: A. Bratanova, C. Mason, D. Evans, E. Schleiger, E. Grimberg, G. Walker, H. Pham, K. Bulled, Truck drivers and automation: A methodology for identifying and supporting workforce transition in the Australian road freight sector, Transport Policy, Volume 176, 2026, 103907, ISSN 0967-070X,.Note: Thanks to the author(s) for willingly sharing their data, code and responding to questions about the methodology. Any errors are my own.Source Data: Correspondence files were developed based on the process outlined in this post, but can be downloaded here. Version 31.0 of the O*NET database was used and the required data files are available here as an archive (data downloaded from O*NET on 17/8/2026).BackgroundIn 2025 I worked as an adviser for a project mapping alternative occupational transition pathways in India to understand longer-term structural changes in the labor market and the impacts of AI. Aside from being an interesting piece of work, it gave me a chance to dig deeper into the literature exploring how to measure human capital and model labor market dynamics.For those unfamiliar with the term human capital, it’s essentially a catch-all category for characteristics that influence a worker’s productivity, such as skills, expertise and emotional intelligence. Unsurprisingly, defining and measuring this is somewhat of an obsession for economists: both to clarify exactly what it means and as there’s solid evidence that it influences social and economic outcomes.Measuring Human CapitalAs enthralling as human capital’s definition might be, I’ll just note that until recently it was conceptualized as some combination of education and experience.1 This isn’t to say that economists believed it was this simple, but just that this was the empirical shorthand that was often used by researchers.Enter Autor, Levy and Murnane (2001)2 who posited that occupations and the demand for skills could be better conceptualized through tasks, such as moving an object, executing a calculation, communicating a piece of information and resolving a discrepancy. For instance, a truck driver might need to operate a vehicle, monitor its condition, plan a route, keep records and deal with customers at delivery. This meant measuring human capital based on what a worker can do rather than the credentials and experience they hold. Although a lot of ground has moved since then3, the basic idea is what makes occupations comparable at all: if a job is a collection of tasks, two jobs built from similar tasks are jobs a worker could more easily move between. A Primer on the O*NETThe US Occupational Information Network (O*NET) (and its predecessor) provides definitions and ratings across a set of characteristics for more than 800 occupations, mapped to the US Standard Occupational Classification (SOC):Every occupation requires a different mix of knowledge, skills, and abilities, and is performed using a variety of activities and tasks. These distinguishing characteristics, or “descriptors”, of an occupation are collected, codified, and described by the O*NET Content Model, This hierarchical model starts with six domains (or categories), describing the day-to-day aspects of the job and the qualifications and interests of the typical worker.Abilities describe the attributes that relate to who a worker is and how they work; while skills and knowledge entail what a worker needs to know in order to perform tasks required in an occupation (see here for more).Source: https://www.dol.gov/agencies/eta/onetThe figure below presents an example of abilities ascribed to Lawyers (23-1011.00). Ratings are provided for each occupation. Speech Clarity, for instance, is provided a rating for both Clergy and Lawyers. Each characteristic is also divided into smaller categories and sub-categories based on common themes: Oral Expression sits under Verbal Abilities, which sits under Cognitive Abilities. This groups more comparable statistics with one another. Source: https://www.onetonline.org/help/online/scalesEach characteristic is then rated based on its level and importance. Importance is meant to measure how critical a characteristic is to a job, whereas the level signifies the level of proficiency required / complexity of the task. For instance, the skill of speaking is important for both lawyers and paralegals, but lawyers are expected to have a higher Level of speaking skill compared to a paralegal (see here):Source: https://www.onetonline.org/help/online/scalesComparing OccupationsBy measuring a detailed set of standardized characteristics, the O*NET makes it possible to and ask more interesting questions about human capital and the labor market. Understanding how similar jobs are to one another is one such example. Workers should find it easier to transition across occupations that require a similar set of skills, abilities and knowledge, which can be a useful thing to know if it’s expected that some industries will face structural changes or be disrupted by new technology, such as artificial intelligence.This is the point of similarity scores: to provide a proxy for how similar two occupations are and the ease of transitioning between them. Like Bratanova et al. (2026), we’ll calculate similarity scores by comparing abilities, skills and knowledge. For the sake of brevity, we won’t consider the viability of transition paths, which will depend on wage differentials, employment demand and other costs associated with making the move. But, Bratanova et al. (2026) did and you should too if you’re intending to use similarity scores in the real world.Methodological DifferencesOne of the motivations for writing this post (and arguing with Claude about the code) was to improve my understanding of the O*NET and build a baseline for future analysis. It’s therefore intended to demonstrate Bratanova et al.’s (2026) approach, rather than precisely replicate their results. The methodology applied here also differs in three important ways:We use a later version of the O*NET database.Our ANZSCO to SOC mapping differ to theirs. This is partially due to updates made to the classification standards and as our SOC to ANZSCO mapping is based on making literal joins across the published correspondences (see here).Bratanova et al. (2026) assign one O*NET occupation to each ANZSCO group, where ours sometimes assigns several. That means averaging characteristics across occupations before comparing them, resulting in our scores describing a typical job in each group rather than a single representative occupation.Project SetupThe code below loads the required packages and defines some the assumptions used throughout the analysis.library(tidyverse)library(janitor)library(readxl)library(ggridges)# Where the data lives.ref_dir_data select(anzsco_label_b, gap_both_mean)sum_onet_gaps |> ggplot(aes(x = gap_both_mean)) + geom_histogram(bins = 60, fill = ref_col_grid) + geom_vline(data = dta_gap_demo, aes(xintercept = gap_both_mean), colour = ref_col_primary, linewidth = 1) + geom_text(data = dta_gap_demo, aes(x = gap_both_mean, y = Inf, label = anzsco_label_b), colour = ref_col_primary, hjust = -0.05, vjust = 1.6, size = 3.4) + theme_minimal(base_size = 11) + theme(panel.grid.minor = element_blank(), plot.title.position = "plot") + labs(title = paste0("Average skill gap from ", ref_occ_origin, " to every other unit group"), subtitle = "Scores run from 100 at the smallest gap to 0 at the largest", x = "Average gap across level and importance (0-100)", y = "Number of unit groups")Examining ResultsHard-Coding Published ResultsBratanova et al. (2026). p. 6The code below hardcodes Table 2 from Bratanova et al. (2026).Note: ANZSCO codes were matched approximately based on job titles, not official correspondences.# Table 2, with each published occupation matched by hand to its closest ANZSCO# 2024 unit group. NA means no close equivalent was identified. rlt_sim_published