Hey Anthony, good article. Last year I had the opportunity to lead this effort (Core Competencies) for Riot Games production and roughly used the same approach in terms of milestones. The most helpful thing for me was actually to define a) what is a competency and what is not, we used a razor of it being combination of knowledge skill and ability demonstrated through behavior b) leveling competency was done in abstract from PM levels thus allowing cleaner proficiency, then it was mapped back to PM levels (allowing proficiency levels to be < than job levels was a good thing) c) i developed and used generic proficiency matrix (shows clear levels of proficiency through generic behaviors around different concepts such as risks, opportunities and 20+ other entities) which I think was the most useful during the exercises.

Finally after workgroups finished drafting we performed dry runs — working with select leaders from the org to calibrate levels and proficiency expectations.

I agree with you that context is super important and that the generic models hardly work for this sensitive project.

Thank you fir the article!