KPIs for Developer Relations Teams

You can't optimize what you can't measure!

How do I measure the impact of my DevRel team? 

This is one of the most common questions I get from founders I work with. DevRel is not necessarily a new organization, but more companies have started adding in a Developer Relations team as a tactic for driving growth. For Dev-First companies, it’s a no-brainer, you have a team focused on building awareness and excitement amongst your target audience.

Most founders I speak with are curious about how to hire and build out this team and how to measure their impact. DevRel as an organization can have so many different focus areas – Advocacy, Documentation, Education, Support, Content Creation – that often I’ve seen teams go rudderless. What I’ve found is the best DevRel teams have a strong mission and a laser focus on specific outcomes. Picking a core focus area will help you and your team align on the right tactics to achieve those outcomes. 

Ultimately, measuring the impact of your DevRel team comes down to understanding what success looks like for you and your company. Each company will have different goals and objectives, so it’s important to tailor the KPIs and metrics you track to those specific goals. By doing this, you can make sure that your DevRel team is focused on the right things and achieving maximum impact.

What is your DevRel team’s mission? 

Before you begin to measure, you need to determine where you focus. We’ll choose a made up company, Kanjo, building a BaaS for Machine Learning Apps, to help articulate different DevRel missions, why they matter and the KPIs that align (and don’t align) to each. 

• We want to build a strong and vibrant community around Kanjo, our new BaaS for Machine Learning Apps 

• Grow Awareness of Kanjo in the Developer Community 

• We want to build a product that developers love! 

• We want to grow the number of developers building ML applications on Kanjo 

What is the simplest way for you to measure against these outcomes? 

We want to build a strong and vibrant community around Kanjo, our new BaaS for Machine Learning Apps 

In this example, you are looking to build community around the product, Kanjo. Community can be a robust or a flimsy term. For some “Community” means a Slack Channel, for others it is the number of users who have downloaded Kanjo. Community is not simply sharing information on the internet. Community in my definition is 1) An excellent sitcom that was too early for its time 2) Relationships based on mutual values, experiences and interests. I also love the definition from the Stanford Social Innovation Review. 

 Members of a community have a sense of trust, belonging, safety, and caring for each other. They have an individual and collective sense that they can, as part of that community, influence their environments and each other. (source)

Community is, therefore, a set of interrelationships, not a relationship between you and a customer, but their relationships with one another. So what are the best ways to measure this?  

Community Forum Traffic and Engagement 

What is interesting about Developers is that Community Forums are part and parcel of their day-to-day working experience. They are regularly searching StackOverflow for answers to questions, participating in conversations in Slack or Discord groups or passively reading HackerNews threads. 

If you believe that community engagement can be a powerful growth lever for your business and you want to invest in a forum, then you’ll be measuring performance based on different forms of participation in these forums (views, upvotes, posts, comments, feedback). You’ll also want to measure the happiness of users in these forums through NPS or a simple feedback tool like a “was this article helpful” dialogue. 

You can also measure external forum activity – like the AWS forums or StackOverflow – but those don’t give you a good sense of your own community’s engagement. It also doesn’t give you an understanding of the impact of your work. Sure, appearing on other forums at a higher rate is a great signal of success, awareness and widespread usage, but it doesn’t help you understand if your community-building efforts are making an impact. 

How not to measure Community Engagement

• Documentation Visits: If a Developer is using your product, it’s very likely they’re visiting your documentation often. This is not a bad thing, for extremely technical products it’s very common for developers to check and reference uncommon commands, or they could be reviewing documentation for a new feature. In this sense, Developers who are actively using documentation are active product users, but they might not be active members of the community. 

• Downloads or Accounts Created: This is a painful one for me to talk about because I truly do not believe that Account Creations or Open Source downloads are a measure of Community. The number of accounts created gives you a gauge of the product’s awareness in the market. Users who get value from the tool, either by hitting an activation metric or similar, are not members of your community either. They’re just users, and those measures tell you how effective the product is at getting a user to value quickly. So please, don’t use Accounts Created or Open Source downloads as a way to articulate your community size. I know it is very sexy in a board deck or as a way to showcase traction to investors, but they are not the same. 

• Blog Post Views: Just…no. The purpose of creating content on a blog is to drive awareness of your product, educate the developer ecosystem and drive conversions. Website views are not a measure of the community that you create around your product. 

Grow Awareness of Kanjo in the Developer Community 

Awareness is a great mission for DevRel teams. So much of what DevRel teams do is engage with developers as a champion of your product. Activities like conference attendance/presentations, Meetups and Hackathons are great tactics for driving awareness. 

Measuring awareness is SO HARD! There’s many proxies for awareness, but they all come with caveats, so in this case it’s very much a pick your poison. 

1. Organic Website Traffic: This can be a powerful measure of awareness, but if you have a marketing organization, some of the awareness can be attributed to their work, along with the work of DevRel. Additionally, Website traffic is a leading indicator of later stage performance metrics, so many CMOs don’t love using it as a primary metric 

2. Documentation traffic: As referenced above, unique Documentation traffic is a strong indicator of product usage, but it might not be the best indicator for awareness, since awareness happens before the product is used. 

3. Downloads or Accounts created: If you have a free or open source product, this is a great measure of awareness. Why? You’re measuring the signups, not the usage of the product. It’s a good measure that tells you abou general interest in your product. 

We want to build a product that developers love! 

Developer Love is challenging to measure because it’s so subjective. When you’re saying you want developers to love your product you want them to have a great experience and tell everyone about that great experience. Some folks might think this is an obvious goal for DevRel teams, but if you think about the roles DevRel plays in an organization, they’re not often building product. They might be doing support and therefore offering your customers a delightful customer experience there, but overall a Net Promoter Score is measuring the quality of your product and user experience. 

We want to grow the number of developers building ML applications on Kanjo 

1. Activation: This is a measure that’s become more widely discussed in recent years because of the growing popularity of Product Led Growth principles. I love Amplitude’s definition of Activation: “The process of getting new users to the aha moment, and then to the point where you get business value back from them is called activation. Activation matters because users who find value and invest within your product early on will likely stick around longer” How does a DevRel team impact this measure? Typically DevRel teams with this type of goal will own Support, Documentation and Education and have a hand in ushering users through the initial parts of building with the product. (I highly recommend reading Amplitude’s guide to product metrics) 

2. Downloads or Accounts created: This is a great secondary measure of impact because you’ll see how many developers you drive to your product, but you won’t be able to measure their success in using the product through this measure. 

Hopefully this is a good primer on DevRel KPIs that will help you think through your DevRel strategy. I would love to hear thoughts on this piece in the comments!