software:metrics
Table of Contents
Metrics
Sources
There are a few places that metrics can be found for membership and classes:
Neon CRM
- source of truth for memberships, membership types, membership-related transactions, and clearances earned for individual members
- also source of truth for which classes were offered to the public, plus registrations for those classes
- Neon CRM has several reporting features, but they suffer from “black box” syndrome and are very difficult to dig into to understand and trust the results.
Airtable
- Stores some metadata about classes that may not be present in Neon (e.g. which instructor taught the class, intended frequency of class to be run)
- Can be used to create public “views” of data, e.g. the tool status tables at equipment. Depending on the context and counter to the name, views can also be read-write and affect the data.
- We have a single (admin) account for Airtable - be advised that changing innocuous things like column names will break various automation systems in mysterious ways.
Google Sheets
- The Master Instructor Hours and Clearance Log feeds into payroll for instructors and includes info about who passed/didn't pass for classes that provide clearance on the tool.
- The Protohaven Welcome and Waiver Form (Responses) sheet is a timestamped and somewhat coherent log of members signing in at the front desk, including info about guest sign-in, waiver acceptance, clearances at the time of sign-in etc.
- Both of these sheets are fed by google form submissions; my long term expectation is that these sheets are converted to instead use Airtable so our data is more centralized and easy to operate on.
- There's also Protohaven Class Feedback (Responses) of customer feedback on classes - this is what's anonymized and sent to Discord's class-automation channel.
Analysis
There's some prior work on understanding the ebb and flow of membership:
- Various scripts in the Scripts & Colabs shared drive folder are hacked-together attempts to pull data from Neon, Airtable, and Sheets.
- The Class Registration Data Extraction colab pulls event and registration data from Neon and dumps it to CSV for later analysis. This can be useful for detecting occurrence/frequency of particular classes, occupancy and earnings etc. It doesn't include any walk-ins (e.g. techs backfilling not-quite-full classes) which are charged through Square.
- The Identify paying, active members for a given period colab crawls through all accounts on Neon CRM, accumulates membership information, and uses the data to compute a histogram of new, leaving, returning, and sustained membership on a month-to-month basis. It's not super optimized and takes several minutes to run.
- Please be careful when executing scripts here, as some of them actually mutate data when you run them.
Materialization of these scripts from previous runs:
- Membership analysis sheet using the output of the paying/active members colab above, plus a couple of rules and charts to track how we're doing on acquisition
Creating New Metrics
Some questions to answer before creating a new metric or dashboard:
- What questions do we always need to answer, on a regular enough basis that it's worth paying (time, attention, effort) to maintain a dashboard/pipeline for? What is this frequency/basis? Is the audience just the M&P committee, the whole board, or broader?
- For each of these questions, can it wait a few minutes, or do we need an answer at any moment? (operational metrics vs higher-level reporting)
- What would be an ideal “data view” or schema that would support deeper, ad-hoc analysis to answer various questions that emerge from the questions in #1?
software/metrics.txt · Last modified: 2024/06/05 16:47 by smartin015