Protohaven Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Metrics ======

===== Sources =====


There are a few places that metrics can be found for membership and classes:

**Neon CRM**

  * source of truth for memberships, membership types, membership-related transactions, and clearances earned for individual members
  * also source of truth for which classes were offered to the public, plus registrations for those classes
  * Neon CRM has several reporting features, but they suffer from "black box" syndrome and are very difficult to dig into to understand and trust the results.


**Airtable**

  * Stores some metadata about classes that may not be present in Neon (e.g. which instructor taught the class, intended frequency of class to be run)
  * Can be used to create public "views" of data, e.g. the tool status tables at [[protohaven.org/equipment]]. Depending on the context and counter to the name, views can also be read-write and affect the data.
  * We have a single (admin) account for Airtable - be advised that changing innocuous things like column names will break various automation systems in mysterious ways.

**Google Sheets**

  * The [[https://docs.google.com/spreadsheets/u/3/d/1dM_b1O7Uzj4qwyyJ-uMTmrE6sIcWowYnzYxhhpcWbGI/edit?usp=drive_web&ouid=104731351084800424616|Master Instructor Hours and Clearance Log]] feeds into payroll for instructors and includes info about who passed/didn't pass for classes that provide clearance on the tool.
  * The [[https://docs.google.com/spreadsheets/d/1UrLZV1uAxW4ziLdy02kSXg15jidRYpktgQP8D5qz2rU/edit|Protohaven Welcome and Waiver Form (Responses)]] sheet is a timestamped and somewhat coherent log of members signing in at the front desk, including info about guest sign-in, waiver acceptance, clearances at the time of sign-in etc.
  * Both of these sheets are fed by google form submissions; my long term expectation is that these sheets are converted to instead use Airtable so our data is more centralized and easy to operate on.
  * There's also [[https://docs.google.com/spreadsheets/d/1LvUfmKBXd15k7WLf8OxXyrxU0ib9pM8czyV0BYs4tqs/edit#gid=1549309011|Protohaven Class Feedback (Responses)]] of customer feedback on classes - this is what's anonymized and sent to Discord's class-automation channel.

===== Analysis =====

There's some prior work on understanding the ebb and flow of membership:

  * Various scripts in the [[https://drive.google.com/drive/u/3/folders/1HP1lX4PmWm_EefC0_tdp9oN_Bu-s-9s1|Scripts & Colabs shared drive folder]] are hacked-together attempts to pull data from Neon, Airtable, and Sheets.
  * The [[https://colab.research.google.com/drive/1LlrLj3vkUepvqL6VKYMVqegMED6AZ_KJ?authuser=3|Class Registration Data Extraction]] colab pulls event and registration data from Neon and dumps it to CSV for later analysis. This can be useful for detecting occurrence/frequency of particular classes, occupancy and earnings etc. It doesn't include any walk-ins (e.g. techs backfilling not-quite-full classes) which are charged through Square.
  * The [[https://colab.research.google.com/drive/1UkpGV1on_rSV0aCdptDOi9HLY3PGYf4g?authuser=3|Identify paying, active members for a given period]] colab crawls through all accounts on Neon CRM, accumulates membership information, and uses the data to compute a histogram of new, leaving, returning, and sustained membership on a month-to-month basis. It's not super optimized and takes several minutes to run.
  * Please be careful when executing scripts here, as some of them actually mutate data when you run them.
   
Materialization of these scripts from previous runs:

  * [[https://docs.google.com/spreadsheets/d/17vqm_xuvpfTXa_o6ES3xk-eFWNgULQoFLdtoERLUtdk/edit#gid=0|Membership analysis]] sheet using the output of the paying/active members colab above, plus a couple of rules and charts to track how we're doing on acquisition

===== Creating New Metrics =====

Some questions to answer before creating a new metric or dashboard:

  - What questions do we always need to answer, on a regular enough basis that it's worth paying (time, attention, effort) to maintain a dashboard/pipeline for? What is this frequency/basis? Is the audience just the M&P committee, the whole board, or broader?
  - For each of these questions, can it wait a few minutes, or do we need an answer at any moment? (operational metrics vs higher-level reporting)
  - What would be an ideal "data view" or schema that would support deeper, ad-hoc analysis to answer various questions that emerge from the questions in #1?