How OpenRubric judging works.
This guide covers the judging flow from import to rankings, the core concepts, the configuration behind a managed event, and how to read GitHub timeline signals responsibly.
What OpenRubric is, and isn't.
OpenRubric helps organizers import submissions, define a shared rubric, invite judges, score projects live, review GitHub commit timelines, and publish track and overall winners with clear evidence behind every decision.
It is not a cheating detector and not an “AI judge.” Every score is a human judgment against a criterion, and every award is made by a person. OpenRubric only surfaces evidence for organizers to review.
From Devpost to final rankings.
The end-to-end flow is six steps:
- Paste a Devpost hackathon URL (or upload a CSV / add projects manually).
- Import projects. Only public metadata is read.
- Paste your rubric; OpenRubric turns it into scorable, weighted criteria.
- Invite judges and assign tracks or specific submissions.
- Judges score live; each judge keeps an independent record.
- Review GitHub timelines, resolve review cases, then export winners.
Rubric, tracks, judges, signals.
Rubric. A set of weighted criteria (default 100 points across Innovation, Technical Complexity, Functionality, Design / UX, Impact, and Presentation). Judges score each criterion with an optional per-line comment.
Tracks. Prize categories. Each submission belongs to a track; winners are computed per track and overall.
Judges. Each judge has their own score record. Judges never overwrite each other. The organizer aggregates judge scores into averages.
Review signals. GitHub-timeline observations framed as questions, mapped to a priority: clean, light, needs review, or high priority.
Signals, not verdicts.
When a GITHUB_TOKEN is configured, OpenRubric scans each repo's creation date, first and last commit, pre-event and post-deadline commits, and contributors. Each observation is phrased as something an organizer might ask about. For example:
“GitHub timeline shows 9 commits before the hackathon start. This does not prove a rule violation, but judges may want to ask which parts were built during the event.”
OpenRubric never auto-deducts points and never uses accusatory language. A project with an unresolved high-priority review case cannot be marked a winner until an organizer resolves it.
What powers a managed event.
For a managed or partner event, we handle all of this for you. It is documented here so organizers know exactly what the system touches and where their data lives. The license permits running an unmodified copy yourself to evaluate OpenRubric or to operate your own event; it does not permit redistributing or modifying it.
Data lives in a Supabase project (Postgres, auth, realtime). Judge scores are isolated per judge by row-level security, and every event can be exported to CSV at any time. These are the settings behind a live deployment:
NEXT_PUBLIC_SUPABASE_URLSupabase project URL (auth + database + realtime)NEXT_PUBLIC_SUPABASE_ANON_KEYSupabase anon key for the browser clientSUPABASE_SERVICE_ROLE_KEYServer-only key for imports and aggregationGITHUB_TOKENRead-only token for live GitHub timeline scansGITHUB_API_MODEL_KEYGitHub Models token (free) for AI project summariesOPENAI_BASE_URLOverride the model endpoint (defaults to GitHub Models)AI_MODELModel id, e.g. gpt-4o-miniNEXT_PUBLIC_APP_URLPublic base URL of your deploymentHuman final decisions.
OpenRubric does not determine cheating or automatically penalize teams. It surfaces evidence for human organizers to review.
Want to see it working? Try the judge demo, watch the 2-minute demo, or email openrubric@gmail.com about a pilot.