Git Auditor
A GitHub repository analysis experiment combining deterministic repository metrics with LLM-assisted evaluation.
Cutting Through GitHub Marketing Fluff
Technical recruiters and engineering managers frequently review GitHub profiles that look impressive on the surface but offer little genuine substance. Profiles are often cluttered with dozens of abandoned tutorial repositories, empty forks, or exaggerated README descriptions that mask minimal actual code. I built Git Auditor as an opinionated forensic audit tool designed to evaluate public GitHub repositories against strict engineering standards. Rather than validating surface-level activity streaks, it distinguishes genuine technical capability from superficial resume dressing.
Deterministic Metrics Before AI Evaluation
Generative AI models are notoriously susceptible to flattery when asked to evaluate code, often praising poorly structured projects as exceptional. I designed Git Auditor around a two-stage evaluation model where cold mathematical metrics precede any AI analysis. The Express backend queries the GitHub REST API to extract hard repository metrics: commit frequencies, contributor counts, file distributions, and documentation completeness. If more than half of a repository consists of boilerplate configuration files, or if an entire project was committed in under 24 hours, the engine applies automatic heuristic score penalties before invoking the LLM.
Calibrated Audits with Fast LLMs
For qualitative repository evaluation, I integrated open-weight language models via Groq's high-speed API, calibrating the system prompt to simulate an experienced senior engineering manager. The model cross-references stated README claims against actual codebase contents. If a project claims to feature advanced machine learning capabilities but includes no relevant dependencies or model scripts, the auditor flags a marketing mismatch. Repositories with substantial code but no documentation are capped at a maximum score of 75, reinforcing the principle that undocumented code cannot be verified.
Stateless Pipeline Design
I deliberately chose to build Git Auditor as a stateless application without persistent database storage. Caching audit results would cause grades to become outdated as developers iterate and improve their repositories over time. Instead, the Node.js backend processes GitHub API payloads entirely in memory, generates deterministic scores, runs the Groq analysis pass, and returns the forensic dossier directly to the React frontend. This keeps the application fast, lightweight, and free of database maintenance overhead.
What Building a Forensic Auditor Taught Me
Building Git Auditor taught me that extracting objective value from generative AI requires anchoring models in deterministic constraints. Unconstrained language models tend to agree with whatever prompt they receive; forcing the model to evaluate code against hard commit data, file ratios, and concrete dependencies produced far more reliable assessments. The project also influenced how I maintain my own public repositories, demonstrating firsthand that clear documentation, transparent commit histories, and meaningful test suites are what truly distinguish solid engineering from superficial presentation.