-
Notifications
You must be signed in to change notification settings - Fork 1
feat(founder): add deterministic founder validation pipeline scaffold #23627
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,32 +1,21 @@ | ||
| # repo_assumptions.md | ||
|
|
||
| ## Verified | ||
| - Node.js 18+, pnpm, and Docker Compose are documented in repository guidance and root tooling. | ||
| - GraphQL + REST API surfaces are present. | ||
| - Multi-agent architecture code exists under `src/agents/`. | ||
| - Governance artifacts exist under `docs/ga/`. | ||
| - Canonical path map assumptions from planning were validated against current tree where possible. | ||
|
|
||
| ## Assumed (now validated) | ||
| - `src/agents` exists. **Validated** | ||
| - `apps/web` exists. **Validated** | ||
| - `.github/workflows/ci-core.yml` exists. **Validated** | ||
| - `.github/scripts` exists. **Validated** | ||
| - `tests/e2e` exists. **Validated** | ||
| - Feature-flag framework exists in some form. **Deferred pending targeted implementation wiring** | ||
| - `.github/workflows` structure exists. | ||
| - `src/api`, `src/agents`, `src/connectors`, and `src/graphrag` are present. | ||
|
|
||
| ## Assumed | ||
|
|
||
| - No existing "founder intelligence" pipeline under `src/agents`. | ||
| - No structured business validation schema for founder idea intake. | ||
| - No lightweight financial modeling module specific to founder pipeline validation. | ||
|
|
||
| ## Must NOT touch | ||
|
|
||
| ## Must-not-touch files | ||
| - Package manager lockfile unless a dependency addition is explicitly required. | ||
| - Existing `docs/ga` pilot artifacts unrelated to this feature. | ||
| - Governance policy baselines unless extending with additive controls and tests. | ||
| - Release workflows unless change is explicitly scoped to release operations. | ||
| - `src/graphrag/core` retrieval logic. | ||
| - Existing CI gate naming conventions. | ||
|
|
||
| ## Validation checklist before PR1 | ||
| - Confirm exact frontend feature location (`apps/web` feature directory and conventions). | ||
| - Confirm API server routing conventions for SSE endpoints. | ||
| - Confirm required status checks in branch protection for this branch. | ||
| - Confirm evidence schema and artifact output locations for run reports. | ||
| - Confirm whether `.github/scripts` or root `scripts/` is canonical for drift/ops additions. | ||
| ## Validation checklist | ||
|
|
||
| ## Readiness assertion | ||
| This recon step establishes present-state facts and constrains future implementation scope to additive, feature-flagged delivery with governance-compatible evidence and policy gates. | ||
| - Confirm schema conventions in `src/graphrag`. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Checklist points to the wrong schema area for this feature. Line 19 references 🤖 Prompt for AI Agents |
||
| - Check evidence ID format consistency with existing governance scripts. | ||
| - Verify agent orchestration interfaces before wiring broader flow. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| import { describe, expect, it } from "vitest"; | ||
|
|
||
| import { FOUNDER_VALIDATION_THRESHOLD, buildFounderEvidenceId } from "../schemas"; | ||
| import { runFounderPipeline } from "../pipeline"; | ||
|
|
||
| describe("founder pipeline", () => { | ||
| it("builds deterministic evidence IDs", () => { | ||
| const first = buildFounderEvidenceId("problem", "acme-billing-delays"); | ||
| const second = buildFounderEvidenceId("problem", "acme-billing-delays"); | ||
|
|
||
| expect(first).toBe(second); | ||
| expect(first.startsWith("EVID-FOUND-problem-")).toBe(true); | ||
| }); | ||
|
|
||
| it("produces stable report artifacts and enforces threshold", () => { | ||
| const output = runFounderPipeline({ | ||
| title: "Automated customer collections assistant", | ||
| summary: "Reduce collections cycle time for SMB finance teams.", | ||
| problem: { | ||
| customer: "SMB controllers", | ||
| pain_point: "Manual collections workflows lose 10+ hours/week", | ||
| existing_alternatives: ["Spreadsheets", "Generic CRMs"], | ||
| }, | ||
| evidence: ["interview-01", "survey-02"], | ||
| }); | ||
|
|
||
| expect(output.stamp.deterministic).toBe(true); | ||
| expect(output.validation.threshold).toBe(FOUNDER_VALIDATION_THRESHOLD); | ||
| expect(output.validation.score).toBeGreaterThanOrEqual(0); | ||
| expect(output.validation.score).toBeLessThanOrEqual(1); | ||
| expect(output.metrics.latency_budget_ms).toBe(2000); | ||
| }); | ||
| }); |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| import { FounderIdeaInput } from "./schemas"; | ||
| import { buildFounderMetrics } from "./scoring"; | ||
| import { scoreFounderIdea } from "./validators"; | ||
|
|
||
| export interface FounderPipelineConfig { | ||
| founder_pipeline_v1: boolean; | ||
| } | ||
|
|
||
| export interface FounderPipelineOutput { | ||
| report: { | ||
| title: string; | ||
| summary: string; | ||
| recommended_action: "advance" | "revise"; | ||
| }; | ||
| validation: ReturnType<typeof scoreFounderIdea>; | ||
| metrics: ReturnType<typeof buildFounderMetrics>; | ||
| stamp: { | ||
| deterministic: true; | ||
| feature_flag: "founder_pipeline_v1"; | ||
| enabled: boolean; | ||
| }; | ||
| } | ||
|
|
||
| export function runFounderPipeline( | ||
| input: FounderIdeaInput, | ||
| config: FounderPipelineConfig = { founder_pipeline_v1: false } | ||
| ): FounderPipelineOutput { | ||
| const validation = scoreFounderIdea(input); | ||
| const metrics = buildFounderMetrics(input); | ||
|
|
||
| return { | ||
| report: { | ||
| title: input.title, | ||
| summary: input.summary, | ||
| recommended_action: validation.passed ? "advance" : "revise", | ||
| }, | ||
| validation, | ||
| metrics, | ||
| stamp: { | ||
| deterministic: true, | ||
| feature_flag: "founder_pipeline_v1", | ||
| enabled: config.founder_pipeline_v1, | ||
| }, | ||
|
Comment on lines
+35
to
+43
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Feature flag does not currently gate decision output. Line 35 can return Proposed fix export function runFounderPipeline(
input: FounderIdeaInput,
config: FounderPipelineConfig = { founder_pipeline_v1: false }
): FounderPipelineOutput {
+ const enabled = config.founder_pipeline_v1;
const validation = scoreFounderIdea(input);
const metrics = buildFounderMetrics(input);
return {
@@
- recommended_action: validation.passed ? "advance" : "revise",
+ recommended_action: enabled && validation.passed ? "advance" : "revise",
@@
- enabled: config.founder_pipeline_v1,
+ enabled,
},
};
}🤖 Prompt for AI Agents |
||
| }; | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| import { createHash } from "node:crypto"; | ||
|
|
||
| export interface ProblemDefinition { | ||
| customer: string; | ||
| pain_point: string; | ||
| existing_alternatives: string[]; | ||
| } | ||
|
|
||
| export interface FounderIdeaInput { | ||
| title: string; | ||
| summary: string; | ||
| problem: ProblemDefinition; | ||
| evidence: string[]; | ||
| } | ||
|
|
||
| export interface ValidationResult { | ||
| score: number; | ||
| passed: boolean; | ||
| threshold: number; | ||
| evidenceIds: string[]; | ||
| checks: { | ||
| problemDefined: boolean; | ||
| evidenceBacked: boolean; | ||
| alternativesListed: boolean; | ||
| }; | ||
| } | ||
|
|
||
| export const FOUNDER_VALIDATION_THRESHOLD = 0.7; | ||
|
|
||
| export function buildFounderEvidenceId(stage: string, payload: string): string { | ||
| const hash = createHash("sha256").update(payload).digest("hex").slice(0, 10); | ||
| return `EVID-FOUND-${stage}-${hash}`; | ||
| } | ||
|
|
||
| export function validateProblemDefinition(problem: ProblemDefinition): ValidationResult["checks"] { | ||
| return { | ||
| problemDefined: Boolean(problem.customer.trim() && problem.pain_point.trim()), | ||
| evidenceBacked: true, | ||
| alternativesListed: problem.existing_alternatives.length > 0, | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Useful? React with 👍 / 👎. |
||
| }; | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| import { FounderIdeaInput } from "./schemas"; | ||
| import { scoreFounderIdea } from "./validators"; | ||
|
|
||
| export interface FounderMetrics { | ||
| latency_budget_ms: number; | ||
| memory_budget_mb: number; | ||
| cost_budget: "negligible"; | ||
| score: number; | ||
| passed: boolean; | ||
| } | ||
|
|
||
| export function buildFounderMetrics(input: FounderIdeaInput): FounderMetrics { | ||
| const validation = scoreFounderIdea(input); | ||
|
|
||
| return { | ||
| latency_budget_ms: 2000, | ||
| memory_budget_mb: 200, | ||
| cost_budget: "negligible", | ||
| score: validation.score, | ||
| passed: validation.passed, | ||
| }; | ||
| } |
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,25 @@ | ||||||||||||||||||||||||
| import { | ||||||||||||||||||||||||
| FOUNDER_VALIDATION_THRESHOLD, | ||||||||||||||||||||||||
| FounderIdeaInput, | ||||||||||||||||||||||||
| ValidationResult, | ||||||||||||||||||||||||
| buildFounderEvidenceId, | ||||||||||||||||||||||||
| validateProblemDefinition, | ||||||||||||||||||||||||
| } from "./schemas"; | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| export function scoreFounderIdea(input: FounderIdeaInput): ValidationResult { | ||||||||||||||||||||||||
| const checks = validateProblemDefinition(input.problem); | ||||||||||||||||||||||||
| const evidenceIds = input.evidence.map((item) => buildFounderEvidenceId("validation", item)); | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| checks.evidenceBacked = evidenceIds.length > 0; | ||||||||||||||||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
In Useful? React with 👍 / 👎.
Comment on lines
+11
to
+13
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Whitespace-only evidence can incorrectly pass validation. Line 13 treats any array entry as evidence, including blank strings. Normalize and filter evidence before generating IDs/checking Proposed fix- const evidenceIds = input.evidence.map((item) => buildFounderEvidenceId("validation", item));
-
- checks.evidenceBacked = evidenceIds.length > 0;
+ const normalizedEvidence = input.evidence
+ .map((item) => item.trim())
+ .filter((item) => item.length > 0);
+ const evidenceIds = normalizedEvidence.map((item) =>
+ buildFounderEvidenceId("validation", item)
+ );
+
+ checks.evidenceBacked = normalizedEvidence.length > 0;📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| const satisfiedChecks = Object.values(checks).filter(Boolean).length; | ||||||||||||||||||||||||
| const score = Number((satisfiedChecks / 3).toFixed(4)); | ||||||||||||||||||||||||
|
|
||||||||||||||||||||||||
| return { | ||||||||||||||||||||||||
| score, | ||||||||||||||||||||||||
| passed: score >= FOUNDER_VALIDATION_THRESHOLD, | ||||||||||||||||||||||||
| threshold: FOUNDER_VALIDATION_THRESHOLD, | ||||||||||||||||||||||||
| evidenceIds, | ||||||||||||||||||||||||
| checks, | ||||||||||||||||||||||||
| }; | ||||||||||||||||||||||||
| } | ||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Stale assumptions now conflict with merged founder scaffold.
Line 8–Line 10 state these modules do not exist, but this PR introduces them under
src/agents/founder. Please update these bullets to avoid misleading follow-up work.🧰 Tools
🪛 LanguageTool
[style] ~10-~10: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ation schema for founder idea intake. - No lightweight financial modeling module s...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
🤖 Prompt for AI Agents