feat: refine generate test cases prompt with detailed requirements and structure

This commit is contained in:
Udhay-Adithya
2025-09-27 18:52:19 +05:30
parent 621835a7a1
commit 9d5518e9df
3 changed files with 48 additions and 15 deletions

View File

@@ -57,7 +57,6 @@ EXPLANATION & REPORT REQUIREMENTS
## Preview of Change ## Preview of Change
### Overview ### Overview
### Coverage ### Coverage
## Next Steps
CONTENT RULES CONTENT RULES
- Summary: 68 line paragraph giving high-level context. - Summary: 68 line paragraph giving high-level context.
@@ -67,7 +66,6 @@ CONTENT RULES
- Planned Change: bullet list (max 3) describing ONLY ONE minimal fix. - Planned Change: bullet list (max 3) describing ONLY ONE minimal fix.
- Preview of Change: bullet list showing before → after differences. - Preview of Change: bullet list showing before → after differences.
- Test Plan > Overview: short paragraph; Coverage: 36 bullet validation checks. - Test Plan > Overview: short paragraph; Coverage: 36 bullet validation checks.
- Next Steps: 23 actionable follow-ups (e.g., add monitoring, retry policy, schema validation).
ACTION POLICY (STRICT) ACTION POLICY (STRICT)
- Provide AT MOST ONE action object in the JSON and ONLY if the request failed (HTTP status >= 400 OR evident logic/config error). - Provide AT MOST ONE action object in the JSON and ONLY if the request failed (HTTP status >= 400 OR evident logic/config error).

View File

@@ -19,21 +19,56 @@ CONTEXT
- Request Body: ${body ?? 'No request body provided'} - Request Body: ${body ?? 'No request body provided'}
TASK TASK
- Generate self-contained JavaScript test code and a concise test plan. - Generate self-contained JavaScript test code AND embed the detailed test plan inside the Markdown "explnation" field (no separate test_plan key).
- Code constraints: - Code constraints:
- No external packages or project-specific helpers/variables. - Single self-invoking async function performing all test calls.
- Only standard language features and built-ins (no external libs). - No external packages, test frameworks, or environment-specific globals (ONLY fetch / standard APIs assumed available; if fetch not guaranteed, include a minimal polyfill using node's https but keep inline and minimal).
- Provide immediately executable code (a single self-invoking async function) that runs the requests and prints results with basic assertions. - NO commented-out code (no disabled code blocks, no // or /* */ comments). The code must be clean and production-ready. (You may not include any comments at all.)
- Must define a tiny inline assert function (e.g., function assert(cond, msg) { if(!cond) throw new Error(msg); }) and use it for validations.
- Must print a clear summary per test case and a final summary line.
- Use meaningful placeholders (e.g., YOUR_API_KEY, user@example.com) when necessary.
- Avoid hard coding secrets; instruct via placeholder.
- No randomness unless deterministic seed shown.
MARKDOWN EXPLANATION STRUCTURE (IN "explnation")
- "explnation" MUST be Markdown with the following sections exactly once:
# API Test Plan
## Overview
## Coverage
## Test Data & Placeholders
- Coverage section: bullet list including at minimum: Positive case, Negative case (auth/error), Edge case (e.g., empty payload or boundary), Validation case (schema/field), Performance sanity (basic latency expectation).
- Overview: 46 line paragraph describing intent of tests.
OUTPUT FORMAT (STRICT) OUTPUT FORMAT (STRICT)
- Return ONLY a single JSON object. No markdown, no extra text. - Return ONLY one JSON object with exactly these top-level keys: "explnation" and "actions".
- Keys: - If test generation is possible: actions must be a single-element array containing the code action.
- explnation: A short explanation of what the tests aim to verify. - If insufficient context (e.g., missing URL and method) and you must refuse generation: actions must be [].
- test_plan: { - Shapes:
"overview": string, {"explnation":"<markdown>","actions":[{"action":"other","target":"test","path":"N/A","value":"<FULL_JS_CODE>"}]}
"coverage": ["positive case", "negative case", "edge case: <describe>", ...]
} CODE ACTION REQUIREMENTS
- actions: [{"action":"other","target":"test","path":"N/A","value": "<full JS code>"}] - action: "other"
- target: "test"
- path: "N/A"
- value: full JavaScript code EXACTLY (no markdown fences, no surrounding explanation, no comments).
JAVASCRIPT CODE REQUIREMENTS
- Use const for immutable references.
- Wrap all logic in (async () => { ... })().
- Provide distinct logical blocks for each test scenario in sequence.
- Provide clear console.log statements before and after each scenario.
- Ensure process exits with throw on first assertion failure.
- At end, log "All tests passed" if sequence completes.
- Do NOT include any commented out code lines.
PROHIBITED
- No additional top-level JSON keys.
- No comments inside code.
- No multiple actions.
VALIDATION REMINDER
- Always ensure placeholders are obvious and safe (e.g., YOUR_API_KEY, SAMPLE_USER_ID, LIMIT_VALUE).
REFUSAL TEMPLATE (when off-topic), JSON only: REFUSAL TEMPLATE (when off-topic), JSON only:
{"explnation":"I am Dashbot, an AI assistant focused specifically on API development tasks within API Dash. My capabilities are limited to explaining API responses, debugging requests, generating documentation, creating tests, visualizing API data, and generating integration code. Therefore, I cannot answer questions outside of this scope. How can I assist you with an API-related task?","actions":[]} {"explnation":"I am Dashbot, an AI assistant focused specifically on API development tasks within API Dash. My capabilities are limited to explaining API responses, debugging requests, generating documentation, creating tests, visualizing API data, and generating integration code. Therefore, I cannot answer questions outside of this scope. How can I assist you with an API-related task?","actions":[]}

View File

@@ -467,7 +467,7 @@ class ChatViewmodel extends StateNotifier<ChatState> {
debugPrint('[Chat] Test code added to post-request script'); debugPrint('[Chat] Test code added to post-request script');
_appendSystem( _appendSystem(
'Test code has been successfully added to the post-request script.', 'Test code has been successfully added to the post-response script.',
ChatMessageType.generateTest); ChatMessageType.generateTest);
} }