ConcentricAI — AI Site Automation
Cross-Site Prompt Execution, Response Capture, and Repeatable Validation
A Python-based automation approach for interacting with multiple AI websites, submitting controlled prompts, capturing responses, and recording execution evidence.
ConcentricAI Multi-Site Playwright Orchestration
Cross-site prompt dispatch, site-adapter isolation, synthetic PII probes, and structured evidence capture.
Executive Summary
ConcentricAI AI Site Automation is a cross-site testing initiative designed to automate the process of opening AI websites, submitting prompts, waiting for generated answers, and capturing the resulting output in a repeatable way. The project addresses the effort involved in manually testing the same prompt or test scenario across several AI services, where page structure, access requirements, response timing, and interaction patterns can differ.
The approach uses Python and Playwright to drive browser interactions. For sites that expose a stable, permitted request interface, a request-based path may be evaluated separately; browser automation remains the common path for sites that require interaction through their user interface. The workflow records useful execution evidence such as the site URL, run status, and response text, including a plain-text response artifact when configured.
Case-Study Summary
| Area | Case-Study Summary |
|---|---|
| Primary goal | Run controlled prompts against multiple AI websites and capture their responses consistently. |
| Core tooling | Python and Playwright; optional HTTP/API checks where an appropriate supported interface exists. |
| Access strategy | Prefer sites that allow unauthenticated use for public test scenarios; handle login-required sites as a separate access state. |
| Evidence | Target URL, prompt/test case identifier, response text, execution outcome, and diagnostic artifacts. |
| Success criteria | A run clearly reports whether a prompt was submitted and a response was captured, or why the scenario could not complete. |
Project Overview
2.1 Problem Statement
Testing AI websites manually is repetitive. A tester must open each site, determine whether it is available without signing in, locate the prompt field, submit a message, wait for the answer, and copy the response into a consistent record. Differences in page layout, asynchronous response generation, bot checks, rate limits, and authentication can make the same workflow behave differently from one site to another.
2.2 Objectives
2.3 Intended Users
Validating cross-site interaction flows, prompt acceptance, and response completion.
Evaluating expected handling and refusal of synthetic sensitive-data patterns.
Reviewing response-capture failures, locator updates, and site-adapter maintenance.
Scope and Test Coverage
| Test Area | What is Checked | Example Outcome |
|---|---|---|
| Site availability | Target URL opens and the page reaches a usable state. | Available / unavailable / navigation error |
| Access state | Whether prompt workflow can proceed without authentication, or requires a session. | No-login / login-required / access blocked |
| Prompt entry | Prompt input or editor can be located and populated. | Input located / locator failure |
| Prompt submission | Submit action is triggered and the site accepts the prompt. | Submitted / submission failed |
| Response generation | Automation waits for response completion or a configured timeout. | Response received / timeout / empty response |
| Response capture | Visible answer text is extracted and written to configured output. | Captured / extraction failed |
| Repeatability | The same scenario can be rerun with consistent identifiers and evidence. | Comparable run records |
| Sensitive-pattern checks | Synthetic PII/PCI/GDPR-style examples can be used to observe site behavior. | Observed response saved for review |
Site coverage is configuration-dependent. A site should be marked supported only after its current flow has been exercised successfully; site names and coverage counts are maintained in an external inventory.
Automation Architecture
The recommended architecture separates the shared execution workflow from site-specific behavior. This keeps navigation, result collection, and reporting consistent while allowing each website to use its own locator strategy and response-completion signal.
| Component | Responsibility |
|---|---|
| Test inventory / config | Stores target site URL, site identifier, access mode, prompt scenario, timeout, and custom settings. |
| Python orchestrator | Selects a site and test case, starts the run, coordinates execution, and records the final status. |
| Playwright browser layer | Opens page, locates prompt control, enters prompt, submits, waits for answer, and extracts visible text. |
| Optional request adapter | Used only when a supported and permitted endpoint is available; not assumed for every website. |
| Session manager | Supports a manually established browser session when authentication is necessary (no hard-coded secrets). |
| Response & evidence writer | Stores response text and metadata: site, URL, timestamp, outcome, and diagnostic information. |
| Diagnostics | Captures failure reason and, where appropriate, screenshots, page state, or logs for debugging. |
concentricai-site-automation/ ├── config/ │ └── sites.yaml # Site URLs, access mode, timeouts ├── tests/ │ ├── prompts/ # Reusable prompt scenarios │ └── test_site_flows.py # Automated test entry points ├── site_adapters/ │ ├── base_adapter.py # Shared interaction contract │ └── <site_name>.py # Site-specific locators and wait rules ├── core/ │ ├── browser_manager.py # Playwright lifecycle and contexts │ ├── session_manager.py # Optional saved-session handling │ └── result_writer.py # Response and metadata output ├── reports/ # Run summaries and response artifacts ├── logs/ # Execution diagnostics └── README.md
End-to-End Test Lifecycle
| Phase | Key Actions | Pass Condition |
|---|---|---|
| 1. Prepare | Load configuration, validate test input, create a run identifier, prepare browser/session. | Inputs are valid and execution can start. |
| 2. Navigate | Open the site URL and wait for a stable page state. | Expected page is reachable. |
| 3. Assess access | Check for prompt UI, login wall, consent dialog, or access restriction. | Access state is classified explicitly. |
| 4. Submit | Find the prompt control, enter the test prompt, and trigger submission. | Prompt is accepted by the interface. |
| 5. Wait and capture | Wait for response completion, extract answer text, handle empty response and timeout. | Non-empty response captured, or clear failure. |
| 6. Persist | Write response text and metadata; save diagnostics on failure. | Artifacts linked to correct site and run. |
| 7. Continue & summarize | Move to the next configured site and summarize overall statuses. | Run ends with per-site actionable outcomes. |
Access and Session Handling
Used for sites where the prompt and response workflow is genuinely available without an account. Highest velocity for broad regression sweeps.
Where permitted, complete the first login manually and reuse a controlled browser session/profile. Never hard-code passwords in source code.
Record the site as login-required, blocked, or unsupported rather than attempting to bypass anti-bot mechanisms or access controls.
Do not assume internal frontend calls are stable public APIs. Use documented or authorized endpoints and respect rate limits strictly.
Test Data and Responsible Validation
The project explores synthetic examples representing sensitive-data categories, including PII, PCI, and GDPR-related patterns. Real customer data is strictly forbidden:
| Category | Safe Test Approach | Handling Guidance |
|---|---|---|
| PII-style patterns | Fictional names, addresses, phone numbers, or identifier-shaped tokens. | Label as synthetic; do not use real individual data. |
| PCI-style patterns | Payment-card test values from approved test sources or invalid synthetics. | Never include real cardholder data, CVV/CVC, or PINs. |
| GDPR-related scenarios | Synthetic personal-data examples checking if the site echoes or refuses data. | Record observed behavior without claiming legal compliance. |
| Prompt-injection probes | Approved, non-harmful test prompts aligned with test plans. | Keep scenarios scoped; document expected vs observed outcome. |
Results, Reporting and Artifacts
| Artifact | Recommended Contents | Primary Use |
|---|---|---|
| Response text (resp.txt) | Captured answer text for a completed prompt. | Quick review and downstream text comparison. |
| Structured run result (JSON) | Run ID, site ID, URL, timestamp, access state, prompt ID, status, duration. | Filtering, aggregation, and regression analysis. |
| Execution log | Navigation, locator, submit, wait, and extraction events with timestamps. | Debugging flaky or changed site flows. |
| Screenshot / page evidence | Failure-state screenshot or minimal diagnostic evidence. | Investigating login walls and locator failures. |
| HTML summary report | Per-site status, response availability, durations, and artifact links. | Readable test-run overview for stakeholders. |
8.1 Suggested Status Model
Challenges and Design Considerations
| Challenge | Impact | Engineering Mitigation |
|---|---|---|
| Different page structures | A generic locator fails across different sites. | Use isolated site adapters and maintain specific locator rules. |
| Dynamic response generation | Fixed sleeps capture partial answers or cause flakiness. | Prefer explicit waits for completion indicators or stable text buffers. |
| Login and signup walls | A site cannot be exercised in an anonymous flow. | Classify access state cleanly and separate authenticated runs. |
| Site UI changes | Selectors break without prior notice. | Capture failure diagnostics and run smoke tests after updates. |
| Rate limits & anti-automation | Runs throttled or blocked. | Conservative concurrency, respectful pacing, record restrictions. |
| Response variability | Non-deterministic AI output. | Assert workflow completion and structure rather than brittle exact matches. |
| Secrets & sensitive data | Saved sessions leak account data. | Keep secrets out of source control and use synthetic inputs. |
Reusability and Future Enhancements
Add a new site entry and adapter without altering the shared Python runner pipeline.
Lightweight recurring checks to detect selector drift or newly erected login barriers.
Parallel multi-site execution calibrated to respect provider rate limits and browser resources.
Compare length, safety refusals, or structural schema across multiple AI vendors.
Execute small stable sweeps on PRs and broad matrix coverage on scheduled nightlies.
Automated TTL policies for prompts, captured answers, screenshots, and session cookies.
Conclusion
ConcentricAI AI Site Automation aims to make multi-site AI interaction testing more repeatable by standardizing navigation, prompt submission, response capture, and evidence collection. Python and Playwright provide a flexible browser-automation foundation, while site-specific adapters and explicit access-state handling help accommodate the differences between AI websites.
The most important engineering outcome is not simply opening a site or sending a prompt: it is producing a traceable result that clearly shows whether the interaction succeeded, what response was captured, and what prevented completion when it did not. With measured run data and a maintained site inventory, this framework supports repeatable QA, response-behavior checks, and future regression automation.