Trust is our foundation

Katalon TrueTest is designed to help quality assurance (QA) teams, developers, and product managers accelerate test creation and improve test coverage. By observing real user interactions in a production or pre-production environment, TrueTest automatically generates robust, maintainable automated tests. This is intended to:

• Generate automated tests: Automatically create test scripts and objects for Katalon Studio based on captured user journeys.
• Identify coverage gaps: Highlight business-critical user flows that are not currently covered by existing test suites.
• Accelerate test maintenance: Simplify the process of updating tests when the application UI changes.

1. Capture

A small JavaScript snippet added to your application passively tracks UI interactions using DOM mutation observers and event listeners. Only structural interaction data is collected—never screenshots or raw user-entered values.

2. Local Filtering & Anonymization

Before leaving the browser, the agent automatically filters and removes sensitive input values. TrueTest does not store sensitive information by default.

3. Secure Processing with Additional PII Safeguards

After transmission, TrueTest performs a second, more comprehensive PII scan and exclusion process. Any remaining sensitive values are removed before the data is persisted. Only cleaned structural interaction data is used to build user sessions, journey maps, and flows.

4. AI-Assisted Generation (Strictly Limited LLM Use)

For selected flows, TrueTest sends a minimal prompt containing only structural metadata (such as DOM attributes and page transitions) to an enterprise LLM endpoint.

TrueTest uses enterprise LLM endpoints with strict data-handling guarantees, ensuring prompts are not logged, stored, or used to train foundation models.

The LLM is used only to generate:

• Object names

• Locator suggestions

• Test case names and descriptions

5. Delivery of Test Artifacts

Generated test objects and test cases are returned to the Katalon Platform and stored in the customer’s Git repository or Katalon Cloud for review and execution.

The following diagram illustrates our multi-layered approach to data protection during the AI processing lifecycle. Your sensitive data is explicitly filtered and anonymized before any interaction with third-party models.

• Zero data retention: We have a contractual, zero-retention and no-log agreement with our AI sub-processor (OpenAI). Customer data submitted for processing is never stored, logged, or used to train or improve third-party AI models. The processing is ephemeral and stateless.
• Proactive data filtering: The TrueTest agent is designed to automatically detect and filter common sensitive data types at the source using regular expressions (regex). This includes Social Security Numbers, emails, IP addresses, phone numbers, credit card information, and passwords.
• Customer-defined exclusions: You have the control to manually define specific UI elements to be excluded from data capture by adding a simple CSS class to your application's code, ensuring sensitive or proprietary information is never captured.
• Bring Your Own Key (BYOK): For customers with stringent data sovereignty requirements, TrueTest supports a BYOK model, allowing you to route all AI processing through your organization's own private LLM instance.
• Complete customer control: AI features are optional. Your organization's administrators have full, granular control to enable or disable AI services at any time.
• Data ownership: You retain ownership of your data. Per our terms of service, any inputs provided to our AI products and any outputs generated by them are considered "Your Data."

• Like all AI systems, TrueTest is a tool to augment human expertise, not replace it.
• Contextual understanding: The AI may not always grasp the full business context or intent behind a complex user flow. Tests generated by TrueTest should always be reviewed by a human QA professional to ensure they align with business requirements.
• Complex dynamic elements: While robust, the AI may occasionally struggle with highly complex, non-standard, or dynamically generated UI elements. Some manual adjustment in Katalon Studio may be required in these edge cases.
• Hallucinations: In rare cases, the AI may generate a test step that is not logical or does not correspond to a real user action. We mitigate this through prompt engineering and by ensuring generated tests are easily editable in Katalon Studio.