What your existing QA suite misses about Arabic

Your pipeline can prove that a customer paid a bill, downloaded a receipt and opened a service request while testing none of the conditions that make the Arabic interface different. The journey passes because the controls work. The same build can still have a left-to-right root, a meter reference whose groups appear in the wrong order, a fallback font that changes every line break, or an Arabic error state that extends beyond its card.

Those defects are not outside QA. They sit outside the suite's current inputs and assertions. English-only routes, short mock data, DOM snapshots and unreviewed visual baselines all create evidence, but the evidence describes the English product more completely than the Arabic one.

Closing the gap starts with an audit of what each test layer can observe. Do not add an isolated “RTL test” and call the suite complete. Add language, direction, content shape, market format and rendered state to the same coverage model used for routes and browsers.

Your suite may be testing the wrong variant#

An Arabic translation file in the build does not mean the Arabic product entered a test. Look at how the suite reaches each journey.

If authentication always begins at /en/sign-in, a test can miss a redirect that drops the Arabic locale. If the test changes language after loading the English shell, it can miss server-rendered metadata and initial CSS selected from the URL. If a component receives Arabic text while its ancestor remains dir="ltr", the test is exercising translated content inside an English layout.

Record the variant as part of every browser case:

locale + route + state + fixture + viewport + browser

For a utility portal, ar-AE + /service-requests/new + server-error + long-address + 390px + Chromium is a testable case. “Service request works” is not. It leaves the locale, direction, data and failure state implicit.

Start Arabic journeys from their public Arabic URL in a clean context. Verify the root language and direction before the first interaction, then keep checking them after authentication, client-side navigation, refresh, error recovery and sign-out. R001 reports document direction that is missing, contradicted or declared only in CSS. R002 reports a missing, non-Arabic or contradictory html[lang].

Do not combine those into one assertion. lang="ar" identifies language; it does not declare an RTL layout. The HTML dir="rtl" guide explains how document direction is established and inherited.

Unit tests cannot prove rendered direction#

Unit tests remain useful for locale resolution, formatter configuration, translation-key selection and pure state transitions. They are the wrong layer for claims that depend on browser layout.

A DOM emulator that does not perform layout cannot tell you whether a label overlaps an action, whether an element extends past the viewport, which font rendered the Arabic text, or where a mixed-direction identifier appears visually. A test that asserts the source string sees logical character order. The browser may display those characters in a different visual order according to the bidirectional algorithm.

Keep deterministic logic in unit tests:

• Arabic locale aliases resolve to the intended product market;
• required translation keys exist for every state;
• explicit formatter options match the market contract;
• plural branches have fixtures for the categories the product supports;
• locale persistence reducers retain the chosen language;
• component variants choose the approved directional asset.

Move geometry, inherited direction, computed styles, font loading and bidi display into a real-browser layer. This is not an argument for replacing unit tests. It is a boundary: a test runner cannot prove behavior it never renders.

Component tests often omit the inherited context#

A component story can use Arabic strings and still run in an LTR document. That setup catches text expansion but hides direction-sensitive behavior inherited from ancestors.

Create one reusable Arabic wrapper for browser-based component tests. It should set both language and direction, load the production Arabic font and apply the same theme tokens as the application shell.

// UtilityArabicFrame is the test wrapper used by browser component tests.
export function UtilityArabicFrame({ children }) {
  return (
    <main lang="ar" dir="rtl" className="utility-theme">
      {children}
    </main>
  );
}

The wrapper is necessary, but it can also hide application bugs if every test supplies direction manually. Keep separate route-level tests that prove the real shell sets lang and dir. Component tests answer “does this card behave inside an RTL context?” Route tests answer “does the product create that context?”

Test components in states that change geometry: loading, empty, populated, permission denied, validation failed and retrying. A single story with a short heading proves very little about a status card that can receive a three-line notice and two actions.

End-to-end tests follow behavior, not reading order#

An end-to-end case often succeeds when it finds controls by stable role or test ID, regardless of where those controls appear or what language surrounds them. That is a strength for functional testing and a blind spot for Arabic presentation.

A payment test might click the intended button even when its visible label remains English. It might submit a meter-transfer form while Arabic text fields are rendered LTR. It might find an element that is clipped but still present in the DOM. Functional success does not prove that a person could find, read or operate the same control.

Add Arabic assertions beside the behavioral ones:

// page and expect are supplied by the browser test runner.
await page.goto("/ar/service-requests/new");

const root = page.locator("html");
await expect(root).toHaveAttribute("lang", /^ar(?:-|$)/);
await expect(root).toHaveAttribute("dir", "rtl");

const submit = page.getByRole("button", { name: "إرسال الطلب" });
await expect(submit).toBeVisible();
await submit.click();

This case confirms the accessible Arabic name and root attributes before interaction. It still does not judge whether the translation is natural, whether the action hierarchy is clear, or whether a sighted user can distinguish the primary action. Those need visual and human review.

Do not rely only on selectors copied from English tests. Role-based selectors with approved Arabic names make untranslated interactive labels visible to the suite. R011 reports untranslated interactive labels such as buttons, links, menu items and tabs. Use test IDs where copy is intentionally variable, then assert language coverage separately rather than making the untranslated text invisible.

DOM snapshots miss visual bidi failures#

Snapshots serialize source order. Copy operations and accessibility trees also expose logical order. None of those proves that a mixed-direction value appears visually in the order its identifier requires.

Consider a meter reference placed after an Arabic label:

Problem

<p dir="rtl">رقم العداد <span>607-48-9231</span></p>

The plain span creates no bidi isolation. In Chromium 151, measured on 2026-09-24, that three-group Western-digit identifier displays as 9231-48-607 after Arabic text. A snapshot still contains 607-48-9231, so it passes while the customer sees a corrupted identifier.

Better

<p dir="rtl">رقم العداد <bdi dir="ltr">607-48-9231</bdi></p>

The second sample gives the identifier its own left-to-right boundary. It preserves the authored order without changing the surrounding Arabic paragraph. R021 reports unisolated bidi hazards such as phones, emails and Latin tokens inside Arabic text.

This browser fact is specific to the measured Chromium version. Check the same fixture in every supported engine. The bidirectional text developer guide covers the underlying algorithm and cases where dir="auto", <bdi> or an explicit direction is appropriate.

The important suite audit is not whether snapshots exist. Ask whether any test observes visual output for mixed Arabic and Latin or numeric data. If not, DOM coverage and display coverage are being treated as the same thing.

Mock data is too clean#

Fixtures built for English behavior tend to contain short names, one-line messages and identifiers that begin with Latin letters. Those inputs avoid the conditions that expose Arabic defects.

Build a hostile but realistic Arabic fixture pack for each product area. A utility portal needs more than translated labels. Include:

• an Arabic service address that wraps across several lines;
• a digit-leading meter reference such as 607-48-9231;
• a Latin-leading equipment code such as GRID-Q4-716;
• a multi-paragraph disconnection warning;
• an account with no consumption history;
• a failed meter-photo upload with a long recovery message;
• a tariff name containing an intentional foreign-language run;
• dates, units and digits approved for the target market;
• loading, stale, partial, permission-denied and server-failed responses.

The two identifiers serve different bidi purposes. The digit-leading value can reorder next to Arabic when it is not isolated. The Latin-leading code stays intact after Arabic text under the measured browser rule, so it is useful as a control fixture, not as proof that isolation is unnecessary everywhere.

Do not fill every test with the longest possible content. Keep a small ordinary fixture and targeted stress fixtures. When every screenshot is extreme, reviewers lose sight of the default experience and failures become harder to attribute.

Data shape should also match the assertion. A long Arabic sentence is suitable for testing wrapping and line height. It does not test mixed-direction boundaries. A machine identifier tests bidi isolation but says nothing about translation quality.

Visual regression can preserve the first mistake#

A visual baseline answers one question: did these pixels change? It does not answer whether the approved pixels were correct.

If the Arabic baseline was captured before anyone checked document direction, an unchanged LTR layout passes forever. If an icon was mirrored mechanically even though its meaning should stay fixed, the baseline protects the wrong asset. If the browser fell back to a system font during baseline creation, later runs can treat the intended webfont as a regression.

Approve Arabic baselines with a written review target. For each capture, name the component state, viewport, fixture, loaded font and directional decisions. Compare its LTR counterpart, but do not demand a literal mirror. Text length changes, some values remain LTR, and semantic icons follow meaning rather than page direction.

Visual tests are strong at geometry:

• a badge shifts onto a heading;
• a long error is clipped;
• a sticky control covers content;
• an action moves to the wrong logical edge;
• the Arabic font changes line breaks;
• a directional icon differs from its approved variant.

They are weak at invisible or interactive facts. A screenshot cannot prove focus order, accessible names, language metadata, caret behavior or whether content beyond an inner scroller can be reached by keyboard. Pair the image with DOM, focus and interaction assertions.

R041 reports visible text-bearing elements that overlap. R042 reports clipped text when content is wider than its box, overflow is hidden and no ellipsis is present. Those findings identify concrete geometry. They do not approve the information hierarchy or the translation.

Screenshots do not cover transient states by default#

Visual suites capture whatever state the script reaches. Toasts that disappear, menus that open after keyboard input, delayed server errors and font swaps can all sit outside the baseline set.

Name transient states explicitly and hold them stable for capture. Use controlled API responses rather than racing a live timeout. Disable animation through a test setting rather than taking a screenshot at an arbitrary frame. Wait for the product's settled condition, not a generic delay.

For the utility portal, the failed photo upload deserves its own state. The test should create the failure, wait for the Arabic error and recovery action, then capture the component at narrow and wide widths. The corresponding interaction test should move focus through the recovery path. One screenshot of the initial empty uploader covers none of that behavior.

Be careful with loaders. Freezing a skeleton can help test its direction and geometry, but a permanent loading fixture does not prove that the transition into loaded content preserves focus or avoids layout shifts. Capture both states and test the transition separately when it matters to the task.

Accessibility automation sees structure, not comprehension#

Automated accessibility tests can catch missing names, invalid relationships and some language metadata. They cannot tell whether Arabic wording is understandable or whether a spoken mixed-language value sounds coherent.

Language and direction still belong in accessibility coverage. Assert Arabic lang at the document root, mark intentional foreign-language runs with their language, and keep DOM order aligned with the task. R080 reports intentional foreign-language runs that are missing a lang attribute. R081 reports flex row-reverse used to fake RTL instead of document direction, where tab order fights visual order.

Then test with a keyboard and the assistive technology in the supported matrix. Listen to the page title, headings, field instructions, error summary, live status and mixed-language identifiers. Verify that focus moves to a useful place after a failed submission and returns predictably after a dialog closes.

A passing automated accessibility scan is evidence about the rules it evaluated. It is not a transcript of the user's experience. Keep manual assistive-technology review attached to the high-risk journeys rather than adding a generic checkbox to every release.

CI can hide Arabic font failures#

Font loading changes geometry. A test worker that blocks the font host, captures too early or uses an image without the production fonts may approve different line breaks from the deployed product.

Wait for fonts before measuring or capturing:

// page and expect are supplied by the browser test runner.
await page.goto("/ar/usage");
await page.evaluate(() => document.fonts.ready);

const heading = page.getByRole("heading", { name: "استهلاك الكهرباء" });
const family = await heading.evaluate(
  element => getComputedStyle(element).fontFamily
);

expect(family).toContain("Noto Kufi Arabic");

This assertion confirms that the computed family list names the expected font. It does not, by itself, prove that the webfont file loaded or supplied the rendered glyphs. Add network evidence or a browser-level font check appropriate to the stack. R050 reports Arabic text falling back past the declared font family, whether measured, declared or caused by a webfont load failure.

Do not solve font nondeterminism by ignoring all text regions in visual comparisons. That removes the geometry most likely to change in Arabic. Make the font asset and loading condition deterministic instead.

Layout assertions often encode LTR assumptions#

Tests can be directionally wrong even when the component is right. An assertion such as “the action's x-coordinate is greater than the label's” bakes an English position into the suite. Reversing the comparison for Arabic is also fragile when the design changes at a responsive breakpoint.

Assert the product rule at the right level. If the requirement is that the action remains inside the card and never covers its notice, test containment and non-overlap. If the requirement is that the action sits at logical end, read the component's computed direction and compare the appropriate edges. If the exact composition matters, use an approved visual baseline.

// card, action and expect are supplied by this component test.
const cardBox = await card.boundingBox();
const actionBox = await action.boundingBox();

expect(cardBox).not.toBeNull();
expect(actionBox).not.toBeNull();
expect(actionBox.x).toBeGreaterThanOrEqual(cardBox.x);
expect(actionBox.x + actionBox.width).toBeLessThanOrEqual(
  cardBox.x + cardBox.width
);

This test is direction-neutral because containment is the real requirement. Add vertical bounds and overlap checks when the component contract requires them. Do not claim that a generic left-versus-right assertion proves RTL correctness.

Physical CSS needs the same audit. A component can pass an English screenshot while left, right, physical margins or a fixed transform carry the wrong assumption into RTL. R030 reports direction-sensitive physical properties without an RTL override. The RTL CSS guide explains where logical properties remove the assumption and where explicit directional variants are still needed.

Coverage reports hide missing dimensions#

Line coverage can reach the branch that renders a service-request error without testing its Arabic copy, direction or geometry. Route coverage can visit every page while opening only the successful state. Screenshot counts can grow while all images use one desktop width.

Track Arabic coverage by risk, not by raw test count. A small matrix is enough to expose gaps:

“No” and “Partial” are useful results. They turn a vague confidence problem into release work. The table should point to fixtures and tests, not become another manually maintained score with no trace to evidence.

Choose representative surfaces by component family and task risk. One dense dashboard can cover table, filter and chart geometry. One form cannot represent every field contract because Arabic text, email, URL and machine identifiers need different direction. Add targeted cases when a release touches a shared component or locale-sensitive dependency.

Add the missing gates in signal order#

Begin with checks that have clear failure conditions and low review cost. They stop broken builds before screenshots and human review.

1. Validate Arabic catalogs, required keys and locale routes during the build.
2. Open Arabic URLs directly and assert live root lang and dir through redirects and navigation.
3. Run unit fixtures for market format, plural selection and locale persistence.
4. Run browser component tests inside a production-like RTL wrapper.
5. Exercise Arabic success and failure states with realistic data at named viewports.
6. Measure document overflow, clipping, overlap, field direction and loaded fonts.
7. Compare approved visual baselines after data, animation and fonts are stable.
8. Send the candidate to Arabic and accessibility reviewers for meaning and interaction.

The order is not a maturity ladder. Each gate answers a different question. It is sequenced so deterministic failures are fixed before expensive review starts.

When a manual review finds a stable regression, add it to the closest automated layer. A clipped notice belongs in a component geometry case. A locale dropped after password recovery belongs in the route journey. An ambiguous Arabic instruction remains a language-review decision even after the approved copy is updated.

Audit the suite from failures backward#

Do not start the audit by counting existing tests. Pick the Arabic failures the product could ship and trace which layer would stop each one.

For every critical journey, ask:

• Can the suite enter the Arabic route without visiting English first?
• Does it prove root language and direction after every navigation boundary?
• Which fixtures contain Arabic expansion and mixed-direction values?
• Which failure, empty, loading and permission states are reachable?
• Does any real-browser test measure overflow, overlap or clipping?
• Are fonts loaded and verified before geometry is captured?
• Who approved the first Arabic visual baseline, and against what decisions?
• Which assertions cover focus order and keyboard recovery?
• Which findings require an Arabic reader or market owner?
• Can a failed result be reproduced from its route, state, fixture and viewport?

Write each uncovered risk into the same backlog as functional test debt. Label the missing observation, not just the tool: “no Arabic server-error fixture for meter transfer” is actionable; “need more RTL tests” is not.

An existing QA suite does not need to be replaced. It needs inputs and assertions that describe the Arabic product rather than assuming the English product is a sufficient proxy.

Make Arabic failures visible to the suite#

The central gap is observability. Unit tests need Arabic market fixtures, browser tests need inherited direction and real geometry, visual tests need reviewed baselines, and manual reviewers need reachable states. Once those inputs are explicit, a passing suite means more than “the English journey still works.”

A Ritla scan can expose rendered Arabic conditions that a behavior-only suite never asserts, giving the team concrete failures to turn into regression tests. Follow that evidence into the guide to testing an Arabic website before launch, and use why translation QA is not enough to assign the remaining product and language judgments.