Running at scale

Collect Lighthouse data for hundreds or thousands of URLs using the PSI API, headless Chrome, or managed services.

Collecting Lighthouse data for many URLs requires planning around hardware, parallelism, and result stability. Before scaling up, read Dealing with score variability — the same variability concerns that affect individual runs are amplified at scale.

Options for running at scale

Option 1: PageSpeed Insights API

The PageSpeed Insights API runs Lighthouse on Google infrastructure and returns results via a simple HTTP request. The default quota is 25,000 requests per day.

Best for: publicly accessible URLs where you do not need to maintain your own test environment.

curl "https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=https://example.com&key=YOUR_API_KEY"

The PSI API offers good reproducibility because it runs in a controlled, stable environment. URLs must be publicly reachable — localhost or firewalled URLs are not supported without an external tunneling solution.

Option 2: Lighthouse CLI on cloud hardware

Running the CLI on your own machines gives you full control over configuration. Most teams wrap the CLI with a bash, Python, or Node script that iterates over a list of URLs.

Popular wrappers:

• lighthouse-batch — run Lighthouse over a list of URLs and produce a summary
• multihouse — parallel Lighthouse runs over multiple URLs

Best for: private URLs, custom configurations, or workflows that need programmatic access to raw results.

Option 3: Managed services

Commercial services handle infrastructure, scheduling, and result storage for you. See the CI Integration page for a list of third-party integrations including WebPageTest, Calibre, DebugBear, and Treo.

Gather mode and audit mode

Lighthouse supports splitting a run into two separate phases using the -G (gather) and -A (audit) flags. This is useful at scale because it decouples browser time (expensive, requires Chrome) from audit time (cheap, CPU-only).

# Phase 1: launch Chrome, collect page artifacts, save to disk
lighthouse https://example.com -G ./artifacts/example

# Phase 2: load saved artifacts from disk, run audits, generate report
lighthouse https://example.com -A ./artifacts/example

Use -GA to run both phases in a single invocation while still saving artifacts to disk for later re-auditing:

lighthouse https://example.com -GA ./artifacts/example

Separating the phases lets you:

• Re-run audits against already-collected artifacts without re-launching Chrome
• Distribute gathering across many machines and consolidate auditing on one
• Iterate on audit logic without incurring browser startup cost

Launching headless Chrome

In CI and server environments, Chrome must run in headless mode. Pass the following flags via --chrome-flags:

lighthouse https://example.com \
  --chrome-flags="--headless --disable-gpu --no-sandbox"

Flag	Purpose
--headless	Runs Chrome without a visible window
--disable-gpu	Disables GPU hardware acceleration (required in many server environments)
--no-sandbox	Disables the Chrome sandbox (required in Docker and some Linux environments)

--no-sandbox reduces Chrome’s isolation. Only use it in environments you control, such as a dedicated CI machine or container. Review the sandbox tradeoffs before enabling this flag.

Launching Chrome programmatically with chrome-launcher

For Node.js scripts that need direct control over Chrome, use the chrome-launcher package that ships with Lighthouse.

npm install chrome-launcher lighthouse

import chromeLauncher from 'chrome-launcher';
import lighthouse from 'lighthouse';
import fs from 'fs';

async function runLighthouse(url) {
  const chrome = await chromeLauncher.launch({
    chromeFlags: ['--headless', '--disable-gpu', '--no-sandbox'],
  });

  const options = {
    logLevel: 'error',
    output: 'json',
    onlyCategories: ['performance'],
    port: chrome.port,
  };

  const result = await lighthouse(url, options);

  await chrome.kill();

  return result.lhr;
}

// Process a list of URLs sequentially.
const urls = [
  'https://example.com',
  'https://example.com/about',
  'https://example.com/contact',
];

for (const url of urls) {
  const report = await runLighthouse(url);
  const score = report.categories.performance.score * 100;
  console.log(`${url}: ${score}`);

  fs.writeFileSync(
    `./reports/${encodeURIComponent(url)}.json`,
    JSON.stringify(report, null, 2)
  );
}

Key points in this example:

• Chrome is launched and killed for each URL. Reusing a Chrome instance across many runs allows state to accumulate in the profile and degrades result reliability above roughly 100 loads.
• onlyCategories: ['performance'] skips audits you do not need, which reduces run time.
• Results are written to individual JSON files for later processing.

Memory and stability tips

Running Lighthouse repeatedly puts sustained pressure on Chrome and the machine. A few practices reduce crashes and noise:

• Restart Chrome between runs. Reusing a single Chrome instance is convenient but accumulates memory and profile state. For reliable results, launch a fresh Chrome for every URL.
• Limit parallelism. Running more Lighthouse instances simultaneously is not always faster — beyond a certain point, resource contention degrades results and causes crashes.
• Use dedicated hardware. Burstable or shared-core instance types (AWS t series, GCP shared-core N1/E2) produce inconsistent timing results. Use fixed-performance instances: AWS m5.large, GCP n2-standard-2, or Azure D2.
• Do not run Lighthouse alongside other CPU-intensive workloads. Anti-virus scans, builds, or other test suites running concurrently skew performance metrics.

Running many Lighthouse instances in parallel on a single machine will cause instability and unreliable performance scores. Scale horizontally across machines rather than vertically on one machine. A single n2-standard-2 instance running one Lighthouse run at a time produces more reliable results than an n2-standard-8 running eight in parallel.

Choosing instance specs

The minimum recommended specs for a single concurrent Lighthouse run:

• 2 dedicated CPU cores (4 recommended)
• 2 GB RAM (4–8 GB recommended)
• Avoid function-as-a-service infrastructure (AWS Lambda, Google Cloud Functions) — cold starts and CPU throttling produce highly variable results

At approximately $0.10/hour and ~30 seconds per test, each Lighthouse report costs roughly$0.0008 on a suitable instance.