Book a demo

Data Scout · Data acquisition

Set up a search once. Get the data on your schedule.

Data Scout collects prices, availability and product details from more than 5,000 sites across 25 verticals and delivers them in the format your systems read. Nobody on your team writes a line of code.

Book a demoRequest sample data
Today’s runs 6 searches · 33 runs
SEARCHOUTPUT 00:00 06:00 12:00 18:00 24:00 Car rental, LAX, SFO, SAN · 14 sites: 18.4k rows. Car rentalLAX, SFO, SAN · 14 sites 18.4k rows Hotels, Orlando, 40 properties · 6 sites: 9.1k rows. HotelsOrlando, 40 properties · 6 sites 9.1k rows Airfare, JFK–LHR, JFK–CDG · 4 sites: 2.2k rows. AirfareJFK–LHR, JFK–CDG · 4 sites 07:00 run retried on a fresh session after a challenge page, delivered 07:14 2.2k rows Power tools, 5,000 SKUs · 5 sites: 61% done. Power tools5,000 SKUs · 5 sites 61% done Tires, 225/65R17 · 6 sites: 3.0k rows. Tires225/65R17 · 6 sites 3.0k rows Ferry, Dover–Calais · 9 sites: 1.1k rows. FerryDover–Calais · 9 sites 1.1k rows now 10:30 DeliveredRunningScheduledRetried and delivered
45,600collection jobs run every day
1.7 millionautomated shopping sessions a day
6.2 billionsearches run in the last 12 months

Protected sites

The sites worth watching are the ones that block crawlers.

Airlines, OTAs, rental brands and big retailers put bot defenses in front of their prices. A typical crawler gets blocked, served a fake price or handed yesterday’s page, often without knowing it. Getting accurate prices through those defenses is our main job, and we have done it for 20 years.

Bot managersPerimeterX (now HUMAN), Akamai Bot Manager, Cloudflare and DataDome score every visitor. Our collection engines are built to work with these defenses, and our team adjusts when a site changes them.
CAPTCHAs and challenge pagesWhen a site throws a CAPTCHA or a press-and-hold check, runs can retry on a fresh session, and our team follows up on anything still missing.
IP blocks and geo-fencingProxies by country help keep each search local, so you see the point-of-sale price your customer sees.
Quality checksSome sites show automated traffic a different price, an old cached page or no inventory at all. Our quality checks compare each run with earlier ones, and our team looks into results that don’t fit the pattern.
Layout changesSites change their pages without notice. Our team updates the scripts when they do, and tells you if a delivery is affected.

Inside Data Scout

Every search, run and result in one workspace.

Data Scout workspace with search inputs, schedule and results

From a question to a file in your system.

You decide what to watch and when. We handle the collection, the clean-up and the delivery.

Charts on every search

Quality and output charts show how competitor results move from run to run, so you see a change before it shows up in your numbers.

Data Scout charts comparing results across runs

Build the search

A guided setup lets you pick sites, markets, dates and products. Or call our API and automate the whole thing.

Data Scout search setup

Find any job in seconds

Star the searches you check most, open their results or change the inputs from one list.

Data Scout job list with starred jobs

Run it when you need it

Start a search now, schedule it for later, or repeat it hourly, daily or weekly.

Data Scout schedule settings

Extract, clean and deliver

Every run produces a clean extract in the layout your systems read, ready to download or send.

Data Scout extract list

Map it to your codes

Bring your own mapping tables. Results arrive in your categories, car classes or SKUs, ready to analyze.

Data Scout data mapping table

Compare every run

Click through past runs to see how results moved, with quality and output charts built in.

Data Scout results embedded in a past run

What comes with it

The data, plus the views your analysts ask for first.

Every Data Scout account includes standard views of the results. If you want more, our analysts build them with you.

  • Competitor price changes, run over run
  • Win, meet and loss against each competitor
  • Trend and calendar views by date
  • Results stored in a data warehouse you can query
  • Custom dashboards built by our team
  • Data science and analytics support
  • Regional store assortment, without sending people into stores
  • Files by SFTP, or feed your models and agents by API

Tell us what to collect.

Describe the sites, markets and dates you need. We will scope the search and send a sample run.

Request sample dataBook a demo