Collecting data from other websites

Let your Base44 app visit other websites and collect information from them, like prices or event details, even without an API. Set it all up in the AI chat.

Your app can visit other websites and collect information from them, like prices, listings or event details, even when the site has no API. It does this by opening a real web browser in the cloud, called browser automation, and working with the page the way a person would. You can also use it to fill in forms on other sites or check that your published app works. You set it up by describing what you want in the AI chat, and the AI writes a backend function that runs the browser for you.

Availability:

• Browser automation is rolling out gradually and may not be available on your account yet. If the AI chat doesn’t open a browser when you ask, it hasn’t reached your account.
• You need the Builder plan or higher to use it, because it runs in your app’s backend functions.

What you can do with it

• Collect listings: Gather the name, date and venue of upcoming events from an events website.
• Compare prices: Check flight or product prices on other sites and save the best options.
• See what’s trending: Pull the top videos under a hashtag or the top stories on a news site.
• Watch for changes: Check a page every morning and get an email when a price or listing changes.
• Fill in forms: Request a quote on a supplier’s website with the details from an order.
• Check your live app: Confirm that sign-up works or that a tracking pixel fires on your custom domain.

---

How it works

Each time your backend function runs, it opens a fresh browser session, does its job, and closes the browser. The AI uses AI-powered steps to find things on the page, click, type, and pull out the information you asked for, so you describe what you want in plain words rather than pointing at parts of the page.

Your app can start a session in 2 ways:

• From your app: Someone clicks a button or opens a page that calls the backend function.
• On a schedule: A workflow runs the backend function at the times you choose, for example every morning.

A browser is the right tool when a website only shows its content after the page runs, or when getting the information takes several steps. If a connector, an API, or a simple page request can get the same data, the AI uses that instead, because it’s faster and costs less.

---

Asking the AI chat

Describe the website, what to do there, and what to do with the result. The AI writes the backend function, runs a test with a real browser, and shows you what came back.

To add browser automation to your app:

1. Open your app in the app editor.
2. Describe the task in the AI chat, including the website address, the information you need, and where to save or show it.
3. Check the result of the AI’s test run, and ask the AI chat to adjust the function if anything is missing or wrong.

Each test the AI runs opens a real browser session, so it uses credits like any other session.

---

Example prompts

Use these prompts as a starting point, and replace the website and details with your own.

To collect data from a website:

Every time I click "Refresh events" on the Events page, open https://example.com/events in a browser, collect each event's name, date, venue and ticket link, and save them to the Event table. Skip events that are already saved.

To fill in a form on another website:

Create a backend function that opens https://example.com/quote, fills in the quote form with the product, quantity and ZIP code from the order, submits it, and saves the price shown on the next page to the order.

To run it on a schedule:

Create a workflow that runs every day at 8:00 AM. It should open https://example.com/pricing in a browser, read the price of each plan, and email me if any price changed since yesterday.

---

Testing your published app

You can use browser automation to check how your published app behaves on its live address, for example on your custom domain. This is useful for problems that only show up on the live site, such as a sign-up email that doesn’t arrive, a tracking pixel that doesn’t fire on your custom domain, a checkout that asks for card verification, or something that only breaks on a phone.

To check the version you’re building in the app editor, use the testing agent instead. It’s made for testing your preview.

For example:

Create a backend function that opens my published app, signs up with a new test account, and confirms that the welcome page loads. Take a screenshot of each step so I can see what happened.

---

Credits and limits

Browser automation uses credits from your app’s workspace each time a session runs, whoever starts it. The cost is made up of:

• Browser time: Charged by the minute the browser is open, rounded up, with a 1-minute minimum.
• AI steps: Each time the AI finds, clicks, or reads something on the page, it uses credits like other AI calls.

Most of the cost usually comes from the AI steps, not the browser time. Learn more about how credits work.

Session limits:

• Session length: A session closes after 5 minutes by default. You can ask the AI chat to allow a session to run for anywhere from 1 to 15 minutes.
• Credits: A session doesn’t start if your workspace is out of credits.
• Busy times: If no browser is free at that moment, the session doesn’t start and your app shows an error. Try again a little later.

By default, the AI makes the backend function check that the person who calls it is signed in to your app, so people you don’t know can’t run sessions on your credits.

---

Using websites responsibly

Before you automate a website, check its terms of use and make sure they allow automated access. Some websites don’t.

Keep these points in mind:

• Blocked access: Some websites block automated browsers or show a CAPTCHA. Browser automation doesn’t get around these, so choose sites that allow automated visits, or use the site’s API or a connector if it has one.
• Signing in: Every session starts signed out, with nothing saved from earlier sessions. If a site needs you to sign in, store the login details as an app secret rather than in your prompt, and only use accounts you’re allowed to use this way.
• Recordings: Sessions keep a recording and logs by default to help fix problems. If you don’t want them, ask the AI chat to turn them off.
• Changing websites: When a website changes its layout, a function that worked before can return less data. Ask the AI chat to check the function again.

---

Troubleshooting

If your browser automation isn’t working as expected, find your situation below.

The function returns no data or the wrong data

The page may have changed, loaded slowly, or shown something the browser didn’t expect, like a cookie banner or a sign-in page. Ask the AI chat to look into it. The AI can take screenshots during a test run to see what the browser saw.

My [function name] function returned no data. Run it again with screenshots at each step, find out what the browser saw, and fix it.

The session ends before the task finishes

Long tasks can reach the session’s time limit. Ask the AI chat to give the session more time, up to 15 minutes, or to split the task into smaller runs.

I get an error that no browser is available

All browsers were busy at that moment. Wait a little and run it again. If it happens on a schedule, move the workflow to a different time.

I get an error about credits

Your workspace doesn’t have enough credits to start a session. Check your credit balance and top up or upgrade your plan.

The website blocks the browser or shows a CAPTCHA

Browser automation can’t get past blocks or CAPTCHAs. Look for an API, a connector, or another source for the same information.