# Crawling Settings

The **Crawling Settings** section gives you precise control over how Zipchat AI processes your website content. By customizing these settings, you ensure the AI focuses on relevant data, improving its responses and creating a better user experience.

<figure><img src="/files/hzzdRkk7vAy25G86icPI" alt=""><figcaption></figcaption></figure>

### Customization Options

**1. Manage Out-of-Stock Products**

* **Include/Exclude Out-of-Stock Products**\
  Choose whether to crawl and index out-of-stock products. Excluding them prevents the AI from suggesting unavailable items, improving customer satisfaction.
* **Out-of-Stock Keyphrase**\
  Identify the keyphrase or HTML element unique to out-of-stock items, like *"Out of Stock"*. Zipchat uses this to accurately skip these pages.

**2. Ignore Irrelevant Page Elements**

* **Add or Remove Elements**\
  Specify parts of your website, like footers, headers, or banners, that the AI should skip when crawling. This ensures it focuses on meaningful content.
* **Use CSS Selectors**\
  Pinpoint elements to ignore using simple CSS syntax:
  * **Tags**: `header` or `footer`
  * **Classes**: `[class*="footer"]`
  * **IDs**: `[id*="promo-banner"]`

**3. Default Ignored Elements**

Zipchat may preconfigure certain elements to be ignored, such as:

* **Footers**: `[id*="footer"]` or `[class*="footer"]`
* **Headers or Navigation Bars**

You can adjust these defaults to better suit your site’s layout.

***

### Why Crawling Settings Matter

1. **Improved Relevance**: Focuses the AI on essential content, ensuring responses are accurate and useful.
2. **Better Shopping Experience**: Avoids suggesting out-of-stock products, reducing customer frustration.
3. **Efficient Performance**: Ignoring unnecessary elements speeds up crawling and enhances AI processing.

***

### Example Use Case

* **Out-of-Stock Products**: Exclude pages with the banner *"Currently Unavailable"*.
* **Ignore Elements**: Skip promotional banners and footers to keep responses clean and focused.

***

### Optimize Your AI for Success

Fine-tuning the **Crawling Settings** ensures that Zipchat AI gathers the most relevant information from your website. By focusing on what matters and ignoring distractions, you create a smarter, more efficient chatbot that delivers exceptional customer support and boosts satisfaction. 🚀


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.zipchat.ai/installation-and-setup/chat-settings/configuration/crawling-settings.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.