Enhancing DocSearch FAQ (#388)

* enhancing DocSearch FAQ

* fixing issues

* fixing issues

Author: Sylvain Pace

docs/source/documentation/2-docsearch-scraper/2-config-options.html.md.erb

@@ -205,12 +205,18 @@ as `lvl1` and `h3` as `lvl2`. `text` is usually any `p` of text.
 
 We recommend making use of at least the three first levels for better relevancy.
 
-### Global selectors _Optional_
+### `global` selectors _Optional_
 
 It's possible to make a selector global which means that all records from the page will have
 this value. This is useful when you have a title that is in the right sidebar and
 the sidebar is placed after the content in the DOM.
 
+`global` attributes should be seen as a way to extract the matching elements from the HTML flow. These global elements will not be considered as breaking ones when we encounter them along the flow. If this token is enabled :
+- we will not create a new record from the current builded stack.
+- we will apply its contextual value (or `default_value` if not matched) to every records
+
+We mostly use this parameter for page that miss context. It enables us to pick up the context from another common part without having duplicates.
+
 ```json
 "selectors": {
   "lvl0": {

docs/source/documentation/4-docsearch-FAQ/1-customize-configuration-file.html.md.erb

@@ -1,8 +1,8 @@
 ---
-title: Customize my configuration file
+title: Customize my DocSearch
 ---
 
-## What content do we recommend indexing?
+## What content do we recommend to index?
 
 ### Code blocks
 
@@ -14,7 +14,7 @@ A good practice would be to emphasize the meaningful underlying part of it
 thanks to a dedicated class, which you will want to add to the `text` selector
 in your configuration file.
 
-For example, in this code snippet, you can exclude the code except the parameter of the function:
+For example, in this code snippet, you can exclude the code but the parameters of the function:
 ```
 <code>
 function(<toIndex>something useful to search for</toIndex>)
@@ -24,7 +24,7 @@ function(<toIndex>something useful to search for</toIndex>)
 
 ### Table of contents
 
-The elements of the table of contents only *target* but do not provide actual content.
+The elements of the table of contents only *target* but do not provide content.
 They are merely a step on the way to reaching relevant content. As a result, they are
 almost always in a different place from the payload. This is why we consider the
 table of contents an obstacle for DocSearch to find the coveted information.
@@ -76,4 +76,4 @@ You can override the way these elements are impacting the search thanks to `cust
     "desc(weight.level)"
   ]
 }
- ```
+ ```

docs/source/documentation/4-docsearch-FAQ/2-global-questions.html.md.erb

@@ -0,0 +1,43 @@
+---
+title: Global wonderings
+---
+
+## What is the general behaviour?
+
+### How often do you scrap my website?
+
+We scrap your website, at least, every 24 hours since documentation is not meant to change frequently.
+Furthermore we update it after every pull request on your configuration. This is the best way to request us to trigger a new crawl. Please leverage your configuration file to do so.
+
+###  I want to add search to my whole website?
+DocSearch is suited for documentation content. If you want to widen the scope of your search to not documentation content. You have two solutions:
+- [Run it on your own](https://community.algolia.com/docsearch/documentation/docsearch-scraper/overview/)
+- Build your own search-UI [thanks to InstanteSearch](https://community.algolia.com/instantsearch.js/) ([example here](https://jsfiddle.net/s_pace/965a4w3o/))
+
+### Does the crawl encompass several domains/sub-domains?
+
+The `start_urls` define the allowed domain for our crawler. Basically, we take the main domain of every URLs. We will not go outside this whitelisted domain list. If you want to encompass a wider domain, please include it as a new start_url.
+
+### Where is hosted the data?
+
+As every Algolia index, everything is stored on our servers with the security and privacy required. You can [find more details in the global documentation pages.](https://www.algolia.com/doc/guides/infrastructure/servers/)
+
+## Feature
+
+### Will it become paid?
+
+Open source is great and we want to support as much as we can. Since these projects mostly have limited resources, every granted project will ever remain free.
+If you want to upgrade your search experience, you can do it with your own Algolia implementation and [apply for a free community plan](https://www.algolia.com/pricing). DocSearch will not be used anymore but still happy to help.
+
+### What data are you collecting, who can get access to it?
+
+We only scrap publicly available data according to your custom selectors (see the how it works part). DocSearch introduces [the algolia analytics](https://www.algolia.com/doc/guides/insights-and-analytics/analytics-overview/) for your DocSearch indices.
+You can ask access to these data by using the private email thread and send us the email addresses of the people to grant access.
+
+### Where can I see the analytics?
+
+Once you are granted (see above), analytics will be available in the Algolia dashboard like any regular application. You will need to select the analytics tabs.
+
+## I am not the owner, what can i do?
+
+You can definitely help improving the search experience of your favourite documentation website by opening issues on the repo. You can also reference the GitHub handle @s-pace if you want us to chime in / provide you a quick demo. You can advocate the owner of the repo by letting them request DocSearch. We would be happy to support you in such regards. Feel free to ping.