feat(search) by superbiche · Pull Request #700 · COVID19Tracking/website

superbiche · 2020-04-23T21:14:26Z

Description

Draft PR to add Algolia search (#693)

Original todo from @kevee, please add items here so we don't get lost :) :

Ordering by index type, i.e. show states first, then blog posts, then pages - if I type "California" I probably want to jump to California's page
A search results page at /search that supports queries - i.e. /search?q=stuff
A search box that goes on the header. Do not worry about design right now, but just make sure it works.
A search autocomplete - we already use Reach's combobox component on the site for the state dropdown, and I like it because of all all the accessibility features in it, but I am not wedded to that component
Environment-aware Algolia index (https://covid-tracking.slack.com/archives/C012DPFRP7D/p1588079930004600)
Target the feature/header-refresh branch

Related Issues

#693

…n query string feat(search): add some styles & links for states/blogPosts results

muamichali · 2020-04-24T16:50:36Z

I think it could be useful to scrape our own tweets and index them as well, since a lot of useful information/thoughts/explanations happen on our twitter feed.

superbiche · 2020-04-25T12:39:24Z

@muamichali yup that's something we can work on, if someone's willing to create the app and send us the API keys on Slack, we'd be happy to give it a shot when we're done with the basic features!

… sections

The search feature was started as a PoC using State hooks, that was extracted to src/context/search-context, using Context hooks for easier reuse (on /search but also in the header). Added withSearch HOC, which wraps withLocation, to re-use searchContext easily accross the site while retaining access to URL query. All logic specific to search, but not to search page, moved to the hook file. Pages that match have their (full, for now) body rendered with some highlighting (deserves some more emphasis to make it more visible). If the body has no hightlighting we return its truncated value. Finally - more like a side effect, but a nice one - the search is triggered whenever the input value is updated. We need to check: 1. Is this going to consume our API quota too fast, and 2. Will this be, OK as the header will be an autocomplete, and this input refreshes the result page instantly? The UX feels right but maybe I'm just happy that it works.

netlify · 2020-04-26T13:20:37Z

This is a preview version of the site.

Built with commit 7eabe66

https://deploy-preview-700--upbeat-lovelace-3e9fff.netlify.app

…esults

…y blocks

schwartzadev

This is just a first pass from visually inspecting the code and chatting with @superbiche over Slack.

I didn't get a chance to run this locally to review, but I'm posting this in the interim regardless

schwartzadev · 2020-04-27T01:12:27Z

src/pages/search.js

+        </div>
+      )}
+      <div className={searchPageStyle.searchResults}>
+        {/* State results */}


These could probably be done DRY-er -- we could have a component for the results and then pass the inner content to the component for:

state results

blog posts

pages

Plus, this should make it easier to add items like Twitter in the future!

Totally agree. Once we've done the first review pass this ought to be a flexible component!

schwartzadev · 2020-04-27T01:13:33Z

src/pages/search.js

+      {searchState.isFetching && (
+        <div>
+          <img
+            src="https://d1j8pt39hxlh3d.cloudfront.net/products/previews/RES3POBSZ353HFVPZOKR/2329_kKbUTLGEVXhuOIJqrT7QvIJFAXvEpQ3z.gif"


This is a placeholder for now -- just flagging it before we push

src/utilities/algolia.js

schwartzadev · 2020-04-27T01:25:35Z

src/pages/search.js

+              <div key={page.objectID} className={searchPageStyle.searchResult}>
+                <PublicationTitle>
+                  {/* FIXME this should be handled during indexing 
+                  (avoids external Link issues) */}


This is tricky...it looks like some Links begin with '/', and others don't. We should probably standardize this across the codebase so we don't have to handle this here. Ideally someone could work some grep magic and find out where that's happening.

@kevee which is preferred?

prefixing Link with /

not prefixing Link with /

Actually we have to prefix links with / otherwise Gatsby will throw warnings.
I'm pretty confident this is an inconsistency issue with the slugs and how these are built.

I could comment out this function to get a precise list of failing links, so we can check these individually

Prefix them all with /

The pages here all have their slug defined in Contentful, which automatically creates the slug based on the title.

@kevee Added getSanitizedSlug to generate slug depending on result type + handle missing starting /

kevee · 2020-04-27T05:11:35Z

This is going to be a feature in a larger project, so let's not merge it into master. I'll make a project branch for it to merge into tomorrow.

gatsby-config.js

kevee · 2020-04-27T11:11:01Z

I've also made a separate admin key for both Netlify and CircleCI scoped to just index management.

superbiche · 2020-04-27T14:20:04Z

Thanks for your feedback @kevee @schwartzadev. Fixes and adjustments coming later today!

jasonlcrane · 2020-04-27T16:19:37Z

I added the autocomplete, using the Reach UI Combobox. I kept the form that Michel has been using on the page as well, for discussion's sake. I am not sure what the best approach is, but one thing I was thinking about was the possibility of ditching this autocomplete, and having the search results with snippets that Michel has displaying on the page go full screen on search (so essentially still autocomplete). There are some potential accessibility concerns there I'm sure that we would need to work around.

superbiche

Did a quick review, some issues to fix but nothing shocking 👍

src/context/search-context.js

superbiche · 2020-04-27T16:40:36Z

src/pages/search.js


-  let searchEvent
+  const getSlug = item => {
+    if (!item.publishDate) {


I think it's risky to rely on item.publishDate, imo adding a type argument to the arrow function would be safer

Yeah, you're right, that's brittle. The type is not known when getSlug is called, though. We could refactor it to loop over those hits arrays separately and set the type there, I think. That could get a little messy as well, but I'll see if I can come up with something that will work.

Imo looping over the results while retaining the type is the way to go, added searchResultTypes const to make it easier and gonna make result keys use keys from this constant so it should be more robust

Used your new keys and getSanitizedSlug function to improve this.

superbiche · 2020-04-27T16:43:42Z

src/pages/search.js

+    return `/blog/${item.slug}`
+  }
+
+  const goToResult = selectedItem => {


It amazes me Combobox doesn't allow choosing what we pass. @kevee do you know if there's a way to pass the item.objectID instead?

superbiche · 2020-04-27T16:46:52Z

src/pages/search.js

+    })
+
+    if (item && typeof window !== 'undefined') {
+      window.location = getSlug(item)


@jasonlcrane check out withLocation HOC, it passes location and navigate if I remember well.
You could add navigate to the component's props export default withSearch(({ search, navigate }) => { so you can then use it to go to the destination while retaining history.
(https://reach.tech/router/api/navigate)

Thanks, good call. Updated.

src/pages/search.js

schwartzadev · 2020-04-27T18:48:32Z

I don't think this is expected behavior...How does the autosearch prioritize results? I would expect the Data xyz pages to be the first results, not Wyoming, etc.

Design-wise, we should have a more clear indicator of what is a query and what is content (the query covid looks the same as deprecated, for example)

We should have spacing between the center bullet and the names/dates -- see the homepage for reference

Is there a reason for returning the whole page content under Pages? I would imagine something like the following would be more helpful:

Pages

Data API:

...text surrounding the query from the user...
...a query is provided by the user via the nav...

Another Page:

...only one match for query.

It looks like the inline rendering is not working properly:

The sentence after Javascript should be on a new line, the text below it should not be indented.

% seems to be a wildcard (I'm getting all the states back, two pages, and both blog posts). If that's true, do we want to include that in production?

The accessibility statement renders as: We welcome your feedback on the accessibility of COVID Tracking Project Website. Please let us -- it looks like the end of the content is cut off. Maybe this has to do with the chunking processes?

…pes constant

…gle constant

Kevin Miller and others added 3 commits April 23, 2020 22:41

chore: Added algolia sample

4d293ac

chore: Fixed object IDs, added pages

76dc4ae

feat(search): cleanup, enable env-based indexes

6c1fcd9

superbiche added enhancement New feature or request DEV labels Apr 23, 2020

superbiche self-assigned this Apr 23, 2020

superbiche changed the title ~~Feature/search~~ feat(search) Apr 23, 2020

WIP: add withLocation HOC, add search behavior to get results based o…

c7e663c

…n query string feat(search): add some styles & links for states/blogPosts results

Michel Tomas added 5 commits April 25, 2020 18:31

chore: add babel bits to make Gatsby transpile to ES6 in Node 'as is'…

0474f77

… sections

feat(search): add page results with highlighted result

099e89b

Merge branch 'master' into feature/search

c609a52

feat(search): chunk pages & blog posts prior to indexation

e5af627

Michel Tomas added 6 commits April 26, 2020 16:24

Merge branch 'master' into feature/search

c58a235

feat(search): add basic separator styles, implement snippeting page r…

86367ff

…esults

refactor(search): don't render the whole result section to avoid empt…

86f3bf9

…y blocks

feat(search): add a really cute loader

5c27d84

fix(search): don't show previous results when fetching

2a93296

fix(search): wrong helper name

5c4b8e3

superbiche requested review from kevee, muamichali and schwartzadev April 27, 2020 01:26

schwartzadev reviewed Apr 27, 2020

View reviewed changes

docs(search): add docblock for esoteric functions

a0381d2

chore: update env var for admin key

ca52c1a

kevee requested changes Apr 27, 2020

View reviewed changes

gatsby-config.js Outdated Show resolved Hide resolved

gatsby-config.js Outdated Show resolved Hide resolved

feat(search): Add autocomplete search

264456d

superbiche commented Apr 27, 2020

View reviewed changes

fix(search): regression on className

b25c9d9

Michel Tomas and others added 5 commits April 28, 2020 03:37

feat(search): extract slug / handling to function, add searchResultTy…

2a14d3a

…pes constant

feat(search): enable Algolia plugin only in build/CI/prod environments

ecdd348

feat(search): make everything search index related derived from a sin…

035b834

…gle constant

feat(search): derive component children from searchResultTypes

8eceedc

feat(search): Simplify navigation from autocomplete results

7eabe66

superbiche changed the base branch from master to feature/header-refresh April 28, 2020 20:22

kevee marked this pull request as ready for review April 29, 2020 15:48

kevee merged commit f7655ae into feature/header-refresh Apr 29, 2020

delete-merged-branch bot deleted the feature/search branch April 29, 2020 16:37

Conversation

superbiche commented Apr 23, 2020 • edited by kevee Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Uh oh!

muamichali commented Apr 24, 2020

Uh oh!

superbiche commented Apr 25, 2020

Uh oh!

netlify bot commented Apr 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

schwartzadev left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

superbiche Apr 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kevee commented Apr 27, 2020

Uh oh!

Uh oh!

Uh oh!

kevee commented Apr 27, 2020

Uh oh!

superbiche commented Apr 27, 2020

Uh oh!

jasonlcrane commented Apr 27, 2020

Uh oh!

superbiche left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

superbiche Apr 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

schwartzadev commented Apr 27, 2020

Pages

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

superbiche commented Apr 23, 2020 •

edited by kevee

Loading

netlify bot commented Apr 26, 2020 •

edited

Loading

superbiche Apr 28, 2020 •

edited

Loading

superbiche Apr 27, 2020 •

edited

Loading