Skip to content

fix(main): UnicodeDecodeError#3

Merged
unclecode merged 1 commit into
unclecode:mainfrom
QIN2DIM:main
May 23, 2024
Merged

fix(main): UnicodeDecodeError#3
unclecode merged 1 commit into
unclecode:mainfrom
QIN2DIM:main

Conversation

@QIN2DIM

@QIN2DIM QIN2DIM commented May 18, 2024

Copy link
Copy Markdown
Contributor

Origin: zh-CN. Requires utf-8 to render a page

 File "T:\_GitHubProjects\Forks\crawl4ai\main.py", line 70, in read_index
    partials[filename[:-5]] = file.read()

UnicodeDecodeError: 'gbk' codec can't decode byte 0xa4 in position 149: illegal multibyte sequence

as below:

- with open(path, "r") as file:
+ with open(path, "r", encoding="utf8") as file:

I'm troubleshooting the same problem elsewhere.

 File "T:\_GitHubProjects\Forks\crawl4ai\main.py", line 70, in read_index
    partials[filename[:-5]] = file.read()

UnicodeDecodeError: 'gbk' codec can't decode byte 0xa4 in position 149: illegal multibyte sequence
@caramale07

Copy link
Copy Markdown

same issue, same solution

@unclecode unclecode merged commit 7381fa9 into unclecode:main May 23, 2024
@unclecode

Copy link
Copy Markdown
Owner

@QIN2DIM Appreciate the collaboration.

unclecode pushed a commit that referenced this pull request Nov 6, 2024
Merging latest changes from main branch
lassedrud pushed a commit to lassedrud/crawl4ai that referenced this pull request Feb 21, 2025
fix(main): UnicodeDecodeError
quangnx99 pushed a commit to quangnx99/crawl4ai that referenced this pull request Apr 17, 2026
- Remove "browser locale is Vietnamese" warning that anchored LLM to VN
- Move URL locale signal to priority unclecode#3 (after explicit page text + footer)
- Add brand HQ fallback (UNIQLO → JP, Samsung → KR, Zara → ES)
- Currency determination now based on page content only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants