[toongod] add support by nthduy · Pull Request #8963 · mikf/gallery-dl

nthduy · 2026-01-30T23:51:40Z

Add support for https://www.toongod.org/ webtoon site.

Implements chapter and webtoon extractors with Cloudflare bypass support:

Features:

Chapter extractor: extracts all images from chapter pages
Webtoon extractor: lists all chapters from webtoon series pages
FlareSolverr integration with session management for efficient Cloudflare bypass
Browser cookie fallback support

Cloudflare Protection:
Site uses Cloudflare protection. Two bypass methods supported:

FlareSolverr (recommended): Automatic challenge solving with session reuse
- Configure: {"extractor": {"toongod": {"flaresolverr-url": "http://localhost:8191/v1"}}}
- Performance: ~0.5s per request (sessions reuse cookies after first challenge)
Browser cookies: Manual cookie export
- Use: gallery-dl --cookies cookies.txt <url>

mikf#6582 (comment)

add 'post' & 'user' extractors

* [pornpics] add category and listing extractors Add support for: - Category pages like /ass/, /milf/, /blonde/ etc. - Listing pages like /popular/, /recent/, /rating/, /likes/, /views/, /comments/ Category pages use JSON pagination like tags/search. Listing pages don't support JSON pagination and use different HTML structure. * [pornpics] simplify category pattern via class ordering - Move PornpicsCategoryExtractor after PornpicsListingExtractor so it acts as catch-all, eliminating need for negative lookahead - Use list comprehension in PornpicsListingExtractor.galleries() * update docs/supportedsites

* [fitnakedgirls] add extractor Add support for fitnakedgirls.com: - Photo galleries (/photos/gallery/) - Category pages (/photos/gallery/category/) - Tag pages (/photos/tag/) - Video posts (/videos/) - Blog posts (/fitblog/) Handles both newer (wp-block-image) and older (size-large) templates. * simplify & fix - use '_extract_title' method - move '_pagination' into base class - update 'FitnakedgirlsTagExtractor' pattern * update docs/supportedsites --------- Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>

Add support for nudostar.com forum (XenForo-based forum site). This is separate from the existing nudostar.py which handles nudostar.tv. Supports: - Thread extraction with pagination - Individual post extraction - Authentication via xf_user cookie or username/password - Internal attachments (both linked and embedded images) - External image host URLs (queued for recursive processing)

- fix website_token extraction - send website_token as 'X-Website-Token' header

support - https://simpcity.cr/ - https://nudostar.com/forum/ (mikf#8333)

mikf#6582 (comment)

- intercept ytdl logging messages and signal error when it emits an error message - remove "ERROR:" etc from ytdl logging messages

support - https://sturdychan.help/ - https://schan.help/ (mikf#8680)

…#8698)

fixes regression introduced in c8fc790

fixes regression introduced in 402f536

9a10203

for example '?order=asc&group=j0fsj3oem3&tlang=en'

fix '400 Bad Request' errors when retrieving more than the first batch of posts.

requested on Discord https://discord.com/channels/SERVER_ID/search?from=USER_ID

…hydration data (mikf#8848)

* Make sure that `img_id`, `audio_id` and `cover_id` fields are always available. The values are set '' where they are not applicable. Having `img_id` is necessary for the default `archive_fmt`, the other fields are handled for consistency. * Allow downloading more than one cover. The previous behavior is kept as-is, but setting the "covers" option to "all" now grabs all available covers. * Add support for downloading subtitles Allows filtering subtitles by source type (ASR, MT) and language. * Ensure archive uniqueness for covers and subtitles. * Update the URL test pattern to include the `image` extension. Although Tiktok may serve the covers with jpeg content, the file ending can be `.image`. The test before 0c14b16 failed because the asserted URL did not match all cover types, but the now used pattern needs the mentioned file ending. * Add support for "creator_caption" subtitles in "LC" format. These subtitles have the keys "Format" set to "creator_caption" and "Source" to "LC". * Add "LC" (Local Captions) as a subtitle source type in the documentation * Code deduplication and renaming subtitle metadata Changed the item type from singular `subtitle` to `subtitles`. Removed the wrong descriptor `cover` from the subtitles fallback title. * Refactor subtitle filtering The filter is now prepared in `_init` to prevent parsing the same config parameter for every item. The `_extract_subtitles` function will still extract if either filter (source or language) matches. * Generate a `file_id` for subtitles Subtitles have multiple fields that determine the unique file, so these are simply concatenated. This is similar to the cover types, only with more variations. * Added tests for subtitles * fix docs entries * fix '"covers": "all"' * simplify some code * Fix fallback title for subtitles Added the missing "f" to the f-string and added "subtitle" to the title. The resulting title will look like "TikTok video subtitle #1234567"

Add extractor for toongod.org webtoon site with Cloudflare bypass support using FlareSolverr proxy.

- Fix line length issues (max 79 chars) - Fix continuation line indentation

Fix folder naming issue where series names included junk suffixes like "Manhwa Afahbb" by extracting titles from breadcrumb navigation instead of URL slugs or H1 tags. - Extract series name from breadcrumb links (always clean) - Fallback to H1 tag with cleaning if breadcrumb fails - Remove "Manhwa", "Webtoon", "Manhua" suffixes - Remove encoded ID patterns (e.g., "Afahbb", "Aeaabb") Before: "Perfect Half Manhwa Afahbb" After: "Perfect Half"

nthduy · 2026-02-01T00:30:24Z

Pushed a new commit to handle an edge case I discovered.

Problem: Some manhwa on ToonGod have strange slugs that break the original H1/slug-based extraction. For example:

https://www.toongod.org/webtoon/perfect-half/ → folders named Perfect Half Manhwa Afahbb
https://www.toongod.org/webtoon/level-1-player-manhwa-aeaabb → folders named Level 1 Player Aeaabb

The issue is ToonGod's chapter URLs contain these suffixes (/webtoon/perfect-half-manhwa-afahbb/chapter-1/), and the chapter extractor was converting the slug to title case as a fallback.

Solution: I changed the approach to extract from breadcrumb navigation since I noticed it's always clean and consistent. Falls back to H1 tag cleaning if breadcrumb fails.

Tested on multiple series:

https://www.toongod.org/webtoon/perfect-half/ → "Perfect Half" ✓
https://www.toongod.org/webtoon/level-1-player-manhwa-aeaabb → "Level 1 Player" ✓
https://www.toongod.org/webtoon/heavenly-demon-cultivation-simulation/ → "Heavenly Demon Cultivation Simulation" ✓
https://www.toongod.org/webtoon/prison-revenge/ → "Prison Revenge" ✓

All tests pass, flake8 clean.

Thanks for your review!

Dragonatorul · 2026-03-15T22:23:29Z

Toongod also uses wpmadara underneath. The base class in #9246 would cover this site too.

Dragonatorul · 2026-03-15T23:10:42Z

Adds Cloudflare solver support

mikf and others added 30 commits December 9, 2025 18:55

[docs/README] add Docker instructions to pull 'dev' images (mikf#6582)

dc16ee4

mikf#6582 (comment)

[chevereto:album] extract 'count' & 'num' metadata (mikf#8604)

8e5d8d8

[redbust] remove module (mikf#6582)

d5f4a3f

mikf#6582 (comment)

[dl:ytdl] forward '_ytdl_manifest_headers' to formats

c2c00d1

[fikfap] add support (mikf#8673)

f5fafd7

add 'post' & 'user' extractors

[facebook] do not match '/permalink' URLs (mikf#8679)

d85fa5f

[gofile] fix extraction (mikf#8681 mikf#8683)

03b45df

- fix website_token extraction - send website_token as 'X-Website-Token' header

[reddit] guess 'mp4' extension for ytdl downloads (mikf#8684)

8140850

[xenforo] implement generic XenForo forum extractors

ab2c03b

support - https://simpcity.cr/ - https://nudostar.com/forum/ (mikf#8333)

[audiochan] extract 'description' texts (mikf#6582)

739e940

mikf#6582 (comment)

[audiochan] relax 'pattern'

05817c5

[dl:ytdl] improve error detection

b891c03

- intercept ytdl logging messages and signal error when it emits an error message - remove "ERROR:" etc from ytdl logging messages

[xenforo] emit AuthRequired errors for 403 downloads

4557ab5

[docs] update 'xenforo' options

db90fe9

release version 1.31.0

d36a441

[audiochan] use proper variable name

0907ba1

[ytdl] respect '--no-skip'

db8dd52

[pixiv] warn about invalid 'PHPSESSID' cookie (mikf#8689)

a8ca947

[2chen] implement generic 2chen board extractors

8f621b3

support - https://sturdychan.help/ - https://schan.help/ (mikf#8680)

[misskey] implement 'order-posts' option (mikf#8516)

a53cc87

[twitter] fix avatar & background downloads with "expand": true (mikf…

85b7f63

…#8698)

[comedywildlifephoto] add 'gallery' extractor (mikf#8690)

774cb1e

[docs] remove 'twitter.username-alt' entries

2980e8f

[boosty] warn about expired 'auth' cookie tokens (mikf#8704)

468570a

[koofer] add 'shared' extractor (mikf#8700)

4f33751

[mastodon] fix "AttributeError: 'parse_datetime_iso'" (mikf#8709)

d497523

fixes regression introduced in c8fc790

[dl:ytdl] fix "UnboundLocalError: 'tries'" (mikf#8707)

207f613

fixes regression introduced in 402f536

mikf and others added 11 commits January 28, 2026 19:37

[exhentai] implement Multi-Page Viewer support (mikf#2616 mikf#5268)

feef91b

[weebdex] make metadata extraction non-fatal no2 (mikf#8954)

a3f164a

9a10203

[weebdex] add 'lang' option, support query params (mikf#8957)

56168fb

for example '?order=asc&group=j0fsj3oem3&tlang=en'

[civitai:user-posts] fix pagination (mikf#8955)

690b3ba

fix '400 Bad Request' errors when retrieving more than the first batch of posts.

[discord] add 'server-search' extractor

532ab71

requested on Discord https://discord.com/channels/SERVER_ID/search?from=USER_ID

[job] add 'output.jsonl' option (mikf#8953)

3445c51

[tiktok] Restructure to allow user extractors to provide their own re…

2d01fef

…hydration data (mikf#8848)

[tiktok] do not fail entire extraction if one post fails (mikf#8962)

01657ca

feat(toongod): add support with FlareSolverr integration

b233ec7

Add extractor for toongod.org webtoon site with Cloudflare bypass support using FlareSolverr proxy.

fix(toongod): fix flake8 linting errors

c93d8f7

- Fix line length issues (max 79 chars) - Fix continuation line indentation

mikf added site:support category:manhwa labels Jan 31, 2026

nthduy force-pushed the feat/toongod-extractor branch 5 times, most recently from 6186e4a to bf23ef8 Compare January 31, 2026 14:39

nthduy added 3 commits January 31, 2026 15:45

fix(toongod): add __init__ to properly initialize URL

a125124

perf(toongod): add FlareSolverr session management

b63ff7b

fix(toongod): add __init__ method

f654cbc

nthduy force-pushed the feat/toongod-extractor branch from bf23ef8 to f654cbc Compare January 31, 2026 14:45

nthduy added 2 commits January 31, 2026 16:11

fix(toongod): set subcategory to match class name

9eb9c6e

nthduy force-pushed the feat/toongod-extractor branch from d9c75d2 to 1407564 Compare February 10, 2026 23:56

mikf force-pushed the master branch from ae7a315 to 325c64f Compare March 28, 2026 12:16

mikf force-pushed the master branch from 8137219 to 9f3d2b3 Compare April 5, 2026 18:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[toongod] add support#8963

[toongod] add support#8963
nthduy wants to merge 7771 commits intomikf:masterfrom
nthduy:feat/toongod-extractor

nthduy commented Jan 30, 2026 •

edited

Loading

Uh oh!

nthduy commented Feb 1, 2026

Uh oh!

Dragonatorul commented Mar 15, 2026

Uh oh!

Dragonatorul commented Mar 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

Uh oh!

Conversation

nthduy commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nthduy commented Feb 1, 2026

Uh oh!

Dragonatorul commented Mar 15, 2026

Uh oh!

Dragonatorul commented Mar 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

19 participants

nthduy commented Jan 30, 2026 •

edited

Loading