Skip to content

Add a "History" tab #2396

Merged
AUTOMATIC1111 merged 22 commits into
AUTOMATIC1111:masterfrom
yfszzx:master
Oct 14, 2022
Merged

Add a "History" tab #2396
AUTOMATIC1111 merged 22 commits into
AUTOMATIC1111:masterfrom
yfszzx:master

Conversation

@yfszzx
Copy link
Copy Markdown
Contributor

@yfszzx yfszzx commented Oct 12, 2022

Last two PR closed is here and here

I think I completed all requirements in this PR comment

    1. this repo uses 4 spaces indentation, no tabs. Fix the indents also fix the double indents to be single ones.
    1. there are unneeded prints
    1. was this tested on images in subdirectories? There is an option to save images to subdirs, for example I save all mine to directories with current date, so I have stable-diffusion-webui/outputs/txt2img-images/2022-10-10/00017-3508344613.png.
    1. how many images did you have in directories when you tested it?
    1. adding jquery is a big decision. This seems to have just a tiny bit of JS so i think it should be possible to write it without jquery.

Originally posted by @AUTOMATIC1111 in #2164 (comment)_

and standardized the code of JS according to suggestions in these PR comment by @moreRGB

It is worth mentioning that the problem of images in subdirectories has been completely solved

I think this “History”page has been running very well and smoothly. I hope it can be merged this time, thanks

——————————————————————————————————

This pull request is trying to build a tab browser of previous pictures in output directory for txt2img and img2img. We can easily browse the previously generated images, display the image generation information in PNG meta, push the image information to "txt2img" or "img2img", or delete the images we no longer need.

I try my best to encapsulate the new code independently to avoid affecting other functions of the project .
To facilitate the operation of HTML controls,I import jQuery Framework in the js script.

OS: both Win10, and Ubuntu Server
Browser Chrome,
Graphics card NVIDIA RTX 2080ti 11GB

@yfszzx yfszzx requested a review from AUTOMATIC1111 as a code owner October 12, 2022 16:59
@AUTOMATIC1111
Copy link
Copy Markdown
Owner

all right, walking slowly is preferred here.

What is this:
firefox_e5Viv6HdBu

Also you don't have to close you PRs, you can just commit more and your new commits will automatically appear here.

@innovaciones
Copy link
Copy Markdown
Collaborator

innovaciones commented Oct 12, 2022

I'm testing this on Colab with Google Drive and it looks nice.

But I'm getting some errors on the console:

When selecting an image from the gallery:

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30) at images_history_set_image_info ((índice):826:13) at HTMLButtonElement.images_history_click_image ((índice):751:5)

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'forEach') at Y (index.e0c646e0.js:55:2374)

When moving to the next page of the gallery:

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30)

index.e0c646e0.js:55 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'forEach') at Y (index.e0c646e0.js:55:2374)

When generating more images:

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30) at requestProgress ((índice):1294:9) at submit ((índice):1431:5) at eval (eval at <anonymous> (index.e0c646e0.js:54:2777), <anonymous>:3:28) at index.e0c646e0.js:56:4850 at HTMLButtonElement.<anonymous> (index.e0c646e0.js:55:2122) at index.e0c646e0.js:4:1266 at Array.forEach (<anonymous>) at HTMLButtonElement.Zn (index.e0c646e0.js:4:1253) at HTMLButtonElement._ (index.adf51434.js:1:2092)

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:3

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30) at requestMoreProgress ((índice):1278:9) at (índice):1262:44

I guess the problem is that the files are in Google Drive and somehow the promise is not responding.

Everything seems to work fine but is a little slow, selecting and loading a picture takes 2-3 seconds, loading a new page takes 10-20 seconds, again I guess this is because everything is loading from Google Drive.

Also instead of "Renew Page" I think it works better: "Reload Page" or "Refresh Page"

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 12, 2022

I'm testing this on Colab with Google Drive and it looks nice.

But I'm getting some errors on the console:

When selecting an image from the gallery:

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30) at images_history_set_image_info ((índice):826:13) at HTMLButtonElement.images_history_click_image ((índice):751:5)

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'forEach') at Y (index.e0c646e0.js:55:2374)

When moving to the next page of the gallery:

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30)

index.e0c646e0.js:55 Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'forEach') at Y (index.e0c646e0.js:55:2374)

When generating more images:

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30) at requestProgress ((índice):1294:9) at submit ((índice):1431:5) at eval (eval at <anonymous> (index.e0c646e0.js:54:2777), <anonymous>:3:28) at index.e0c646e0.js:56:4850 at HTMLButtonElement.<anonymous> (index.e0c646e0.js:55:2122) at index.e0c646e0.js:4:1266 at Array.forEach (<anonymous>) at HTMLButtonElement.Zn (index.e0c646e0.js:4:1253) at HTMLButtonElement._ (index.adf51434.js:1:2092)

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:3

Uncaught TypeError: source.indexOf is not a function at ShadowRoot.<anonymous> ((índice):358:30) at requestMoreProgress ((índice):1278:9) at (índice):1262:44

I guess the problem is that the files are in Google Drive and somehow the promise is not responding.

Everything seems to work fine but is a little slow, selecting and loading a picture takes 2-3 seconds, loading a new page takes 10-20 seconds, again I guess this is because everything is loading from Google Drive.

Also instead of "Renew Page" I think it works better: "Reload Page" or "Refresh Page"

@innovaciones
This problem has puzzled me for a long time. Finally, I found that it is not my problem, but a bug in the previous code in file \javascript\contextMenu.js. in line:97
if(source.id && source.indexOf('check_progress')>-1){ return }

I have issued an issue to report this error,

source.id && source.indexOf('check_progress

source.id 'source' must be a DOM
source.indexOf 'source' must be a string

So it's conflicting

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 13, 2022

all right, walking slowly is preferred here.

What is this: firefox_e5Viv6HdBu

Also you don't have to close you PRs, you can just commit more and your new commits will automatically appear here.

I also don't know what strange things this is

Maybe it's because there was another . cpkt model file in my webui directory, but I removed it and executed “git commit”. But git tell me “nothing to commit, working tree clean”

I hope some experts can tell me

And I did not close the previous PRs, which seems to be automatically closed? It seems that there is too much to learn about github

@innovaciones
Copy link
Copy Markdown
Collaborator

@innovaciones This problem has puzzled me for a long time. Finally, I found that it is not my problem, but a bug in the previous code in file \javascript\contextMenu.js. in line:97 if(source.id && source.indexOf('check_progress')>-1){ return }

I have issued an issue to report this error,

source.id && source.indexOf('check_progress

source.id 'source' must be a DOM source.indexOf 'source' must be a string

So it's conflicting

You are right, this bug is from the new contextMenu feature from @dfaker, I commented out the code and the errors are gone

@dfaker
Copy link
Copy Markdown
Collaborator

dfaker commented Oct 13, 2022

Gah, never the exciting things, addressed in #2447

@evanjs
Copy link
Copy Markdown

evanjs commented Oct 13, 2022

It is worth mentioning that the problem of images in subdirectories has been completely solved

If I understand correctly, this implementation seems to account for a maximum of 1 sub-directories.
Not sure if there's a practical solution for handling an arbitrary depth of folders at the moment, as deeply nested structures often imply a need for such -> a large number of files, etc. ...

I am currently trying to hack something together with glob), but I wanted to at least make sure the point is made.
Happy to be wrong as well 😄

Otherwise, My non-deeply-nested directory seem to appears as intended.


Update: Not suggesting this should be implemented 1:1 or anything, but I was able to at least load my history using rglob
Initial changes
Fix get_current_image

@JC-Array
Copy link
Copy Markdown
Contributor

Maybe it's because there was another . cpkt model file in my webui directory, but I removed it and executed “git commit”. But git tell me “nothing to commit, working tree clean”

This founds like the file is still tracked in git. Have you tried the command “git rm ”?

After this you should be able to run the commit to remove it.

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 14, 2022

Maybe it's because there was another . cpkt model file in my webui directory, but I removed it and executed “git commit”. But git tell me “nothing to commit, working tree clean”

This founds like the file is still tracked in git. Have you tried the command “git rm ”?

After this you should be able to run the commit to remove it.

thank you ,The problem has been solved

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 14, 2022

If I understand correctly, this implementation seems to account for a maximum of 1 sub-directories. Not sure if there's a practical solution for handling an arbitrary depth of folders at the moment, as deeply nested structures often imply a need for such -> a large number of files, etc. ...

I am currently trying to hack something together with glob), but I wanted to at least make sure the point is made. Happy to be wrong as well 😄

Otherwise, My non-deeply-nested directory seem to appears as intended.

I have considered the problem of deep nesting, but the subdirectories mode of webui does not generate deep nesting directories, so I have not considered this possibility. If there is such a requirement in actual use, we can make further improvements. Thank you for your suggestions

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 14, 2022

all right, walking slowly is preferred here.

What is this: firefox_e5Viv6HdBu

Also you don't have to close you PRs, you can just commit more and your new commits will automatically appear here.

@AUTOMATIC1111 All known problems have been solved, and a function of continuously deleting multiple files has been added, and some improvements have been made to prevent accidental deletion of files

Expect my work to be merged into master brach of webui

And I strongly recommend combine ‘’PNG Info‘’ block with my “History” block in the future,just add a gradio.Image to my page,to make the main page more concise.

@evanjs
Copy link
Copy Markdown

evanjs commented Oct 14, 2022

If I understand correctly, this implementation seems to account for a maximum of 1 sub-directories. Not sure if there's a practical solution for handling an arbitrary depth of folders at the moment, as deeply nested structures often imply a need for such -> a large number of files, etc. ...

I am currently trying to hack something together with glob), but I wanted to at least make sure the point is made. Happy to be wrong as well 😄

Otherwise, My non-deeply-nested directory seem to appears as intended.

I have considered the problem of deep nesting, but the subdirectories mode of webui does not generate deep nesting directories, so I have not considered this possibility. If there is such a requirement in actual use, we can make further improvements. Thank you for your suggestions

Just to clarify, setting Directory name pattern (under Saving a directory) to anything greater than one directory deep will reproduce this issue.

Mine is currently set to [model_hash]/[prompt_words].

Either way, this would be good to get merged if Auto is okay with it and it works with most users/configurations 👍

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 14, 2022

If I understand correctly, this implementation seems to account for a maximum of 1 sub-directories. Not sure if there's a practical solution for handling an arbitrary depth of folders at the moment, as deeply nested structures often imply a need for such -> a large number of files, etc. ...

I am currently trying to hack something together with glob), but I wanted to at least make sure the point is made. Happy to be wrong as well 😄

Otherwise, My non-deeply-nested directory seem to appears as intended.

I have considered the problem of deep nesting, but the subdirectories mode of webui does not generate deep nesting directories, so I have not considered this possibility. If there is such a requirement in actual use, we can make further improvements. Thank you for your suggestions

Just to clarify, setting Directory name pattern (under Saving a directory) to anything greater than one directory deep will reproduce this issue.

Mine is currently set to [model_hash]/[prompt_words].

Either way, this would be good to get merged if Auto is okay with it and it works with most users/configurations 👍

I see, I will fix immediately

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 14, 2022

deep nesting directories

deep nesting directories problem is solved

@evanjs
Copy link
Copy Markdown

evanjs commented Oct 14, 2022

deep nesting directories

deep nesting directories problem is solved

Finally got around to testing the fix
Browsing, image details display, and deleting all seem to work nicely

Images from both deeply nested directories and top-level files also seem to behave the same, as expected.

I can confirm that deeply nested directories indeed seem to be handled properly now, at least on my system.

Thank you!

@AUTOMATIC1111 AUTOMATIC1111 merged commit 6b77af7 into AUTOMATIC1111:master Oct 14, 2022
@AUTOMATIC1111
Copy link
Copy Markdown
Owner

Well, loading a page takes about 10 seconds on my data, because I have a lot of images and this implementation needs to list them all, but apart from that, seems to work.

I would recommend considering an implementation where go through directories in last-modified orders, and stop early, after you've gotten enough iles to show on page - this way no matter how large your collection, you will be able to display results quickly.

Another issue of course is big galleys slow down the browser to a crawl, but we are waiting for gradio to fix that.

@shoraaa
Copy link
Copy Markdown

shoraaa commented Oct 14, 2022

Consider adding an option to disable this feature completely for now, since big galleys really slowing the browser. I can even browsing the history section properly since it too laggy.
Possible option is to cache and load only up to specified limit of last modified images (like #AUTOMATIC1111 suggest), and only cache another batch of images when needed.

@innovaciones
Copy link
Copy Markdown
Collaborator

Sadly now is not working with Google Colab (with or without Google Drive), I guess the "deep nesting directories" feature is the culprit, before that it worked fine.

error

Error from Colab console:

File "/content/stable-diffusion-webui/modules/images_history.py", line 54, in page_index_change return get_recent_images(dir_name, page_index, 0, image_index, tabname) File "/content/stable-diffusion-webui/modules/images_history.py", line 25, in get_recent_images f_list = os.listdir(dir_name) FileNotFoundError: [Errno 2] No such file or directory: 'content/outputs/txt2img-images'

Error from browser console:

Failed to load resource: the server responded with a status of 500 ()
Uncaught (in promise) API Error

Comment thread modules/images_history.py
for p in d[1:]:
dir_name = os.path.join(dir_name, p)
with gr.Row():
renew_page = gr.Button('Renew Page', elem_id=tabname + "_images_history_renew_page")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you plan to do another PR for the folder issue, I like to suggest to rename some names

"Renew Page" -> simply "Refresh" or even better "🔄" as this is used already on the settings page
"End Page" -> "Last Page"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I second this, like I already said "Renew Page" is not a good fit, "Refresh" works better

@moreRGB
Copy link
Copy Markdown

moreRGB commented Oct 14, 2022

No such file or directory: 'content/outputs/txt2img-images'

A check if the folder (the same is true for img2img) would be sufficent I guess. It also crashes on fresh runs.

@innovaciones
Copy link
Copy Markdown
Collaborator

No such file or directory: 'content/outputs/txt2img-images'

A check if the folder (the same is true for img2img) would be sufficent I guess. It also crashes on fresh runs.

The folder is actually there, but for some reason is not detecting it

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 15, 2022

Well, loading a page takes about 10 seconds on my data, because I have a lot of images and this implementation needs to list them all, but apart from that, seems to work.

I would recommend considering an implementation where go through directories in last-modified orders, and stop early, after you've gotten enough iles to show on page - this way no matter how large your collection, you will be able to display results quickly.

Another issue of course is big galleys slow down the browser to a crawl, but we are waiting for gradio to fix that.

too many file in a dir

Sadly now is not working with Google Colab (with or without Google Drive), I guess the "deep nesting directories" feature is the culprit, before that it worked fine.

error

Error from Colab console:

File "/content/stable-diffusion-webui/modules/images_history.py", line 54, in page_index_change return get_recent_images(dir_name, page_index, 0, image_index, tabname) File "/content/stable-diffusion-webui/modules/images_history.py", line 25, in get_recent_images f_list = os.listdir(dir_name) FileNotFoundError: [Errno 2] No such file or directory: 'content/outputs/txt2img-images'

Error from browser console:

Failed to load resource: the server responded with a status of 500 () Uncaught (in promise) API Error

I suggest you try changing “”Paths for saving “ in setting page,Then create the directories manually instead of automatically generated by the system

1 similar comment
@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 15, 2022

Well, loading a page takes about 10 seconds on my data, because I have a lot of images and this implementation needs to list them all, but apart from that, seems to work.

I would recommend considering an implementation where go through directories in last-modified orders, and stop early, after you've gotten enough iles to show on page - this way no matter how large your collection, you will be able to display results quickly.

Another issue of course is big galleys slow down the browser to a crawl, but we are waiting for gradio to fix that.

too many file in a dir

Sadly now is not working with Google Colab (with or without Google Drive), I guess the "deep nesting directories" feature is the culprit, before that it worked fine.

error

Error from Colab console:

File "/content/stable-diffusion-webui/modules/images_history.py", line 54, in page_index_change return get_recent_images(dir_name, page_index, 0, image_index, tabname) File "/content/stable-diffusion-webui/modules/images_history.py", line 25, in get_recent_images f_list = os.listdir(dir_name) FileNotFoundError: [Errno 2] No such file or directory: 'content/outputs/txt2img-images'

Error from browser console:

Failed to load resource: the server responded with a status of 500 () Uncaught (in promise) API Error

I suggest you try changing “”Paths for saving “ in setting page,Then create the directories manually instead of automatically generated by the system

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 15, 2022

Well, loading a page takes about 10 seconds on my data, because I have a lot of images and this implementation needs to list them all, but apart from that, seems to work.

I would recommend considering an implementation where go through directories in last-modified orders, and stop early, after you've gotten enough iles to show on page - this way no matter how large your collection, you will be able to display results quickly.

Another issue of course is big galleys slow down the browser to a crawl, but we are waiting for gradio to fix that.

How about create an automatic sorting function to sort all pictures in the directory into different directories by date?

It is necessary to change the directory structure of "Paths for saving".
Subdirectories of txt2mg or img2img or extras directory must be the date directory first, and the directory according to the “Directory name pattern“ the under the date directory, for example

output\txt2img\20221015\[model_hash]/[prompt_words]

If you think it's OK to force the directory structure to change, I can implement the function of of automatically arranging pictures and “go through directories in last modified orders“

@yfszzx
Copy link
Copy Markdown
Contributor Author

yfszzx commented Oct 15, 2022

Sadly now is not working with Google Colab (with or without Google Drive), I guess the "deep nesting directories" feature is the culprit, before that it worked fine.

error

Error from Colab console:

File "/content/stable-diffusion-webui/modules/images_history.py", line 54, in page_index_change return get_recent_images(dir_name, page_index, 0, image_index, tabname) File "/content/stable-diffusion-webui/modules/images_history.py", line 25, in get_recent_images f_list = os.listdir(dir_name) FileNotFoundError: [Errno 2] No such file or directory: 'content/outputs/txt2img-images'

Error from browser console:

Failed to load resource: the server responded with a status of 500 () Uncaught (in promise) API Error

I know the reason now, see this:

https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

windows has a limit of path length, maybe other OS also has similar rule
@AUTOMATIC1111 the method of adding prompt to file name or dir name has great hidden trouble

@evanjs
Copy link
Copy Markdown

evanjs commented Oct 15, 2022

@yfszzx I think we should be okay now that 4ed99d5 was merged.
At least relating to performance issues from displaying a large number of images.

The above commit bumped the gradio dependency to 3.5.0, which included a fix to performance issues for Gradio and how it displayed images (previously raw base64 for each instance, if I understand correctly).

See also: gradio-app/gradio#2265

This was referenced Oct 16, 2022
JustAnOkapi pushed a commit to JustAnOkapi/stable-diffusion-webui that referenced this pull request Mar 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants