Skip to content

Extend azure.free_disk_space to Windows as well#1966

Draft
jakirkham wants to merge 4 commits intoconda-forge:mainfrom
jakirkham:azure_win_cleanup
Draft

Extend azure.free_disk_space to Windows as well#1966
jakirkham wants to merge 4 commits intoconda-forge:mainfrom
jakirkham:azure_win_cleanup

Conversation

@jakirkham
Copy link
Copy Markdown
Member

In addition to Linux, enable azure.free_disk_space on Windows. This helps free up some space in cases where it is needed. Also it can fix build issues where a system Python is accidentally pulled into builds causing hard to diagnose issues.

Also switch Miniforge downloads to use PowerShell. This removes the reliance on the system Python for this step (making the aforementioned cleanup possibly). Additionally it slims the step down a bit and makes it a bit easier to parse.


Checklist

  • Added a news entry

@jakirkham jakirkham force-pushed the azure_win_cleanup branch 3 times, most recently from 32ed6d4 to 2df379e Compare June 28, 2024 08:14
@h-vetinari
Copy link
Copy Markdown
Member

👍

Xref #1949, which also contains a work-around.

@jakirkham
Copy link
Copy Markdown
Member Author

Thanks Axel! 🙏

Was finding the deletion time on Azure to be pretty slow. Tried every trick I could think of with robocopy

If you know of better ways to delete on Windows, please let me know

Agree changing the build directory path is worth considering

Copy link
Copy Markdown
Member

@h-vetinari h-vetinari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to guard this on CONDA_BLD_PATH somehow? I don't think it makes sense to spend time cleaning up C:\ if the build then happens in D:\ by default

"CONDA_BLD_PATH": "D:\\\\bld\\\\",

Speaking of D:\, I'm not sure there'd even be anything to clean up there? So I guess we'd have a three-stage ladder of escalating options, each taking more time but providing more disk space:

  • default (build on D:\; fastest, least space)
  • switch CONDA_BLD_PATH to C:\ (slower, more space)
  • also set azure.free_disk_space (slowest, most space)

@jakirkham
Copy link
Copy Markdown
Member Author

Agreed

We may also want to put this under another option like azure.free_disk_space: win so that doing true doesn't add this (similar to the docker option on Linux)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants