Chinese char in filename is ignored

[baohengtao]

seems Chinese char is cleaned by this function.

  def clean_filename(filename: str) -> str:
      """Replaces invalid chars in filenames with '_'"""
      result = filename.encode(
          "utf-8").decode("ascii", "ignore")
      invalid = '<>:"/\\|?*\0'
  
      for char in invalid:
          result = result.replace(char, '_')
  
      return result

the output "假期_01.jpg" is "_01.jpg" without Chinese char

In [1]: clean_filename("假期_01.jpg")
Out[1]: _01.jpg  

[AndreyNikiforov]

Am I understanding the problem correctly that icloudpd saves file with Chinese chars replaced with '_'?

[baohengtao]

Yea, suppose the actual filename is "假期_01.jpg", the filename downloaded by icloudpd will be "_01.jpg" and "假期“ is removed from the filename.

[boredazfcuk]

Is it possible that this is also the reason why quite a lot of Chinese users report "bad password" errors?

[AndreyNikiforov]

Is it possible that this is also the reason why quite a lot of Chinese users report "bad password" errors?

unlikely. the code for removing unicode chars from filenames seems to be there for ages

[boredazfcuk]

Yeah, just wasn't sure if something similar was happening when processing passwords... or if Unicode character support was something that needed to be added when capturing text and possibly omitted by default.

[AndreyNikiforov]

added 1.18.0