[Task Runner API] [Flower Interoperability] Pull ConnectorFlower, FlowerTaskRunner/FlowerDataLoader, and FlowerInteropServer/FlowerInteropClient into core OpenFL

[kminhta]

Note: This PR is a branch from #1598 so it includes Collaborator related commits that'll be mostly discarded once that PR is pulled and this one is rebased. The reason for the relatively large PR compared to the aggregator and collaborator PRs is because these had some interdependencies that was easier to handle at once.

• [Task Runner API] [Flower Interoperability] Integrate changes for flower-app-pytorch workspace's custom Aggregator into core component #1585
• [Task Runner API] [Flower Interoperability] Integrate changes for flower-app-pytorch workspace's custom Collaborator into core component  #1598
• remaining components...

Summary

This PR pulls in the ConnectorFlower, related interop components (FlowerInteropServer & FlowerInteropClient), and the FlowerTaskRunner & FlowerDataLoader back into core OpenFL as well as provides consistency in how gRPC ports are handled using the auto key and a hash of the plan (ref: #1433 (comment))

Type of Change (Mandatory)

Specify the type of change being made.

• Feature enhancement (improved gRPC port assignment in Flower components)
• Refactor (pulling in workspace components)

Description (Mandatory)

• Pulls in ConnectorFlower
  • rename the plan default connector.yaml -> connector_flower.yaml
  • move a few settings from plan.yaml into connector_flower.yaml and set some of the ports to auto
  • expand args in ConnectorFlower to reflect the new yaml key hierarchy (no longer embedded under superlink_params and flwr_run_params - this was necessary to be able to modify a subset of the default values without needing to include all the values in plan.yaml
  • add flwr_dir to the plan, which sets the FLWR_HOME directory (important for SGX and general security or else flower will build their app outside the workspace inside of /.flwr)
• Pulls in FlowerInteropServer / FlowerInteropClient
  • updates imports to reflect new directory structure
  • update FlowerInteropServer to add the automatically set port instead of dynamically adjust it itself
• Pulls in FlowerTaskRunner / FlowerDataLoader
  • rename the plan default task_connector.yaml -> task_flower.yaml
  • add defaults to prepare interop server, pass in client port, and local simulation
  • automatically assign port numbers
  • local simulation added to further adjust port to avoid port conflicts. uses collaborator name to create a hash then dynamically set port using from openfl.utilities.utils import generate_port
  • add flwr_dir to the Task Runner plan, which sets the FLWR_HOME directory (important for SGX and general security or else flower will build their app outside the workspace inside of /.flwr)
  • makes adjustment to install_flower_FAB to reflect new handling of flwr_dir and to minimize directory traversal
• new generate_port() which uses the same logic when getting the agg_port during the plan resolve in order to also auto generate gRPC ports for the connector/interop components. Updates openfl/federated/plan/plan.py to reflect this when initially resolving the plan
• handle path security by removing <workspace>/src/util/is_safe_path() and using OpenFL's built-in is_directory_traversal()
• pin flwr-nightly -> this version enables us to remove our patches, but later versions toward 1.9 stable release interferes with SGX (need to further investigate)
• refactoring in plan to support changes above
• Need to verify in SGX
• Need to update README and possibly docstrings

Testing

Manual in non-SGX and SGX

[teoparvanov]

This is great, thanks @kta-intel !