Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions changes/+m2m.added
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Added the ability to assign secrets to a secrets group when using the `secrets_group` module.
Added the ability to assign static group associations when using the `dynamic_group` module.
Added the ability to assign prefixes to a cloud network when using the `cloud_network` module.
Added the ability to assign cloud networks to a cloud service when using the `cloud_service` module.
Added the ability to assign vrfs to a device, virtual machine, or virtual device context when using the respective module.
Added the ability to assign clusters to a device when using the `device` module.
Added the ability to assign ip addresses to a device interface or vm interface when using the respective module.
Added the ability to assign prefixes or vlans to a location when using the `location` module.
Added the ability to assign provider networks to a provider when using the `provider` module.
Added the ability to manage custom field choices to a custom field when using the `custom_field` module.
Added the ability to manage metadata choices to a metadata type when using the `metadata_type` module.
12 changes: 12 additions & 0 deletions changes/+module.deprecated
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Deprecated the `secrets_groups_association` module. Use the `secrets` option of the `secrets_group` module instead.
Deprecated the `static_group_association` module. Use the `static_group_associations` option of the `dynamic_group` module instead.
Deprecated the `cloud_network_prefix_assignment` module. Use the `prefixes` option of the `cloud_network` module instead.
Deprecated the `cloud_service_network_assignment` module. Use the `cloud_networks` option of the `cloud_service` module instead.
Deprecated the `vrf_device_assignment` module. Use the `vrfs` option of the `device`, `virtual_machine`, or `virtual_device_context` module instead.
Deprecated the `device_cluster_assignment` module. Use the `clusters` option of the `device` module instead.
Deprecated the `ip_address_to_interface` module. Use the `ip_addresses` option of the `device_interface` or `vm_interface` module instead.
Deprecated the `prefix_location` module. Use the `prefixes` option of the `location` module instead.
Deprecated the `vlan_location` module. Use the `vlans` option of the `location` module instead.
Deprecated the `custom_field_choice` module. Use the `custom_field_choices` option of the `custom_field` module instead.
Deprecated the `metadata_choice` module. Use the `metadata_choices` option of the `metadata_type` module instead.
Deprecated the `provider_network` module. Use the `provider_networks` option of the `provider` module instead.
46 changes: 46 additions & 0 deletions docs/getting_started/contributing/modules/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -291,3 +291,49 @@ The next few lines manipulate the data and prepare it for sending to Nautobot.
- Converts any fields that are namespaced to prevent conflicts when searching for them (e.g. device_role, ipam_role, rack_group, etc.)

If all those pass, it sets the manipulated data to `self.data` that is used in the module util apps.

## Inline M2M Fields

+++ 6.2.0

Some parent modules support inline many-to-many (M2M) association fields. These are declared in the `M2M_FIELDS` global dict in `utils.py`, which maps each endpoint to its supported M2M fields and their association API endpoints:

```python
M2M_FIELDS = {
"devices": {
"vrfs": "vrf_device_assignments",
"clusters": "device_cluster_assignments",
},
"locations": {
"prefixes": "prefix_location_assignments",
"vlans": "vlan_location_assignments",
},
# ... etc.
}
```
Comment thread
jvanderaa marked this conversation as resolved.

### How M2M Fields Are Processed

During `__init__`, M2M fields are handled separately from regular fields to prevent collisions with `_find_ids`:

1. **Extraction**: M2M field data is popped from the data dict *before* `_find_ids` runs on the parent data. This prevents M2M field names (like `prefixes`) from being mistakenly resolved as direct FK relationships.
2. **Child resolution**: Each M2M object's child key (e.g., `vrf`, `prefix`, `ip_address`) is individually run through `_find_ids` to resolve names to UUIDs. This supports both simple strings (`vrf: "My VRF"`) and dicts for disambiguation (`ip_address: {address: "10.0.0.1/24", namespace: "Global"}`).
3. **Payload stripping**: M2M field names are added to `remove_keys` so they are stripped from the REST API payload sent for the parent object.

After the parent object is created or updated, `_ensure_object_exists` calls `_process_m2m_fields` which iterates through each M2M field and:

- Fetches current associations from the API
- Compares desired vs current using normalized comparison keys
- Applies the requested state (`merge`, `replace`, or `delete`) via bulk create/delete operations

### Adding M2M Support to a New Module

If a new association endpoint is added to Nautobot and you want to support inline management:

1. Add the mapping to `M2M_FIELDS` in `utils.py`
2. Add the M2M argument to the parent module's `argument_spec` following the standard structure (state/objects/child_key suboptions)
3. Add matching `DOCUMENTATION` with suboptions
4. Ensure the child key is in `CONVERT_TO_ID` and has appropriate `QUERY_TYPES`, `ENDPOINT_NAME_MAPPING`, and `ALLOWED_QUERY_PARAMS` entries
5. Add integration tests covering merge, replace, delete, and idempotency

No changes to the module's `main()` function or constructor are needed -- the framework auto-detects M2M fields from `M2M_FIELDS` based on `self.endpoint`.
207 changes: 207 additions & 0 deletions docs/getting_started/how-to-use/inline_m2m.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
# Inline Many-to-Many Associations

+++ 6.2.0

Many Nautobot models have many-to-many (M2M) relationships with other models. For example, a device can be associated with multiple VRFs, a location can have multiple prefixes, and a secrets group can contain multiple secrets.

Previously, managing these associations required using separate standalone modules (e.g., `vrf_device_assignment`, `prefix_location`, `secrets_groups_association`). Now you can manage them inline on the parent module using the M2M field options.

## Supported Parent Modules and Fields

| Parent Module | M2M Field | Child Object Key | Description |
|---|---|---|---|
| `device` | `vrfs` | `vrf` | VRF assignments |
| `device` | `clusters` | `cluster` | Cluster assignments |
| `virtual_machine` | `vrfs` | `vrf` | VRF assignments |
| `virtual_device_context` | `vrfs` | `vrf` | VRF assignments |
| `device_interface` | `ip_addresses` | `ip_address` | IP address to interface |
| `vm_interface` | `ip_addresses` | `ip_address` | IP address to VM interface |
| `location` | `prefixes` | `prefix` | Prefix to location |
| `location` | `vlans` | `vlan` | VLAN to location |
| `cloud_network` | `prefixes` | `prefix` | Cloud network prefix assignments |
| `cloud_service` | `cloud_networks` | `cloud_network` | Cloud service network assignments |
| `secrets_group` | `secrets` | `secret` | Secrets group associations |
| `dynamic_group` | `static_group_associations` | `associated_object_type` / `associated_object_id` | Static group memberships |
| `custom_field` | `custom_field_choices` | `value` | Custom field choices |
| `metadata_type` | `metadata_choices` | `value` | Metadata type choices |
| `provider` | `provider_networks` | `name` | Provider networks |

## M2M Field Structure

All M2M fields follow the same structure:

```yaml
<m2m_field>:
state: merge # merge (default), replace, or delete
objects:
- <child_key>: "value"
- <child_key>: "another value"
```

### States

- **merge** (default): Adds the specified associations without removing existing ones. Safe for incremental changes.
- **replace**: Enforces exactly the listed associations. Any existing associations not in the list are removed.
- **delete**: Removes only the specified associations. Other existing associations are left intact.

## Basic Examples

### Adding VRFs to a Device

```yaml
- name: Create a device with VRF associations
networktocode.nautobot.device:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
name: "my-router"
device_type: "Cisco CSR1000v"
role: "Router"
location: "Main Site"
status: "Active"
vrfs:
objects:
- vrf: "Management VRF"
- vrf: "Production VRF"
state: present
```

Since no `state` is specified on the `vrfs` field, it defaults to `merge` -- the VRFs are added without affecting any other existing VRF associations.

### Adding IP Addresses to an Interface

The child key accepts either a simple string or a dictionary for more specific lookups:

```yaml
# Simple string -- looks up the IP address by address
- name: Associate IP addresses with an interface
networktocode.nautobot.device_interface:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
device: "my-router"
name: "GigabitEthernet0/0"
ip_addresses:
objects:
- ip_address: "10.0.0.1/24"
state: present

# Dictionary -- useful when disambiguation is needed (e.g., multiple namespaces)
- name: Associate IP address with namespace specified
networktocode.nautobot.device_interface:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
device: "my-router"
name: "GigabitEthernet0/0"
ip_addresses:
objects:
- ip_address:
address: "10.0.0.1/24"
namespace: "Production"
state: present
```

### Adding Secrets to a Secrets Group

Some M2M associations have extra fields beyond just the child identifier. For secrets group associations, `access_type` and `secret_type` are part of the association itself:

```yaml
- name: Create secrets group with secret associations
networktocode.nautobot.secrets_group:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
name: "Device Credentials"
secrets:
objects:
- secret: "admin-username"
access_type: "SSH"
secret_type: "username"
- secret: "admin-password"
access_type: "SSH"
secret_type: "password"
state: present
```

## Managing Association State

### Merge (Default)

Merge adds new associations without removing existing ones. Running the same task twice is idempotent.

```yaml
# First run: adds VRF-A
- name: Add first VRF
networktocode.nautobot.device:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
name: "my-router"
vrfs:
objects:
- vrf: "VRF-A"
state: present

# Second run: adds VRF-B, VRF-A is untouched
- name: Add second VRF
networktocode.nautobot.device:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
name: "my-router"
vrfs:
objects:
- vrf: "VRF-B"
state: present
# Result: device has both VRF-A and VRF-B
```

### Replace

Replace enforces exactly the listed set of associations. Any existing associations not in the list are removed.

```yaml
# Device currently has VRF-A and VRF-B
- name: Replace all VRFs with only VRF-C
networktocode.nautobot.device:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
name: "my-router"
vrfs:
state: replace
objects:
- vrf: "VRF-C"
state: present
# Result: device has only VRF-C (VRF-A and VRF-B removed)
```

### Delete

Delete removes only the specified associations. Other associations are left intact.

```yaml
# Device currently has VRF-A and VRF-B
- name: Remove only VRF-B
networktocode.nautobot.device:
url: "{{ nautobot_url }}"
token: "{{ nautobot_token }}"
name: "my-router"
vrfs:
state: delete
objects:
- vrf: "VRF-B"
state: present
# Result: device has only VRF-A
```

## Diff Output

When M2M fields change, the diff output includes the before and after state as sorted lists of child object UUIDs:

```json
{
"diff": {
"before": {
"vrfs": ["<uuid-of-vrf-a>"]
},
"after": {
"vrfs": ["<uuid-of-vrf-a>", "<uuid-of-vrf-b>"]
}
}
}
```
74 changes: 74 additions & 0 deletions meta/runtime.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,77 @@ plugin_routing:
deprecation:
removal_version: "7.0.0"
warning_text: "Use networktocode.nautobot.graphql_info or networktocode.nautobot.graphql_facts instead."
secrets_groups_association:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.secrets_group module with the
'secrets' option instead.
static_group_association:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.dynamic_group module with the
'static_group_associations' option instead.
cloud_network_prefix_assignment:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.cloud_network module with the
'prefixes' option instead.
cloud_service_network_assignment:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.cloud_service module with the
'cloud_networks' option instead.
vrf_device_assignment:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.device, networktocode.nautobot.virtual_machine,
or networktocode.nautobot.virtual_device_context module with the
'vrfs' option instead.
device_cluster_assignment:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.device module with the
'clusters' option instead.
ip_address_to_interface:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.device_interface or
networktocode.nautobot.vm_interface module with the
'ip_addresses' option instead.
prefix_location:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.location module with the
'prefixes' option instead.
vlan_location:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.location module with the
'vlans' option instead.
custom_field_choice:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.custom_field module with the
'custom_field_choices' option instead.
metadata_choice:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.metadata_type module with the
'metadata_choices' option instead.
provider_network:
deprecation:
removal_version: "7.0.0"
warning_text: >-
Use the networktocode.nautobot.provider module with the
'provider_networks' option instead.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ nav:
- How to Use:
- Modules: "getting_started/how-to-use/modules.md"
- Inventory: "getting_started/how-to-use/inventory.md"
- Inline M2M Associations: "getting_started/how-to-use/inline_m2m.md"
- Advanced Usage - Modules: "getting_started/how-to-use/advanced.md"
- EDA Event Source Plugin: "getting_started/how-to-use/eda.md"
- Migrating from query_graphql: "getting_started/how-to-use/graphql_migration.md"
Expand Down
Loading
Loading