Update the `specification` directory structure article by konrad-jamrozik · Pull Request #28464 · Azure/azure-rest-api-specs

konrad-jamrozik · 2024-03-26T20:50:06Z

Link for easy review of the resulting document:

https://github.com/Azure/azure-rest-api-specs/blob/main/documentation/directory-structure.md

A follow-up to:

https://dev.azure.com/azure-sdk/internal/_git/azure-sdk-docs-eng.ms/pullrequest/942

As I continue to work on overhauling our documentation.

This PR contributes to:

Documentation: Deduplicate specs directory structure (folder layout) guidance azure-sdk-tools#7628

Notable changes:

I removed a lot of details about service groups and added a note we do not support new service group going forward, just existing ones.
- There is a lot of related discussion in the email thread with subject RE: Typespec questions
I deduped some information about TypeSpec and instead linked to relevant articles.
I removed mention of profile. The repository has profile and profiles folders but these directories appear to be legacy to me, untouched for 3 years.
I removed mention of "component entities" from several places. I believe this was related to the fact there was a 1 to 1 mapping between ARM services and Service Tree service components.
I removed the mention that a GA can have a component preview. This should not be allowed.

Additional notes:

Here: [TypeSpec Requirement] Fail if spec contains OpenAPI generated from TypeSpec, but no tspconfig.yaml #28444 we proposed the name of spec family for what in this article is called either resource provider or service group. We should sort out this nomenclature.
I am going to dedup following article: https://eng.ms/docs/products/azure-developer-experience/design/api-specs/api-dir-layout. I will link to the PR with the dedup here once I have it.
Related work and discussion: Enforce unified API versions in RP specs #28466
Relevant email threads with subjects:
- Re: Enforcing Unified API Versions in azure-rest-api-specs
  - Previously Re: TypeSpecValidation and brownfield specs in RPSaaSMaster
- Re: Preview specs in stable folder - wrong documentation?
- RE: Typespec questions

TODOs:

Dedup and incorporate this PR info from this page
Consider adding and mentioning aka.ms link. See e.g. aka.ms/azsdk/api-dir-layout.

openapi-pipeline-app · 2024-03-26T20:50:09Z

Next Steps to Merge

✅ All automated merging requirements have been met! To get your PR merged, see aka.ms/azsdk/specreview/merge.

openapi-pipeline-app · 2024-03-26T20:50:10Z

Swagger Validation Report

️️✔️BreakingChange succeeded [Detail] [Expand]

There are no breaking changes.

️️✔️Breaking Change(Cross-Version) succeeded [Detail] [Expand]

There are no breaking changes.

️️✔️CredScan succeeded [Detail] [Expand]

There is no credential detected.

️️✔️LintDiff succeeded [Detail] [Expand]

Validation passes for LintDiff.

️️✔️Avocado succeeded [Detail] [Expand]

Validation passes for Avocado.

️️✔️SwaggerAPIView succeeded [Detail] [Expand]

️️✔️TypeSpecAPIView succeeded [Detail] [Expand]

️️✔️ModelValidation succeeded [Detail] [Expand]

Validation passes for ModelValidation.

️️✔️SemanticValidation succeeded [Detail] [Expand]

Validation passes for SemanticValidation.

️️✔️PoliCheck succeeded [Detail] [Expand]

Validation passed for PoliCheck.

️️✔️SpellCheck succeeded [Detail] [Expand]

Validation passes for SpellCheck.

️️✔️Lint(RPaaS) succeeded [Detail] [Expand]

Validation passes for Lint(RPaaS).

️️✔️PR Summary succeeded [Detail] [Expand]

Validation passes for Summary.

️️✔️Automated merging requirements met succeeded [Detail] [Expand]

_{Posted by Swagger Pipeline | How to fix these errors?}

openapi-pipeline-app · 2024-03-26T20:50:10Z

Swagger Generation Artifacts

️️✔️ApiDocPreview succeeded [Detail] [Expand]

_{Posted by Swagger Pipeline | How to fix these errors?}

openapi-pipeline-app · 2024-03-26T20:50:11Z

PR validation pipeline restarted successfully. If there is ApiView generated, it will be updated in this comment.

konrad-jamrozik · 2024-03-26T21:05:30Z

+TypeSpec folders are `specification/playfab/Playfab`.  
+`rpnamespace` names present are: `Microsoft.Automation` (in `automation` rp), `Microsoft.Batch` (in `batch` rp) and `Microsoft.Playfab` (in `playfab` rp).
+
+## Folder Structure for service groups


@JeffreyRichter @rkmanda I significantly shortened this section and added a note of this not being allowed going forward. Can you review?

Lets setup a meeting and discuss.

maririos · 2024-03-26T21:31:08Z

+If TypeSpec sources are present, OpenAPI specs in the `resource-manager`
+and `data-plane` folders are generated from TypeSpec sources.
+
+## Resource provider namespace (`rpnamespace`) folders


Should this be higher in the list? before TypeSpec sources?

rkmanda

🕐

konrad-jamrozik · 2024-03-27T04:46:31Z

The doc edited in this PR doesn't seem to currently take into account the following special case from the email thread Re: TypeSpecValidation and brownfield specs in RPSaaSMaster by @JeffreyRichter, and should be properly documented here:

Some "services" MIGHT be divided into sub-services; for example: azure-rest-api-specs/specification/hdinsight/resource-manager/Microsoft.HDInsight at main · Azure/azure-rest-api-specs (github.com)
Here Microsoft.HDInsight (the parent) has preview & stable folders.
But there is also a HDInsightsOnAks folder which also has preview & stable.

This would surface to customers a 2 separate Azure services, and each could version independently.
We would ship 1 SDK for HDInsights and a completely separate SDK for HDInsightsOnAks.
This example is OK with me/us.

This probably means I should un-delete & update this part:

If the AutoRest configuration file (aka. the readme.md) is placed out of sub-service/sub-component folders, then there will be only one SDK package that holds all sub-services/sub-components. If the file is placed in each sub-service/sub-component folder, then there will be separate SDK packages of each sub-service/sub-component. Ensure to consult Azure SDK ArchBoard for SDK packaging strategy when consolidating AutoRest configuration file for SDK generation.

lfraleigh · 2024-04-04T14:32:47Z

cc @josefree

maririos · 2024-04-04T18:40:08Z

FYI @ladonnaq

weshaggard

Left some feedback but looks good to me overall.

To unblock the PR, as rkmanda doesn't have time to review it. I'll incorporate any feedback in a follow-up PR.

rkmanda · 2024-04-11T06:05:10Z

+
+For example, [`specification/containerservice`] is a `service group` for both `aks` and `fleet` services.
+
+The doc for `aks` is [Azure Kubernetes Service]. It points to aks REST reference e.g. for [API version `2024-01-01`][aks REST reference 2024-01-01],


doc for aks is [Azure Kubernetes Service].

will be good to have a link to the doc here

It is indeed hyperlinked. I am using markdown reference links. It links to https://learn.microsoft.com/en-us/azure/aks/

mikekistler · 2024-04-11T12:49:25Z

+
+Each `README.md` describes a single `service` and is used as an SDK package and documentation for each version of the service.
+Inside the `README.md` file there are lists of paths to OpenAPI spec `.json` files making up given service version.
+


I think we should describe specific requirements on the README.md here. For example, it should contain a "default" tag that refers to "current" version of the service (the ref docs depend on this).

To avoid duplication I wanted to keep this section lightweight. But perhaps we should add it to https://aka.ms/azsdk/autorest.

Note this doc mentions:

Both the /resource-manager and /data-plane folders must contain an AutoRest configuration file, README.md. Learn more about this file at aka.ms/azsdk/autorest.

mikekistler · 2024-04-11T12:53:31Z

+
+The `specification` folder is the root folder for all service specifications.
+
+Each child of the `specification` folder corresponds to a `service` specification for given Azure team. Here we denote such folder as `<azureTeam>`.


I think we should change all instances of "azureTeam" to "azureService". We should not "ship our organization structure" in our public API surface, but using "azureTeam" suggests that we do that.

@mikekistler can you consult this with @JeffreyRichter ? My concern is that if we call it azureService then we end up with conflicting terminology where an azureService has inside it a serviceGroup in some cases, which IMO is very counter-intuitive.

konrad-jamrozik mentioned this pull request Mar 26, 2024

[TypeSpec Requirement] Fail if spec contains OpenAPI generated from TypeSpec, but no tspconfig.yaml #28444

Merged

konrad-jamrozik requested review from JeffreyRichter, maririos, mikeharder, mikekistler and weshaggard March 26, 2024 20:55

konrad-jamrozik commented Mar 26, 2024

View reviewed changes

Comment thread documentation/directory-structure.md Outdated

konrad-jamrozik commented Mar 26, 2024

View reviewed changes

Comment thread README.md

konrad-jamrozik added the Central-EngSys This issue is owned by the Engineering System team. label Mar 26, 2024

maririos approved these changes Mar 26, 2024

View reviewed changes

rkmanda previously requested changes Mar 27, 2024

View reviewed changes

rkmanda reviewed Mar 27, 2024

View reviewed changes

Comment thread documentation/directory-structure.md Outdated

konrad-jamrozik mentioned this pull request Mar 27, 2024

Enforce unified API versions in RP specs #28466

Open

konrad-jamrozik commented Mar 27, 2024

View reviewed changes

Comment thread README.md

weshaggard reviewed Mar 27, 2024

View reviewed changes

Comment thread documentation/directory-structure.md Outdated

Konrad Jamrozik added 9 commits April 3, 2024 14:17

updating directory structure

b787685

ongoing

b792702

ongoing

89eb49f

ongoing

83e4992

remove extra empty line

21821e2

remove empty lines

4481b43

reorder references

7e22c8e

update

886e055

add resource provider list

6f4513c

Add missing period

85bd310

konrad-jamrozik mentioned this pull request Apr 4, 2024

Avocado-related: block PRs on violations of non-uniform version of SDK package in README.md Azure/azure-sdk-tools#8023

Open

maririos approved these changes Apr 4, 2024

View reviewed changes

maririos reviewed Apr 4, 2024

View reviewed changes

Comment thread documentation/directory-structure.md Outdated

konrad-jamrozik mentioned this pull request Apr 4, 2024

Doc update: API versions and branches reference Azure/azure-sdk-tools#7915

Closed

Update directory-structure.md: add short link

ee5ac33

konrad-jamrozik mentioned this pull request Apr 4, 2024

Documentation: Deduplicate specs directory structure (folder layout) guidance Azure/azure-sdk-tools#7628

Closed

weshaggard reviewed Apr 4, 2024

View reviewed changes

Comment thread documentation/directory-structure.md Outdated

weshaggard reviewed Apr 4, 2024

View reviewed changes

Comment thread documentation/directory-structure.md Outdated

weshaggard reviewed Apr 4, 2024

View reviewed changes

Comment thread documentation/directory-structure.md

weshaggard approved these changes Apr 4, 2024

View reviewed changes

Merge branch 'main' into users/kojamroz/dir_struct

229e663

konrad-jamrozik mentioned this pull request Apr 5, 2024

Add some info about documentation/samplefiles* and documentation/api-scenario #28601

Merged

Konrad Jamrozik added 3 commits April 5, 2024 13:23

apply review remarks

1add6f2

add mention of service groups in spec folder section

28b960c

minor fixups

b072d2b

konrad-jamrozik merged commit 835b124 into main Apr 5, 2024

konrad-jamrozik deleted the users/kojamroz/dir_struct branch April 5, 2024 21:50

rkmanda reviewed Apr 11, 2024

View reviewed changes

Comment thread documentation/directory-structure.md

mikekistler reviewed Apr 11, 2024

View reviewed changes

Comment thread documentation/directory-structure.md

mikekistler reviewed Apr 11, 2024

View reviewed changes

wiboris pushed a commit to wiboris/azure-rest-api-specs that referenced this pull request May 7, 2024

Update the specification directory structure article (Azure#28464)

0602b69


		For example, [`specification/containerservice`] is a `service group` for both `aks` and `fleet` services.

		The doc for `aks` is [Azure Kubernetes Service]. It points to aks REST reference e.g. for [API version `2024-01-01`][aks REST reference 2024-01-01],


		Each `README.md` describes a single `service` and is used as an SDK package and documentation for each version of the service.
		Inside the `README.md` file there are lists of paths to OpenAPI spec `.json` files making up given service version.


		The `specification` folder is the root folder for all service specifications.

		Each child of the `specification` folder corresponds to a `service` specification for given Azure team. Here we denote such folder as `<azureTeam>`.

Conversation

konrad-jamrozik commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openapi-pipeline-app Bot commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Next Steps to Merge

Uh oh!

openapi-pipeline-app Bot commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Swagger Validation Report

Uh oh!

openapi-pipeline-app Bot commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Swagger Generation Artifacts

Uh oh!

openapi-pipeline-app Bot commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

konrad-jamrozik Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rkmanda Mar 27, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maririos Mar 26, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

rkmanda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

konrad-jamrozik commented Mar 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lfraleigh commented Apr 4, 2024

Uh oh!

maririos commented Apr 4, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

weshaggard left a comment

Choose a reason for hiding this comment

Uh oh!

rkmanda Apr 11, 2024

Choose a reason for hiding this comment

Uh oh!

konrad-jamrozik Apr 15, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mikekistler Apr 11, 2024

Choose a reason for hiding this comment

Uh oh!

konrad-jamrozik Apr 15, 2024

Choose a reason for hiding this comment

Uh oh!

mikekistler Apr 11, 2024

Choose a reason for hiding this comment

Uh oh!

konrad-jamrozik Apr 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

konrad-jamrozik commented Mar 26, 2024 •

edited

Loading

openapi-pipeline-app Bot commented Mar 26, 2024 •

edited

Loading

openapi-pipeline-app Bot commented Mar 26, 2024 •

edited

Loading

openapi-pipeline-app Bot commented Mar 26, 2024 •

edited

Loading

openapi-pipeline-app Bot commented Mar 26, 2024 •

edited

Loading

konrad-jamrozik Mar 26, 2024 •

edited

Loading

konrad-jamrozik commented Mar 27, 2024 •

edited

Loading

konrad-jamrozik Apr 15, 2024 •

edited

Loading