Reorganization of some top level modules

[sphuber]

Fixes #2311

There are a few top-level modules whose purpose was not fully clear and
partially as well as partially overlapping, which caused the content to
be quite unorganized and scattered. Here we reorganize these modules
after having clearly defined their purpose:

• aiida.common: for generic utility functions and data structures
• aiida.manage: for code that manages an AiiDA instance
• aiida.tools: for code that interfaces with third-party libraries

The module aiida.control has been removed. Code that interacted with
internal components have been moved into cmdline or manage where
applicable. The aiida.control.postgres has been moved to the
aiida.manage.external module which is reserved for code that have to
interact with system wide components, such as the postgres database.

The module aiida.utils has been removed. Generic functionality has
been placed in aiida.common and more specific functionality has been
placed in the appropriate sub folders.

[sphuber]

The only two things not done yet, are the aiida.common.utils.Prettifier class and its related functions, and the aiida.common.orbital module. The latter, together with OrbitalData and ProjectionData are too specific for aiida-core and should be probably be moved to aiida-materials-science plugin. We could do move it to aiida-quantumespresso for the time being, but I am not sure if other plugins also use it.

[coveralls]

Coverage increased (+7.4%) to 68.676% when pulling ad86ad3ae546ed01abdce6a05ef30e8af4d2fd4d on sphuber:fix_2311_reorganization_modules into 14ecf15 on aiidateam:provenance_redesign.

[coveralls]

Coverage increased (+7.4%) to 68.676% when pulling ad86ad3ae546ed01abdce6a05ef30e8af4d2fd4d on sphuber:fix_2311_reorganization_modules into 14ecf15 on aiidateam:provenance_redesign.

[coveralls]

Coverage increased (+7.4%) to 68.711% when pulling 8249edd on sphuber:fix_2311_reorganization_modules into 14ecf15 on aiidateam:provenance_redesign.

[ltalirz]

Wow, thanks a lot @sphuber !
I'll look through this tomorrow, ok?

[giovannipizzi]

Thanks a lot @sphuber !!
For aiida.common.orbital, I don't think we should move it to aiida-quantumespresso - it was implemented there on purpose to be generic, and it's also used by at least aiida-wannier90.

I would suggest to either keep it there or move to to a folder that we specifically use for data extensions (I don't know if it really belongs to tools - we could still have a subfolder in some aiida.xxx package to keep these things) until we decide if we want to open this repository for materials science. Or just keep it there for now and put a comment in the issue for the materials science repo for data (I've the feeling there's already an open issue).

For the prettifier, it's used by the kpoints/bands exporters, but it could still be general - also in this case maybe it's ok to keep it where it is?

For the rest, I let @ltalirz accept it - let me know if you also want me to give a look. Is there already a wiki page or somethings similar explaining what the folders are for?

[sphuber]

@giovannipizzi agreed on the OrbitalData and ProjectionData, but is the aiida.common.orbital module really necessary in aiida-core. As far as I can tell it only defines RealHydrogenOrbital as a subclass of Orbital, but it is not used anywhere. Is this used in wannier90? I see that aiida-quantumespresso uses these classes in the projwfc parser

[ltalirz]

Is there already a wiki page or somethings similar explaining what the folders are for?

We now documented the intended content of the folders in the respective __init__.pys.
I think that's already a big step; one could reuse these docstrings via automodule to have them also in the documentation, somewhere near the forward compatibility guarantees #2313

[ltalirz]

In case we need to make changes for pre-commit, let me know and I'll re-approve