Make Groups great again by yakutovicha · Pull Request #2329 · aiidateam/aiida-core

yakutovicha · 2018-12-07T23:51:31Z

Fixes #2075 and fixed #160

aiida.orm.importexport
- While importing data from the export file allow to specify user-defined
  group and to put all the imported data in this group
aiida.common.utils
- Remove get_group_type_mapping function which was mapping
  machine-specific group names with the user-friendly ones
- Add escape_for_sql_like that escapes % or _ symbols provided by user
aiida.orm.groups
- Add GroupTypeString enum which contains all allowed group types:
  data.upf (was data.upf.family)
  auto.import (was aiida.import)
  auto.run (was autogroup.run)
  user (was empty string)
- Remove Group.query and Group.group_query methods, as they are redundant
aiida.orm.data.upf:
- Set UPFGROUP_TYPE to GroupTypeString.UPFGROUP_TYPE
- Replace the usage of Group.query by QueryBuilder in get_upf_groups
  and get_upf_family_names methods
aiida.orm.autogroup:
- set VERDIAUTOGROUP_TYPE to GroupTypeString.VERDIAUTOGROUP_TYPE
aiida.cmdline.commands.cmd_group
- Add verdi group copy
- Add option to show all available group types
- Add defaulf for group_type option
- Replace Group.query with QueryBuilder in verdi group list
- Remove usage of the get_group_type_mapping() function
aiida.cmdline.commands.cmd_import
- Add the possibility to define a group that will contain all imported nodes.
aiida.cmdline.params.types.group
- Add the possibility to the GroupParamType to create groups if they don't exist
aiida.backend*:
- Rename type and name to type_string and label for the database models
Improve documentation for django and sqla backends migration

yakutovicha · 2018-12-07T23:55:16Z

Also fixes #2075.

Group documentation is still missing

yakutovicha · 2018-12-08T14:21:15Z

This commit should also fix the ambiguity of Group type as reported in #160 since we now rename type with type_string

yakutovicha · 2018-12-08T14:29:42Z

@giovannipizzi I looked at all Group-related issues and commented them in this PR. What is still missing is:

add group documentation. Btw @ltalirz what is the right place for it? Looks like now it is quite significantly reshaped.
Fix all the tests that fail because of the group name->label and type->type_string change

ltalirz · 2018-12-09T20:57:18Z

@yakutovicha I suggest to add a section Grouping Data under Working with AiiDA that explains how one should use groups, labels, etc. in AiiDA.

Btw, you can now inspect the docs of the provenance-redesign branch here:
https://aiida-core.readthedocs.io/en/provenance_redesign/

yakutovicha · 2018-12-10T15:20:21Z

@yakutovicha I suggest to add a section Grouping Data under Working with AiiDA that explains how one should use groups, labels, etc. in AiiDA.

Btw, you can now inspect the docs of the provenance-redesign branch here:
https://aiida-core.readthedocs.io/en/provenance_redesign/

thanks @ltalirz

yakutovicha · 2018-12-10T23:25:38Z

As we discussed today with @giovannipizzi I should also:

add name and type as deprecated options to ensure the back-compatibility

coveralls · 2018-12-11T09:20:21Z

Coverage increased (+0.005%) to 68.191% when pulling 6c525c8 on yakutovicha:provenance_redesign into f9b9b8b on aiidateam:provenance_redesign.

yakutovicha · 2018-12-11T17:30:55Z

thanks @sphuber, your trick with auto groups did work

yakutovicha · 2018-12-11T20:22:51Z

I think it is now done. @sphuber would you like to review the PR? I am not sure whether @giovannipizzi will have time for that.

yakutovicha · 2018-12-11T21:36:56Z

I am not sure why travis fails from time to time. I restarted the test and everything went fine.

sphuber · 2018-12-12T13:14:07Z

@yakutovicha we merged migrations yesterday, so there will be conflicts if we update the branch. Please rebase your branch on the current provenance_redesign and squash your commits, then I will have a look. Thanks

yakutovicha · 2018-12-12T20:40:50Z

@yakutovicha we merged migrations yesterday, so there will be conflicts if we update the branch. Please rebase your branch on the current provenance_redesign and squash your commits, then I will have a look. Thanks

it is done now

sphuber

Very nice work @yakutovicha , I have a few comments, but should not take too long to address them

1) aiida.orm.importexport: while importing data from the export file allow to specify user-defined group and to put all the imported data in this group 2) aiida.common.utils: a) remove get_group_type_mapping function which was mapping machine-specific group names with the user-friendly ones 3) aiida.orm.groups a) Add GroupTypeString (Enum) which contains all allowed group types: data.upf (previously was 'data.upf.family') , auto.import (previously was 'aiida.import'), auto.run (previously was 'autogroup.run'), user (previously was '') b) Remove Group.query and Group.group_query functions, as they are redundant. Their functionality can be replaced by QueryBuilder() 3) aiida.orm.data.upf: a) Set UPFGROUP_TYPE from GroupTypeString.UPFGROUP_TYPE.value defined in aiida.common.utils b) Replace the usage of Group.query() by QueryBuilder() in get_upf_groups and get_upf_family_names functions 4) aiida.orm.autogroup: set VERDIAUTOGROUP_TYPE from GroupTypeString.VERDIAUTOGROUP_TYPE.value defined in aiida.common.utils 5) aiida.common.utils: add escape_for_sql_like that escapes % or _ symbols provided by user 6) aiida.cmdline.commands.cmd_group.py a) Add copy option b) "list": replace Group.query() with QueryBuilder c) Add option to show all available group types d) Add defaulf for group_type option e) remove usage of the get_group_type_mapping() function 7) aiida.cmdline.commands.cmd_import.py: add a posibility to chose the group where all the imported nodes will be put in. 8) aiida.cmdline.params.types.group.py add a possibility to the group parameter to create new group 9) dj and sqla backends: in Group (and tests, resapi) replace name with label and type with type_string 10) Add Groups documentation 11) Keep name and type for back-compatibility 12) Improve documentation for django and sqla backends migration

yakutovicha · 2018-12-13T16:40:13Z

@sphuber I implemented your suggestions, please review it again.

yakutovicha requested a review from giovannipizzi December 7, 2018 23:51

sphuber requested changes Dec 13, 2018

View reviewed changes

yakutovicha added 2 commits December 13, 2018 15:00

Groups: enforce type_string to be of GroupTypeString type

6c525c8

sphuber approved these changes Dec 13, 2018

View reviewed changes

sphuber merged commit 65dfecb into aiidateam:provenance_redesign Dec 13, 2018

This was referenced Dec 13, 2018

Option to add all groups to node of choosing on verdi import #2075

Closed

Ambiguous role of column 'type' in DbGroup #160

Closed

Conversation

yakutovicha commented Dec 7, 2018 • edited by sphuber Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yakutovicha commented Dec 7, 2018

Uh oh!

yakutovicha commented Dec 8, 2018

Uh oh!

yakutovicha commented Dec 8, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ltalirz commented Dec 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yakutovicha commented Dec 10, 2018

Uh oh!

yakutovicha commented Dec 10, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coveralls commented Dec 11, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yakutovicha commented Dec 11, 2018

Uh oh!

yakutovicha commented Dec 11, 2018

Uh oh!

yakutovicha commented Dec 11, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sphuber commented Dec 12, 2018

Uh oh!

yakutovicha commented Dec 12, 2018

Uh oh!

sphuber left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yakutovicha commented Dec 13, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

yakutovicha commented Dec 7, 2018 •

edited by sphuber

Loading

yakutovicha commented Dec 8, 2018 •

edited

Loading

ltalirz commented Dec 9, 2018 •

edited

Loading

yakutovicha commented Dec 10, 2018 •

edited

Loading

coveralls commented Dec 11, 2018 •

edited

Loading

yakutovicha commented Dec 11, 2018 •

edited

Loading