Skip to content

Ported job calculation to use coroutines for tasks#1827

Merged
sphuber merged 1 commit into
aiidateam:developfrom
muhrin:transport_yield
Jul 31, 2018
Merged

Ported job calculation to use coroutines for tasks#1827
sphuber merged 1 commit into
aiidateam:developfrom
muhrin:transport_yield

Conversation

@muhrin
Copy link
Copy Markdown
Contributor

@muhrin muhrin commented Jul 31, 2018

Fixes #1828

This way the event loop has a chance to schedule other operations while
a transport request is made i.e. if multiple job calculations all
request the same transport (a common scenario) then in the past they
would all run as one loop callback effectively blocking the loop (and
crucially pika heartbeats). Now they are schedule as seperate callbacks
thanks to yield, hopefully, allowing the loop to schedule other
necessary tasks in between.

This way the event loop has a chance to schedule other operations while
a transport request is made i.e. if multiple job calculations all
request the same transport (a common scenario) then in the past they
would all run as one loop callback effectively blocking the loop (and
crucially pika heartbeats).  Now they are schedule as seperate callbacks
thanks to yield, hopefully, allowing the loop to schedule other
necessary tasks in between.
@muhrin muhrin requested a review from sphuber July 31, 2018 11:00
@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #1827 into develop will increase coverage by 8.52%.
The diff coverage is 87.75%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1827      +/-   ##
===========================================
+ Coverage    58.16%   66.68%   +8.52%     
===========================================
  Files          317      317              
  Lines        32394    32409      +15     
===========================================
+ Hits         18843    21613    +2770     
+ Misses       13551    10796    -2755
Impacted Files Coverage Δ
aiida/transport/transport.py 62.2% <66.66%> (+0.36%) ⬆️
aiida/work/transports.py 90.56% <89.13%> (+45.56%) ⬆️
aiida/orm/querybuilder.py 74.16% <0%> (+0.24%) ⬆️
aiida/common/setup.py 74.63% <0%> (+0.41%) ⬆️
aiida/control/computer.py 80.9% <0%> (+0.9%) ⬆️
aiida/orm/implementation/general/node.py 78.5% <0%> (+0.95%) ⬆️
aiida/orm/implementation/general/group.py 60.82% <0%> (+1.03%) ⬆️
aiida/common/hashing.py 83.87% <0%> (+1.07%) ⬆️
aiida/orm/implementation/django/authinfo.py 82.85% <0%> (+1.42%) ⬆️
... and 42 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7c552b1...2c40178. Read the comment docs.

Copy link
Copy Markdown
Contributor

@sphuber sphuber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge transport request granted

@sphuber sphuber merged commit ca2778f into aiidateam:develop Jul 31, 2018
@muhrin muhrin deleted the transport_yield branch September 3, 2018 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants