profile bot runs

[beckermr]

We should profile the run of the bot code to see where we are spending most of our time.

[viniciusdc]

Hi, what bout using cProfile to profile the information of the run and save it somewhere to check it locally with KCacheGrind (or gprof2dot)? This way we can see besides the actual time the current workflow of the calls. It will be also possible to create an interactive visualization at the web services using Django and Chart.js with that information.

[viniciusdc]

OK, besides just to check: when you say the run of the bot code you mean the run function or all the run_bot process? (I should've said it before)

[beckermr]

all of the code that makes prs

[viniciusdc]

all of the code that makes prs

I think that is just update_PRs, right ? The others at run and make_graph just load some informations. (We actually have something at the audits, if I remember )

[beckermr]

sounds right yes!

[CJ-Wright]

I think the main thing to profile is all the code in auto_tick.py since this is where the PRs are made (and the migrations are actually performed).

[viniciusdc]

I think the main thing to profile is all the code in auto_tick.py since this is where the PRs are made (and the migrations are actually performed).

Yup, you are alright I thought about that but I was not sure if we would profile the migration too, thanks for the comment.

[CJ-Wright]

I would profile everything, we need a better view of what is slow so we can fix it.

[viniciusdc]

What do you think ? #1071

[beckermr]

i'd be great to see the profile data.

[viniciusdc]

i'd be great to see the profile data.

Inside the client logs ?

[beckermr]

the profiler outputs data on what is taking time. i'd be great to have us all look at that data together.

[viniciusdc]

Yup, I agree with you, but I don't get your point, the code is supposed to output the data into a file and them use another tool to see it's information more graphically, but if it's needed to output the data at the terminal window it's also possible. The idea to output the data, was cause this way we can monitor all the function calls and times in a graph, to compare the using memory and other informations

[beckermr]

yup. So can you run the profiler on the code and attach the data here?